Physical Security: The Electric Grid’s Dirty Little Secret
Physical security requirements for the electric grid—and their enforcement—are largely non-existent 6 years after the Metcalf attack
At approximately 1:00 a.m. on April 16, 2013, a major PG&E transformer substation in Metcalf California was attacked. The attack was well-planned and sophisticated. One year later, the Metcalf station was struck again when the fence was cut open and, the facility entered and tools were stolen.
Obviously, the physical security situation had not improved much in the intervening year. In fact, PG&E’s credibility was shot when its public statements about its physical security improvements were contradicted by a leaked internal memo.
The April 2013 Metcalf attack was not the only physical attack on critical components of the North American electric grid:
- On June 11, 2014 there was an attack by Improvised Explosive Device (IED) on the Nogales Station in Arizona.
- On December 4, 2014, Hydro-Québec suffered a power outage (and narrowly avoided a province-wide blackout) when a pilot attacked the grid by airplane interrupting the flow of electricity to the United States.
- On September 25, 2016 there was a rifle attack on the Buckskin Substation in Utah.
However, the attack on the Metcalf substation—and the other attacks—shouldn’t have been a surprise. A year before the Metcalf attack, the National Academies published a report titled: Terrorism and the Electric Power Delivery System. The report discussed physical security of high-voltage transformers noting:
High-voltage transformers are of particular concern because they are vulnerable to attack, both from within and from outside the substation where they are located. These transformers are very large, difficult to move, custom-built, and difficult to replace. Most are no longer made in the United States, and the delivery time for new ones can run to months or years.
Then, one year after the Metcalf attack, the Wall Street Journal ran two alarming stories:
Assault on California Power Station Raises Alarm on Potential for Terrorism. April Sniper Attack Knocked Out Substation, Raises Concern for Country’s Power Grid
Smith, Rebecca. Wall Street Journal. February 5, 2014
U.S. Risks National Blackout From Small-Scale Attack. Federal Analysis Says Sabotage of Nine Key Substations Is Sufficient for Broad Outage
Smith, Rebecca. Wall Street Journal. March 12, 2014
Huge physical security vulnerabilities were identified—so what was done?
(Spoiler Alert: The answer is “Goose Egg”)
One would think that action would be taken. At first, a lot of paper flew. After the February 5, 2014 Wall Street Journal article, the Senate sent a letter on February 7, 2014 to the Federal Energy Regulatory Commission (FERC), to ask them what they were doing to protect the grid. And FERC Responded telling the Senate that:
“Since the attack on the Metcalf facility in April 2013, the Commission’s staff has taken responsive action together with NERC, other federal and state agencies, and transmission and generation asset owners and operators.”
So we are okay. right? Action has been taken?
Nope.
The physical security of our critical transformers and facilities remains a complete mess in 2019.
Problem #1: The standard—CIP-014-2 (Physical Security)—is a joke.
As a result of Metcalf, FERC ordered NERC to develop a physical security standard. Yes, that’s right—the industry is self-regulated and writes their own standards. NERC submitted their proposed standard (known as CIP-014-1) on May 23, 2014.
FERC issued an order on November 20, 2014 literally ordering NERC to change one word. (The word was: “widespread” and was used 30 times in the proposed standard. This word—a slight of pen by NERC’s attorneys—would have excluded many facilities from falling under the standard.)
On October 2, 2015, FERC approved the “Physical Security” standard, known as CIP-014-2. What does the physical security standard require? Well, it requires very little:
- Requirement 1: Each Transmission Owner shall perform a risk assessments of its Transmission stations and Transmission substations.
- Requirement 2: Each Transmission Owner shall have an unaffiliated third party verify the risk assessment [e.g., a peer grid company would meet the requirement—”Hey, I’ll show you mine if you show me yours”].
- Requirement 3: If a Transmission Owner operationally controls an identified Transmission station or Transmission substation, it must notify the Transmission Operator that has operational control of the primary control center.
- Requirement 4: Each Transmission Owner shall conduct an evaluation of the potential threats and vulnerabilities of a physical attack to each of their respective Transmission station(s), Transmission substation(s), and primary control center(s).
- Requirement 5: Each Transmission Owner shall develop and implement a documented physical security plan(s) that covers their respective Transmission station(s), Transmission substation(s), and primary control center(s).
- Requirement 6: Each Transmission Owner shall have an unaffiliated third party review the evaluation performed under Requirement R4 and the security plan(s) under Requirement R5 [e.g., a peer grid company would meet the requirement—”Hey, I’ll show you mine if you show me yours”].
That’s it. All you have to do is have a binder with a bunch of papers labeled “Physical Security Plan” and have anybody you choose review your “risk assessment,” “evaluation” and “security plan(s)”. No need for it to be anybody who knows anything about physical security.
And there is no requirement as to what the “Physical Security Plan” must include—or even that it be effective. Nobody with regulatory authority even has to even approve it—All you need is somebody to “review” it. What if the “reviewer” happens to say “this plan sucks?” It doesn’t matter. The only requirement is that the three ring-binder be “reviewed.” Check! I guess most any papers in a three-ring binder will do!
That unapproved three-ring binder of papers is what is standing between your family and a widespread power outage caused by a terrorist attack.
Oh, did you notice that generation plants (that’s where electricity is made) are not included in NERC’s physical security standard? Commissioner Cheryl LaFleur even testified before Congress that an attack on a single generation plant can cause a cascading outage.
Moreover, in her May 20, 2014 re-nomination hearing, Congress asked Commissioner LaFleur: “will the physical security standard recently passed by NERC adequately protect the public from electric grid outage caused by terrorist attack?” Her written answer:
NERC’s petition to approve the physical security standard was filed with the Commission for review on May 23, 2014. It would be inappropriate for me to judge the merits before interested parties have an opportunity to submit comments to the Commission, so that we can consider all relevant arguments. I assure you that I will carefully consider the proposal and all filed comments to ensure that NERC’s filing does adequately protect the public.
A bureaucratic non-answer. Now after almost 9 years of Commissioner LaFleur’s “leadership”—and six years after the Metcalf attack—the public is not “adequately” protected.
(But let us not get “into the weeds” on physical security, as this can be truly upsetting, especially for those dependent on reliable electricity—intensive care patients in hospitals, those dependent on kidney dialysis, diabetics needing insulin refrigeration, etc. FERC has told us “responsive action” has been taken and that’s all we need to know.)
Problem #2: Enforcement of CIP-014-2 seems nonexistent
Okay. Even if the physical security standard is bull shit, it is better than nothing right? If all of the companies at least are doing this, it makes us a little safer, right? As long as NERC is enforcing this standard, we are at least a little safer, right?
NERC must be all over this in its audits of the utilities, right?
So, how many times since Metcalf have utilities been cited for violations of standard CIP-014-2?
Four.
That’s right. I’m not golfing. I mean “four” as in the numeral. We have had hundreds of physical attacks (that the public knows about) yet, the standard has only been cited four (4) times in the six (6) years since the Metcalf attack. What does that mean?
It means one of three things:
- Either the electric utility industry totally got its “sierra” together after Metcalf and all of our transformers are secure and NERC can find no violations. (Note: That would also require that they are actually assessing the effectiveness of the plans, which the standard does not require but this would assume that NERC and all 1,500 regulated entities are going way above and beyond the call of the standard that they wrote.)
- Or all the companies have a three-ringed binder marked “Physical Security Plan.”
- Or NERC and the Regional Entities are pulling a “Sergeant Schultz” and just not looking. (“I see nothing…I know nothing!”)
Unless what is actually happening is #1 – then this standard and regulatory scheme are not working. Here are the facts.
- There are 1,500 entities regulated by NERC.
- There are likely over 2000 EHV LPTs (Extra High Voltage Large Power Transformers) in the United States and tens of thousands of LPTs.
- There have been 4 citations for non-compliance with the BS physical security “standards” since Metcalf.
- The American people are not stupid. We see these transformers unguarded behind the chain-link fence as we drive up the road or walk our dogs.
So how seriously does NERC take physical security? Not very judging by their lack of effort to update their website.
Here is a screenshot of NERC’s website on “Physical Security” taken on April 20, 2019. It is talking about CIP-014-1. This standard has been outdated since October 2, 2015.
NERC’s physical security website has not been updated in 3 and 1/2 years. What does that tell us? I guess #1 was a long shot anyway.
So let’s take a look at the 4 times NERC found CIP-014-2 violations:
- In NP19-4-000 (one Violation—which everybody knows is Duke Energy Corp.), Duke apparently excluded one substation from its risk assessment because they didn’t think it met the criteria for inclusion.
- In NP18-14-000 (one violation), the “Unidentified Registered Entity” failed to do a risk assessment on one substation due to a “management oopsy.”
- And in NP17-29-000 (two violations), the “Unidentified Registered Entity” failed to include one control center in it’s 1) risk assessment and 2) security plan (two violations) because an employee who knew what they were doing left the company, leaving nobody else who knew what they were doing.
You will notice that all 4 of these “violations” are administrative in nature and have nothing to do with whether there is actually meaningful physical security in place.
That’s it for NERC Physical Security enforcement since Metcalf!
A more detailed history of “Physical Security” standards
At the risk of getting a bit geeky, a recitation of the history of the physical security and sabotage reporting standards is instructive.
CIP-001-1 (Sabotage Reporting) became effective on June 4, 2007. It was cited 404 times between 6/4/2008 and 5/26/2011. It them morphed into CIP-001-1a (February 2, 2011) and CIP-001-2a (August 2, 2011)—neither of which were EVER cited.
Meanwhile, EOP-004-1 (Disturbance Reporting), which covered “equipment damage” among other things, was cited 16 times between 2009 and 2013.
NERC began to look at merging CIP-001 and EOP-004 “to eliminate redundancies” and on June 20, 2013, FERC approved merging CIP-001-2a (Sabotage Reporting) and EOP-004-1 (Disturbance Reporting) into EOP-004-2 (Event Reporting). (CIP-001-2a Sabotage Reporting and EOP-004-1 Disturbance Reporting were then “Retired.”) EOP-004-2 covers reporting “damage or destruction of a facility.” EOP-004-2 and its successors have never been cited.
So here is the enforcement history of these various standards:
- 404 Citations issued for CIP-001-1 (Sabotage Reporting) between 2008 and 2011
- 16 Citations were issued for EOP-004-1 (Disturbance Reporting) between 2009 and 2013—not all related to damage.
Metcalf happened on April 16, 2013, but then…
- No citations have been issued for EOP-004-2 (effective June 20, 2013)
- No citations have been issued for EOP-004-3 (effective November 19, 2015)
- No citations have been issued for EOP-004-4 (effective January 18, 2018)
And adding in the CIP-014 physical security Standard:
- No citations have been issued for CIP-014-1
- 4 citations have been issued for CIP-014-2
- NP19-4-000 (one Violation)
- NP18-14-000 (one violation)
- NP17-29-000 (two violations)
There are Solutions
The Department of Energy as well as many commercial companies offer various solutions to defeat ballistic, explosive and electromagnetic threats. Here are just a few, These are not endorsements, simply evidence that physical security solutions exist:
- Idaho National Laboratory – Armor The Grid
- Metalex – How to Protect the Grid with Stronger Security Fencing
- Siemens – Bullet Resistant Power Transformers
- BTI – Ballistics Transformer Protection
- ArmorCore – Used For Securing Nation’s Grid
- Durasystems – Barriers
- Southern States LLC – Physically Securing Substations
The military has been doing physical security of critical facilities longer than anybody. Since the military is dependent on the civilian electric grid, DoD has a dog in the fight and expertise but no actual authority to do anything. Similarly, DHS and DOE have expertise and resources—and also have a stake in grid security—but no direct authority. (Although DOE does have authority in an emergency under the FAST Act—and arguably this is an emergency.) FERC is the key player here.
I wonder what we would find if FERC got a physical security “Red Team” together? What if FERC hired some retired Army Green Berets, Navy SEALs and Marine Raiders to check these three-ring binders?
Conclusion
Physical security for the electric grid still appears largely non-existent 6 years after Metcalf attack. The standard is weak and the enforcement seems absent.
There is little public evidence that anything substantial has been done since Metcalf to secure our critical transformers and control centers. The standard doesn’t require anything other than a peer-reviewed risk assessment, evaluation and a three-ring binder labeled “Physical Security Plan” (which needs not be approved by anyone who knows what they are doing—just “reviewed” by somebody the utility chooses).
The first step is that FERC needs to coordinate with DoD, DOE and DHS to “Red Team” the electric utilities on physical security. (The same “Red Team” concept would work for cybersecurity, EMP and GMD hardening as well.) We should help the willing to fix themselves and we should “Black Hat” regulate the repeat CIP standard violators. NERC should not be involved. This needs to be a government verification that the industry’s self-regulation of a critical infrastructure is working.
Anything less than immediate action by FERC to evaluate the physical security of the electric grid—Including NERC’s “regulation” of the standard—is unacceptable.
###
References:
Senate Metcalf Letter Dated February 7, 2014, Regarding Physical Attack on Metcalf Substation
- Click for PDF copy of February 7, 2014 letter from Senate to FERC
- Click for PDF copy of February 11, 2014 letter from FERC to Senate
- Click for February 12, 2014 Statement of Acting Chairman Cheryl A. LaFleur
Metcalf Substation Attack, California (4/16/2013)
- Assault on California Power Station Raises Alarm on Potential for Terrorism
- VIDEO: Metcalf Sniper Attack – Wall Street Journal
- Metcalf Attack: NBC Reports on PG&E Security Memo
- Snipers Coordinated an Attack on the Power Grid, but Why?
- Sophisticated but low-tech power grid attack baffles authorities
- Sniper attack on California power grid may have been ‘an insider,’ DHS says
Hydro-Québec Grid Attack (12/4/2014)
- Pilot to be sentenced in sabotage that crippled Quebec power grid
- Pilot gets 7 years in prison for attacking Hydro-Québec network
- ‘Pilot to the stars’ nearly crippled entire Hydro-Québec network
Noglaes Station Attack (6/11/2014)
- Arizona substation attacked with bomb
- Sabotage at Nogales station puts focus on threats to grid
- Concern widens over sabotage at Nogales power station
Buckskin Substation Attack, Utah (9/5/2015)
- Power company offers rare $50K reward for information on vandalism
- Substation attack is new evidence of grid vulnerability
- Sniper attack on Utah substation highlights grid vulnerability
[wpedon id=”5868″ align=”center”]