SANS Industrial Control Systems Security Blog

One CIP, Two CIP, Red CIP, Blue CIP

This blog was written by - Tim Conway with contributions, edits, and research from Ted Gutierrez and Kevin Perry

 

9188511-web-Military-Binoculars Looking at the Ukraine cyber-attacks through the various lenses of NERC CIP

Following the cyber-attacks which impacted the Ukrainian electric system on December 23, 2015 there were a number of public statements and discussions asking if the North American electric system was susceptible to similar targeted cyber-attacks and the impact such an attack would have. Now the electric sector is again facing more questions following the CBS news report on December 21, 2016, of yet another possible cyber-attack on the Ukrainian electric system and statements in the report suggesting that a targeted US electric grid attack would be even worse to those experienced in the Ukraine.

Similar to the discussions of a year ago, it is likely that two camps will emerge with numerous variations of familiar themes:

Position 1) This type of an attack would not work against the US system, due to the protections in place under the NERC CIP regulations and reliability focused investments over the past decades.

Position 2) The US electric system is more susceptible as it is highly automated and would suffer a longer outage period if manual restoration efforts were required.

The reality is more complicated than either of these positions and must take into consideration multiple variables, and like all things associated to NERC CIP, you must start the debate by saying "it depends." I will focus on providing some thoughts to consider when discussing these two positions.

 

9188714-web-Globe-Earth The CIP that saved the world (aka Position 1)

The North American Electric Reliability Corporation (NERC) performs a number of key roles in ensuring the reliability of the electric system. These responsibilities range from standards development, events analysis, compliance audits, penalty enforcement, and operating the Electricity - Information Sharing and Analysis Center (E-ISAC). While NERC has developed many standards focused on the operation and planning of the Bulk Electric System (BES), I will focus only on the NERC Critical Infrastructure Protection (CIP) standards.

The CIP Standards have existed in one form or another for over a decade, with NERC enforcement authority going back to 2007. The CIP standards have been routinely included in the top ten lists of most violated NERC Reliability standards of all time. It indicates the challenges in implementing a balanced CIP program defined by appropriate cybersecurity controls and compliance approaches that capture and demonstrate performance and management of the program. The CIP Standards have changed and matured significantly over time and industry efforts continue to highlight those benefits and the overall strength of the regulation. The topic of shifting from compliance focused standards to a balanced regulation addressing reliability, cybersecurity, and compliance is featured in a November 3, 2015 SANS webcast titled "How NERC and CIP are making a difference". Following the Ukraine event, the topic of NERC CIP effectiveness was covered in a March 24, 2016 blog post, where Ted Gutierrez mapped the CIP controls that may have helped an organization in defending against a similar targeted attack. Later, in a May 25th, 2016 industry meeting Kevin Perry from the SPP Regional Entity delivered a presentation titled: "Would CIP Standards Have Saved the Ukraine". I have attempted to summarize the great work conducted by both individuals in the graphic below.

CIP to UA

In the graphic above, the various adversary elements that were demonstrated during the Ukraine events of December 23, 2015 have been mapped to a mitigating control that is required in the current NERC CIP regulation. Based on this view without any additional context it would appear that CIP would have saved the day or at a minimum would have resulted in forcing the adversaries to utilize different capabilities or attack vectors. Now we will consider some of the nuances of this position that are important to understand during the discussions.

 

 

9188319-web-Watch-Person-Arrow You must be this tall to ride

With the authority granted to NERC under the Federal Power Act, NERC governs the Bulk Power System (or Bulk Electric System (BES) of North America. The definition of BES has lots of inclusion and exclusion elements, however for simplicity consider transmission facilities operating at 100kV and above, generation resources, and Control Centers to be in scope. The lower voltage facilities are generally categorized as distribution level assets in the overall electric system. While the 2015 attacks in the Ukraine did target some substation environments operating at 110kV, the majority of the impacted substations were below 110kV and the way the Electric system is segmented in the country places those facilities under the control of the distribution operator not the bulk power system transmission operator (i.e. essentially the 110 kV circuits impacted were considered distribution-level assets).

The majority of the US electric distribution system (similar voltage level to most of the targeted Ukraine assets in 2015) does not fall under the NERC CIP regulations. The exception to this in North American are specific distribution elements (under frequency load shed, under voltage load shed, remedial action schemes, special protection systems and facilities associated with blackstart resources) that are subject to NERC CIP regulation consideration because of their potential impact on the BES.

For this reason, much confusion arose in discussions surrounding the effectiveness and sufficiency of NERC CIP to prevent a similar attack against the US electric system. If an adversary with similar capabilities selected three target North American organizations and attempted a copycat attack against distribution only level assets, they may never encounter an asset that was subject to the NERC CIP Standards. Just for the sake of clarity, this is not a surprise; the NERC Standards have always focused on protecting the most critical components of the electric system and enforcing requirements based on the Federal authority delegated to the ERO. Most distribution systems fall under state-level authority for any requirements or regulation.

 

 

9189966-web-Audience-Facing-7-People-2 Fifty shades of CIP

The next major nuance to include in the discussion around position 1 is the understanding that you need to specify which NERC CIP you are talking about. During the "old days" of CIP versions 1-3 if an asset was in scope and subject to CIP as an identified Critical Cyber Asset, then it was subject to the majority of the requirements - an everything for everything approach. CIP versions 5/6 introduced the concept of criteria based impact ratings as well as a systematic approach to grouping assets. This new paradigm resulted in the applicability of certain requirements to systems that are at a High impact rating criteria, fewer requirements for those systems at a Medium impact rating, fewer still for those at a Medium impact without external routable connectivity, and the least amount of requirements for those systems categorized as Low impact (again just for the sake of saying it - this is not a surprise. More requirements for the assets that by definition pose the greatest risk to the reliability of the electric system and fewer requirements for assets that pose a lower risk). Therefore, in a discussion about NERC CIP controls and the effectiveness of those controls in mitigating specific attacker capabilities or attack vectors, it is essential to establish an understanding of which CIP you are talking about.

When considering an assessment of the requirement by requirement applicability variations across the High, Medium, and Low impact criteria and what each set of required controls in a CIP program would look like, I considered an exercise we performed in our SANS ICS 456 course. In this exercise we leveraged an established and implemented framework for determining an entity's cyber capabilities and maturity, the ES-C2M2 (Electricity Subsector Cybersecurity Capability Maturity Model). Using this model we ran assessments against the CIP v5/6 Standards themselves, and developed capability and maturity reports for the requirements applicable at a High impact facility, a Medium impact, Medium impact with no ERC, and a Low impact system. These assessment reports help to quickly identify the variations in controls and effectiveness across the version 5/6 CIP programs.

I have provided the resulting ES-C2M2 assessment reports of the CIP Standards for High, Medium, and Low impact programs. At a quick glance you can see which domains have coverage across the Maturity Indicator Levels, and which ones have gaps. The legend in the bottom left of the image indicates whether a control is Fully Implemented, Largely Implemented, Partially Implemented, or Not Implemented. Keep in mind this is an assessment of the Standards themselves, so it is really indicating the degree to which a control is required across the High, Medium, and Low impact ratings of NERC CIP and how that aligns with the MIL 1, 2, and 3 levels of ES-C2M2

 

 

First, the results of the High impact evaluation, including supporting information from the ES-C2M2 reports to help understand the output.

ES High

ES domain

ES MIL

 

 

Next, the Medium Impact with ERC evaluation report:

ES Med

 

 

And now the Medium Impact without ERC:

ES Med no ERC

 

 

And last, the Low Impact evaluation:

ES Low

 

 

Based on the ES-C2M2 evaluations, a couple of things become immediately clear:

1) We definitely need to change our discussions around the protections provided by NERC CIP, as there needs to be more specificity around which CIP you are talking about and something that may have been true in versions 1-3 may no longer be true in versions 5/6.

2) All impact rating evaluations show strong capabilities in the Risk Management, Information Sharing and communications, as well as in Cybersecurity Program Management.

3) All impact rating evaluations indicate weakness in Threat and Vulnerability Management as well as Supply Chain and External Dependencies Management domains.

4) Regardless of associated impact rating level (H, M, or L) there is a tremendous amount of Cyber Assets that are now "in scope" of NERC CIP that previously were not included under entity defined Risk Based Assessment Methodologies. Many of these newly in scope Cyber Assets are at a Low impact level and as such there is a large number of electric entities that now have CIP programs due to the identification of Low Cyber Assets.

5) The ES-C2M2 assessments shown above were performed on the CIP Standards directly. Meaning, the assessment output reports are a "perfect world" evaluation of the requirements not an entity implementation, which may not adequately or completely be performing what the requirement states.

 

 

9189700-web-Earth-Connected-2 Our similarities are greater than our differences (aka Position 2)

This position is largely based on a couple of basic theories;

1) The engineering, architecture, and operation of the US electric system is not tremendously different than the systems found in the Ukraine.

2) The amount of automation, communications inter-dependencies, and associated attack vectors in the US are similar or greater when compared to the impacted Ukrainian sites.

3) The ability to roll vehicles, to dispatch personnel, and to perform manual operations may take longer in the US than it did at the impacted Ukrainian organizations due to the significant centralization efforts of dispatch and field personnel that has taken place here.

4) There are far more potential target facilities and organizations to choose from in the US.

5) With an understanding of an entities protections and controls in place, even at the highest NERC CIP impact level, there is an expectation that an adversary would bring appropriate tools and capabilities to counter those protections and controls to achieve its objectives.

 

 

9189791-web-Cheers-Glasses Why CIP from a sub-requirement when you can gulp from a standard

Personally, I tend to sit in the position 2 group, however I am greatly encouraged by the voluntary efforts that organizations are pursuing to implement CIP beyond the specific applicable requirements. While there is likely not a definitive right or wrong position in this discussion, I believe it is important to make sure all parties engaged in this debate have a firm understanding of the scope and variations that exist amidst the CIP world.

It should also be recognized that a NERC Registered Entity can always choose to implement controls that go beyond what is required within the standards. This approach has been demonstrated by some North American entities who have internally implemented a common CIP Internal controls program across all systems regardless of the individual impact rating requirements for each system.

 

 

9188846-web-Moving-Goal-PostAs CIP continues to grow and expand into areas previously not in scope, I anticipate an ever increasing set of standards that reaches the highest levels of capability and maturity across all domains. As many countries and other sectors look to CIP as a goal, CIP continues to raise the bar and move the goal post to ensure the reliability of the electric system we all rely on.

 

 

*Note - if you are interested in obtaining the ES-C2M2 worksheets developed in the CIP H, M, & L assessments, I will post the files in our ICS forum.

 

Tim Conway

If you're interest in attending ICS456: Essentials for NERC Critical Information Protection, I'm teaching at these upcoming events. http://www.sans.org/u/osf

To view all our upcoming courses and events, click here.

Upcoming SANS ICS Training Opportunity: ICS SUMMIT 2017

  • Choose from 5 ICS world-class security courses
  • Hear from keynote speakers @RobertMLee & @ElectricFork
  • Win your way through our ever-popular ICS Game Night
  • Understand attack concepts used against control environments w/ live demos

 

6 Comments

Posted December 28, 2016 at 2:33 PM | Permalink | Reply

Andy Bochman

Brilliance as usual from Tim, and with no small amount of levity to boot. But alas our policy makers' and grid risk measurers' question still remains unanswered:

Posted December 28, 2016 at 5:55 PM | Permalink | Reply

tconway

Thanks Andy! and unfortunately there is not a super blog capable of answering the questions that will always remain for policy makers and those responsible for measuring the risk to the grid

Posted December 28, 2016 at 10:48 PM | Permalink | Reply

James Ball

I think this statement is key "With an understanding of an entities protections and controls in place, even at the highest NERC CIP impact level, there is an expectation that an adversary would bring appropriate tools and capabilities to counter those protections and controls to achieve its objectives." While the Ukraine attack was fairly sophisticated from a coordination perspective the TTP were not. I maintain that it is not realistic to expect the average utility to survive a cyber attack from a sophisticated adversary, particularly if that adversary has state resources available to support their efforts. Further- I think the segmentation into low, medium, and high is useful from a power systems perspective it ignores the fact that often these are linked by common networks and controlled by common systems. That to my mind is dangerous- the "system high" approach used in the classified world is more appropriate- and more expensive.

Posted December 28, 2016 at 11:04 PM | Permalink | Reply

tconway

Thanks for the comment James. Couple items > I agree the current HML approach, and systematic approach available in CIP is a significant improvement to the CIP of old, however the inter dependency and interconnected risks that you mention are alive and well and deserving of our attention, analysis, and defense. In regards to the determined nation state adversary comment > I agree and have always recommended system defenders perform mitigations in a way that not only mitigates a specific attack approach, but focuses more on mitigating what an attacker is trying to achieve in the larger operation. This shift in thinking acknowledges that an adversary approach will change based on defenses in place and specific mitigations will be bypassed in follow on attacks, while establishing additional mitigations and capabilities that disrupt, reduce, or eliminates the ability for an adversary to achieve an effect on the operation will not only improve Reliability of the Electric system, but also improve the resiliency of the cyber assets used to operate it.

Posted December 31, 2016 at 4:59 PM | Permalink | Reply

Doug Rhoades

To round out the discussion I think we need to also keep in mind how many millions, perhaps billions, of dollars have been spent on CIP controls in the US and look at ROI when compared to a reliability event such as happened in the Ukraine. Perhaps the C2M2 mapping would help guide better investments in individual CIP requirements going forward?

Posted January 2, 2017 at 5:52 AM | Permalink | Reply

Sam Chanoski

The Ukraine attack against a North American utility would have been different, for sure ''" maybe a bit harder for the adversary but probably still doable in many places at many times. As James said, the TTPs used there were nothing special, nothing rising to the level of nation-state