SANS Industrial Control Systems Security Blog

Confirmation of a Coordinated Attack on the Ukrainian Power Grid

SANS ICS

This post was written by Michael J. Assante, SANS ICS Director:

After analyzing the information that has been made available by affected power companies, researchers, and the media it is clear that cyber attacks were directly responsible for power outages in Ukraine. The SANS ICS team has been coordinating ongoing discussions and providing analysis across multiple international community members and companies. We assess with high confidence based on company statements, media reports, and first-hand analysis that the incident was due to a coordinated intentional attack.

The attackers demonstrated planning, coordination, and the ability to use malware and possible direct remote access to blind system dispatchers, cause undesirable state changes to the distribution electricity infrastructure, and attempt to delay the restoration by wiping SCADA servers after they caused the outage. This attack consisted of at least three components: the malware, a denial of service to the phone systems, and the missing piece of evidence of the final cause of the impact. Current evidence and analysis indicates that the missing component was direct interaction from the adversary and not the work of malware. Or in other words, the attack was enabled via malware but consisted of at least three distinct efforts.

The Multiple Elements
The cyber attack was comprised of multiple elements which included denial of view to system dispatchers and attempts to deny customer calls that would have reported the power out. We assess with high confidence that there were coordinated attacks against multiple regional distribution power companies. Some of these companies have been reported by media to include specifically named utilities such as Prykarpattyaoblenergo and Kyivoblenergo. The exact timeline for which utilities were affected and their ordering is still unclear and is currently being analyzed. What we do know is that Kyivoblenergo provided public updates to customers, shown below, indicating there was an unauthorized intrusion (from 15:30 ? 16:30L) that disconnected 7 substations (110 kV) and 23 (35 kV) substations leading to an outage for 80,000 customers.

Screen Shot 2016-01-06 at 10.12.55 PM

The key significance here is that 80,000 customers comprise a significant portion of their residential load. Power was restored to all customers by (18:56L). They also reported technical failures with their call line interfering with receiving customer's calls as shown below.

Screen Shot 2016-01-06 at 10.13.20 PM

Quick action by utility staff to switch to "manual mode" and restore the system was impressive. Statements from utility staff to local media indicated the distribution system was being run without the benefit of their SCADA as it was still infected. Field staff at the impacted power companies manned required substations, transferring from "automatic to manual mode", and manually re-closed breakers to energize the system. Restoration varied but all services were restored in 3-6 hours. It is important to note that there are risks operating your system without the benefit of an automated dispatch control center and utilities that are more reliant on automation may not be able to restore large portions of their system this way. In many ways, the Ukrainian operators should be commended for their diligence and restoration efforts.

Cyber Attack Milestones as Reported To Date:
From what has been reported, here is the information to date that we are confident took place. The exact timing of the events is still being pieced together.

  • The adversary initiated an intrusion into production SCADA systems
  • Infected workstations and servers
  • Acted to "blind" the dispatchers
  • Acted to damage the SCADA system hosts (servers and workstations)
    • Action would have delayed restoration and introduce risk, especially if the SCADA system was essential to coordinate actions
    • Action can also make forensics more difficult
  • Flooded the call centers to deny customers calling to report power out

Probable Cyber Attack Milestones as Reported to Date:
In analyzing the evidence and reports there are still missing pieces to the attack. Understanding the initial foothold of the adversary, the eventual impact, and the types of systems in place can help to make assessments on what the adversary likely had to have done but the items stated below are currently probable and not known. We are working to verify and uncover more information.

  • The adversaries infected workstations and moved through the environment
  • Acted to open breakers and cause the outage (assessed through technical analysis of the Ukrainian SCADA system in comparison to the impact)
  • Initiated a possible DDoS on the company websites

Malware Enabled but Not Likely Malware Caused
It is interesting and important to understand the role of the malware sample SANS ICS previously reported that came from one of the infected networks. There have been two prominent theories in the community and speculation to the media that either the 'KillDisk' component was just inside the network and unrelated to the power outage (a reliability issue where malware just happened to be there) or that the 'KillDisk' component was directly responsible for the outage. It is our assessment that neither of these are correct. Malware likely enabled the attack, there was an intentional attack, but the 'KillDisk' component itself did not cause the outage.

It is also important to note that many of the samples being analyzed in the community to date as reported by others are not guaranteed to have been involved in this incident. The malware campaign reported, tied to BlackEnergy and the Sandworm team by others, has solid links to this incident but it cannot be assumed that files such as the excel spreadsheet and other malware samples recovered from other portions of that campaign were at all involved in this incident. It is possible but far too early in the technical analysis to state that. The type of analysis being done by the security researchers and companies assessing this is valuable analysis and they should be commended. At the worst it will provide lessons learned and training opportunities for the community. But analysts should be careful not to overstate current analysis of malware samples due to their link to the larger campaign as being specific to this incident. Simply put, there is still evidence that has yet to be uncovered that may refute the minutia of the specific components of the malware portion of the attack.

More importantly, the link of the KillDisk wiper to the actual cause of the outage is not likely. This is stated because power systems and SCADA schemes simply do not work in that manner. In other words, the incident observed with consideration to timing, sites, and impact does not at all align with the narrative of the 'KillDisk' component itself causing the impact. I have observed the loss of many SCADA systems for periods of time that resulted in no outage or impact to the power system. Running a power system without the benefit of your SCADA system at the distribution-level adds risk, but without something to change the ?state' (for example to force a circuit to de-energize) then the system will continue to serve power. We assess currently that the malware allowed the attackers to gain a foothold at the targeted utilities, open up command and control, and facilitate the planning of an attack by providing access to the network and necessary information. The malware also appears to have been used to wipe files in an attempt to deny the use of the SCADA system for the purposes of restoration to amplify the effects of the attack and possibly to delay restoration.

Final Thoughts
We are very interested in helping power utilities learn as much as they can from this real world incident. We would also note the competent action by Ukrainian utility personnel in responding to the attack and restoring their power system. As a community the power industry is dedicated to keeping the lights on. What is now true is that a coordinated cyber attack consisting of multiple elements is one of the expected hazards they may face. We need to learn and prepare ourselves to detect, respond, and restore from such events in the future. The SANS ICS team will be continuing our analysis and presenting findings and updates to the community in multiple formats. On Jan 20th we will host a webcast focusing on understanding the industrial control systems and SCADA networks of the Ukrainian power grid to identify what was even possible in terms of attack scenarios. Following that, we will release more information at the SANS ICS Summit with a full breakdown of what we know and its value to the community. Finally, we will be releasing a comprehensive whitepaper on the incident in our Defense Use Case (DUC) series in our ICS Digital Library. The DUC will highlight both the cyber and physical components to this incident and the lessons learned for the community.

We sincerely thank all the effort going on in the community by numerous passionate researchers and companies across both the information technology and the ICS community. It takes all of us working together to understand and respond to these types of incidents.

Michael AssanteBio: Michael Assante is currently the SANS lead for Industrial Control System (ICS) and Supervisory Control and Data Acquisition (SCADA) security and co-founder of NexDefense an Atlanta-based ICS security company. He has also served as CSO of NERC, several high-level positions at Idaho National Labs, and CSO of American Electric Power.

12 Comments

Posted January 9, 2016 at 10:57 PM | Permalink | Reply

Chris Wolski

Thanks for keeping this real and up to date.

Posted January 10, 2016 at 3:27 PM | Permalink | Reply

Jake Brodsky

I wouldn't be so quick to dismiss the wiper component, Mike. If I were trying to shut down a grid to where black-starts would be needed, I would first trip a series of breakers and then I would kill the SCADA system to prevent it from being used for the recovery.

Posted January 10, 2016 at 6:21 PM | Permalink | Reply

robertmlee

Jake, that is actually consistent with our current analysis. Mike and the team aren't dismissing that the wiper was involved in the attack (thus the malware enabled but not malware caused comment). It very likely was used to blind the operators and delay restoration efforts. But the wiper itself did not cause the outage. There are a lot of folks in media stating that the wiper itself could have caused the power outage and impact ''" that is just not a consistent narrative with what has been observed. We entirely agree with you that KillDisk played a role in the amplifying effect though.

Posted January 11, 2016 at 2:58 PM | Permalink | Reply

Michael W. David

Mike
Great feedback. Will use it in classes this week.

Posted January 11, 2016 at 3:17 PM | Permalink | Reply

steven

Nicely done.
You indicate missing pieces, with initial attacker foothold being one of those.
Any word on the suspected attack vector? Remote access, phishing, removable media (USB), etc''?
I'd be interested to know.
Thanks,
Steven

Posted January 12, 2016 at 7:49 AM | Permalink | Reply

Ryu

I agree with the possibility that critical control commands were sent to the substation devices from the infected SCADA systems. I think there was some reason why the KillDisk component targeted to the komut.exe and sec_service.exe processes. Because the sec_service.exe is used for serial to ethernet connection service in both Eltima Serial to Ethernet Connector and ASEM Ubiquity, the adversary may targeted this process to intercept communication between SCADA system and substation devices and send some critical control messages.

Posted January 12, 2016 at 11:21 PM | Permalink | Reply

Munish Verma

Thanks for a detailed and more accurate version of the events, which unfolded.

Posted January 14, 2016 at 3:45 PM | Permalink | Reply

Steve Rawson

Is the webcast for the 20th still on? I don't see it posted at https://www.sans.org/webcasts/upcoming

Posted January 18, 2016 at 7:15 PM | Permalink | Reply

robertmlee

No, there's more information unfolding so we are pushing it back into Feb.

Posted January 15, 2016 at 9:09 PM | Permalink | Reply

Daryl Wheat

Most older SCADA systems in Petrochemical plants sit on top of older PLC's. The SCADA side, with my experience can go down completely and the PLC operation and control would not be comprised. You'd just be blind to the PLC's operation from the HMI side. That was a major selling point to those of us in the 90's that were using these platforms. We trusted the PLC to keep our operations and plants safe. HMI's were just a necessary evil. Most programmers setup a Genius Block (GE PLC platform) to hold last value. So that Discrete Output block signal holding that relay to feed that critical power stayed on even if you jerked the Bus cable out of the genius block. Attack on the HMI side would probably do little in most settings. Understanding which Bits needed turning on and off would be the key. Watching and learning the HMI side would give you that information. Wiping the HMI was just blowing up the bridge.
If you cant get to the fire, cant fight it''''''

Posted February 3, 2016 at 10:09 AM | Permalink | Reply

Dirk

Maybe the ''KillDisk' component was used only to wipe there traces?

Posted February 9, 2016 at 4:37 AM | Permalink | Reply

Franky Thrasher

Thanks to you and the SANS team for some true insight to this incident.