How processors can guard against cyberattacks
A cybersecurity attack at a food processing plant can lead to production disruptions, theft of intellectual property or even a food safety failure
For many manufacturing companies—and food and beverage are no different—cybersecurity events are happening more often than you think.
These cyberattacks include the all-too-familiar ransomware outbreaks, theft of intellectual property (recipes, customer lists, product research, etc.), or attacks that may even cause an operational problem with a processing or packaging line. And, it’s quite possible that because of a cyberattack, food safety could be threatened or compromised. Unfortunately, the number of events reported to law enforcement is the tip of the iceberg—most go unreported, because talking about a cyberattack is a little like airing dirty laundry.
Some scary numbers
According to Verizon’s “2018 Data Breach Investigations Report,” cybercriminals are still finding success with the “same tried-and-tested techniques,” and their victims are still making the same mistakes. In 2018, there were 53,308 security incidents and 2,216 data breaches in 65 countries—and they’re the ones that were actually reported.
“We’re Only in It for the Money” was the title of a Frank Zappa album, but it’s also the reason for most data breaches. The Verizon report says that 76 percent of recorded data breaches were financially motivated—as opposed to the fabled teen hacker.
But, while a kid hacker having fun is no longer high on the list of the Verizon report’s worries, there is still a potential for a real problem. A 16-year-old from Australia found his way into Apple Inc., stealing 90 gigabytes of files over the course of a year, reported BBC News. While the kid apparently hoped to impress Apple with his hacking prowess—so much so that he wanted to work there when he grew up—there was the issue that customer data may have been included within all the downloaded data, said one news source. Of course, Apple said no, that didn’t happen. But, what if it was customer account info, or even worse, proprietary product design and release information? Could it wind up in the wrong hands—say, a competitor?
So, where are the worst data breaches? The Verizon study says most data breaches and incidents (304 and 22,788 respectively) occur in the public sector. Information, financial and healthcare groups are the most targeted after the public sector. Manufacturing had 71 data breaches and 536 incidents—that we know about. The Verizon report notes that while all-industry data shows most cyberattacks are opportunistic, 86 percent of attacks in manufacturing are targeted, which usually includes stealing planning, research and product development information. Almost half (47 percent) of breaches involved the theft of intellectual property to gain a competitive advantage.
According to Verizon, 73 percent of cyberattacks are perpetrated by outsiders. Members of organized criminal groups are behind half of all breaches, with nation-state or state-affiliated actors involved in 12 percent. Though 28 percent of attacks involved insiders, detecting this type of threat is often difficult. After all, it’s hard to know if someone is using legitimate access for nefarious reasons.
Your SCADA system—is it safe?
“Control systems are evolving to more Ethernet-based networks and Windows-based components and operating systems. This evolution offers greater convenience and easier connectivity to external systems. That is also the downside,” says Steve Pflantz, CRB associate and 2017 president of the International Society of Automation (ISA) and current chair of the Automaton Federation.
With the desire to connect and share data and being able to make remote connections into the plant, plus IIoT, there are more things interconnected than ever before, and that sets a plant up for a certain amount of risk, says Pflantz. “The risk to the control system is not just from the business system. It can come from the control system and put the business systems at risk as well. Cyberattack can be an equal opportunity threat.”
For example, in the not-too distant past, two major retailers had customers’ credit card information accessed by way of a virus or malware put on their system from an HVAC technician’s laptop that was infected and connected at one of their store locations while they were repairing the HVAC system, adds Pflantz.
“Hackers are constantly finding new ways to get into systems,” says Tony Baker, Rockwell Automation portfolio manager for ICS cybersecurity. “As they develop new tactics and find new pathways, we have to discover these threats and respond.” Security threats are growing and evolving. Now, it’s not a matter of if, but when they will take aim at your industrial control system (ICS), says Baker.
Today, an attempt of breaking into a system can come at any level and from anywhere. Therefore, most major controls companies (e.g., ABB, Rockwell, Schneider and Siemens) combat this by taking a layered, defense-in-depth (DiD) strategy, born in the military and supported in IEC 62443. For example, ABB breaks its DiD strategy into three layers: Plant Security, Network Security and System Integrity. The DiD approach helps protect against possible security threats.
“There are a multitude of possible security threats against process control networks,” says Melissa Topp, ICONICS senior director of global marketing. “These include malware, denial of service, phishing, password cracking, SQL injection and man-in-the-middle scenarios.” However, each has countermeasures that can be performed to prevent a worst-case scenario. ICONICS, working in partnership with Bedrock Automation (maker of very secure PLCs), has compiled a list of cybersecurity threats and the means to identify and prevent them within industrial control systems, adds Topp.
For malware, Topp recommends that users stay updated on OS patches, use trusted and reputable anti-virus software and limit the amount of excess software installed. For denial of service attacks, IT departments will use preventive measures like air gapping and isolating networks, setting limited IP ranges and load balancing/scaling web servers.
For phishing, users should complete a corporate training course; institute email rules to block .zip, .exe, .bat and other potentially dangerous file types; and separate email and SCADA networks. To stop password cracking, users should encrypt password storage and implement maximum password-attempt rules and a minimum level of password complexity.
For SQL injection, users should verify that the third-party products they use ensure quality product design to prevent vulnerabilities, stay updated on OS patches and enforce security credentials on all write-access points. For man-in-the-middle attacks, IT professionals should enforce use of trusted certificates, ensuring encryption and mutual authentication, and use firewalls and network segmentation, concludes Topp.
“In food processing control systems, recipes are typically stored on a Windows PC, which then sends commands to a PLC/DCS to execute a selected recipe,” says Gary Pratt, PE and president of ControlSphere Engineering. “Unfortunately, both the consumer-grade OS on the PC and the communication link represent significant cybersecurity vulnerabilities.”
This two-part system evolved due to the historic limitations of PLC/DCS hardware and software, adds Pratt. Fortunately, these limitations have been addressed with the latest hardware and the IEC61131-3 programming language. This combination now allows the entire operation to be consolidated directly on a cybersecure controller (such as the Bedrock OSA).
“In addition to eliminating the cybersecurity threat, this approach eliminates the dependency on sole-source hardware by using the broadly accepted IEC standard language, improves uptime by reducing the volume of hardware and simplifies development/maintenance by consolidating on one hardware platform and one programming environment,” says Pratt.
Industrial controllers not exempt from hacking
“The focal point for attacks on industrial operations and critical infrastructure has centered on industrial controllers,” says Michael Rothschild, director of product marketing at Indegy, a provider of ICS network monitoring systems. These controllers manage every process of food and beverage production. Because of their reliability, many of these devices have been in place for years. They are the workhorses of virtually every industrial facility, which is why they are ground zero for attacks, adds Rothschild.
Even though a controller may not use a Microsoft OS doesn’t mean it can’t be attacked. “Many HMI products are based on Microsoft operating systems; most DCS and PLC-based systems, however, are not,” says Larry Grate, director of technology at PREMIER System Integrators, a CSIA Certified Member. “They [DCSs and PLCs] are still vulnerable to compromise—look what happened to Triconex recently with the HatMan virus,” says Grate.
Grate advises using a passive intrusion detection system (IDS) to monitor network traffic and some type of security information and event management (SIEM) system to aggregate switch, firewall and server logs for threat hunting.
“Most PLCs are not Windows or Linux based, but use real-time OSs such as GHS Integrity or Wind River,” says Albert Rooyakkers, founder and CEO of Bedrock Automation. “The single most effective step to protect process control networks is to build cyber protection into the control, I/O, power and other connected devices.”
Featured in FE’s Engineering R&D three years ago, Bedrock took to designing a completely different PLC, one with cybersecurity built in. Rooyakkers says a hardened, cybersecure PLC should include:
- Secure boot with execution starting in ROM
- Powerful processors and hardware crypto acceleration
- Hardware true random number generator
- Strong algorithms with adequate key lengths
- Robust, scalable and secure CA and key management system
- Physical resistance to tampering
Is this the level of cybersecurity and hardening you need in a PLC? Some public utilities have adopted the technology because they can’t afford to be hacked as the consequences could be grave. If you operate your own wastewater treatment facility on site, this technology might just be a good fit.
Just as you keep a PC up to date with OS and application software patches, industrial controllers should also be part of a patch strategy, says Grate.
Patches are announced on ICS vendor sites, and another great place to see an aggregation of almost all vendor patches is the Department of Homeland Security’s ICS-CERT website (https://ics-cert.us-cert.gov). In fact, you can get on a mailing list to stay up to date with all ICS manufacturers’ updates and patches.
If an ICS must be online, protect it
“When industrial controllers were first deployed, they were not connected and interconnected,” says Rothschild. Today’s advances in technology have put these devices online, making them accessible to and targets for hackers. Furthermore, controllers were not built with protections against security threats or human error. Outsiders, insiders and outsiders masquerading as insiders are all potential adversaries capable of taking over machines for nefarious purposes. Rothschild warns that a carefully executed cyberattack can accomplish as much, if not more damage, than physical warfare.
“We see two primary attack vectors in industrial environments,” says Patrick McBride, chief marketing officer at Claroty, a provider of ICS network threat detection systems. “The first is gaining direct access to the operational network through exploit of vulnerabilities or compromise of access credentials. The second is spillover from the business network.”
Direct access hacks are not uncommon. “We increasingly see hackers working to access systems through trusted sources—the workers and third-party vendors that access these systems every day,” says Rockwell’s Baker. “Those in a company’s supply chain that access their systems should adopt the same security standards.”
Ragnar Schierholz, head of cybersecurity, ABB Industrial Automation division, recommends a top-down approach to securing systems. “Rather than starting by just implementing lots of security controls such as firewalls, anti-virus, entry control, etc. across your network, it’s best to start by making sure you fully understand what you have, what vulnerabilities you have, who is likely to target your system and how and what your critical systems and processes are.”
Administrators must ensure they have a complete and detailed view of the operational network, the assets connected to it and the normal communication patterns, says McBride. “You can’t protect what you can’t see.” With this detailed view, administrators must ensure the network is properly segmented. This includes not just segmenting the business network from the operational network to prevent threats from spilling over from one to the other, but also micro-segmenting within the operational network to ensure an actor can’t move laterally within the network if they do gain access.
There was a day when ICS networks never had connections to a business network, but that’s almost improbable and not realistic today—except for a nuclear power plant or other critical operation. “Industrial control systems should be isolated on their own networks—independent of any extraneous business network,” says Scott McCausland, director of data services at Process and Data Automation - Data Services Group, a CSIA Certified Member. There should be strict firewall rules allowing specific traffic to and from each network. This forces only the appropriate applications to have access via very specific means to data that could flow between them. It is imperative that each zone has specific conduits for data flow. This is one level of security that can be very effective, says McCausland.
Access control systems can be hacked, too
Keeping food safe from intentional physical attack is a good reason to use an access control system (ACS). An ACS supposedly keeps the bad actors out. However, like any other networked control system, an ACS can be hacked, abused and thwarted—just as much from insiders as from outsiders.
The insider threat is often the least understood and least considered when looking at physical controls, says Grate. These controls should be designed and installed by firms that specialize in secure building access and monitoring, or they can often be easily overridden and become a liability instead of an asset.
“The basic principle should always be: No one can go where they do not belong,” says McCausland. Access control systems facilitate having people being where they belong, and security cameras can be implemented to help catch people who attempt to violate the ACS rules. If the ACS is connected to a network for evaluation of data and viewing logs, the appropriate network configurations should be in place to limit the ability of that system to interfere with other systems and reduce the possibility of hacking.
“Employees should be educated on the importance of each security control that has been put in place,” says ABB’s Schierholz. “One vulnerability around this type of technology is users allowing others to tailgate through the entry control system as it’s considered an inconvenience to wait or is time consuming with a large group of people.” The SIEM system should be able to log use of entry/exit systems so analysts can detect when the system has been misused.
“Building security is key to protecting manufacturing assets,” says PREMIER’s Grate. Access to operator control rooms, manufacturing IT assets and plant floor operator stations must be controlled separately from general access. When logins are not practical for safety reasons, HMIs should be monitored by cameras to assist in determining what a threat actor may have done.
Criminals take advantage of the often-overlooked principles behind an ACS: authentication and authorization, says McCausland. Authentication is used to validate that a user has access to the system, and authorization handles what the authenticated user should be able to access. Often times, generic credentials (authentication) are provided to users, and those credentials are authorized to access all parts of the system. A good start is to audit your access control system policies.
“Limiting access to only those who absolutely need it reduces the exposure and the risk of unwanted users accessing critical systems and software,” says CRB’s Pflantz. “Remember, the access control system is often another networked computer system that has to be installed with cybersecurity in mind. Early in a facility design, you need to look at locations for network and computer systems and determine logistics and strategies for these controls.”
Video and wireless devices—back doors to hackers
Video (internet protocol [IP]) cameras are an integral part of access control, and in most cases today, internal cameras are connected to a wired Ethernet. Outside, it may be a different story—cameras may be wireless (Wi-Fi) devices and a source for hackers to either “take down,” use as an entryway into your network or even turn them into “bots,” that is, using the cameras’ processors to do other dirty work.
“The same methods used to take down OT or control systems can be used to take down video surveillance systems,” says CRB’s Pflantz. Many are Ethernet-based cameras and components, so if someone does a denial-of-service attack, the video signals don’t get through to be viewed or recorded. Processors must protect against physical connection to the network or access via a wireless access point. Use of wireless cameras is handy, but adds vulnerability to a wireless system, adds Pflantz.
Wireless video cameras should exist on an isolated network with very specific methods of transferring data to other business-related networks, says McCausland. Those wireless networks should have proper access control policies and secure connection protocols enabled to reinforce best practices. Start small and audit the policies, turn on the passwords, make sure that the Wi-Fi network that hosts the cameras can’t access your business systems.
Non-corporate devices—external video cameras or other personally owned devices—should be on well-controlled guest networks that prevent peer-to-peer access to corporate or ICS networks, says Grate. The IT department should have control and visibility into who owns any Wi-Fi devices in the case that event monitoring turns up a compromise coming from these devices.
Speaking of wireless devices, in a FA&M 2018 talk, Cory Garlick, Rockwell Automation, North American industry manager, said that 54 percent of manufacturing facilities suffered an intellectual property loss in the past year due to Wi-Fi break-ins.
Obviously, you want to password protect all wireless devices, and while you might like to give them all fixed IP addresses so you know who and what’s on the system, that may not always be practical, so careful monitoring of all wireless devices obtaining an IP address through DHCP is a good idea.
Preventing intrusion is one important measure of keeping the bad guys out, and that can be obtained by using the latest security features the devices bring (e.g., WPA2, authentication, etc.), but this is only a first step, says Stefan Woronka, business development and regional management, Industrial Security Services, Siemens. A processor also needs to put a monitoring system in place to ensure that the manufacturer starts detecting potential attacks, as there is no 100 percent security.
Another good—but not so obvious—tip comes from CRB’s Pflantz. While you might want a lot of power for your Wi-Fi network at home, maybe it’s not such a good idea for a manufacturing plant. Case in point: Go wardriving and see how many Wi-Fi networks are still not protected—and especially printers that are completely open directly to wireless devices. Why not print a warning message to the owner that the printer may be an open door to the rest of the network?
“You can physically locate the wireless access points such that you limit the ability to pick up a signal outside of your facility or protected space,” advises Pflantz. Adjust signal strength and antenna orientation to mitigate someone’s ability to sit outside of your facility and hack away. You can survey within the facility to make sure you have a wireless signal where needed, and to a great extent, where not needed. In addition, most wireless routers have power adjustments. Set the power for what’s needed for coverage, but not beyond.
On the inside: Disgruntled and fired employees
Cybersecurity break-ins don’t have to originate from outside the walls of your facility—like Russia, China or your fiercest competitor. Disgruntled employees or “exited” employees can be a serious threat—though the latter should have all their accounts deactivated immediately upon termination of employment to help prevent any subsequent network breach, says ICONICS’ Topp.
There should be policies in place to disable user access and passwords and revoke badges, says CRB’s Pflantz. Inside or recently-formerly-inside threats are a lot harder to defend against than the random outside threat.
“IDS Tools like Darktrace and Claroty can help identify traffic patterns that are not typical for your network, often identifying an insider threat,” says PREMIER’s Grate. Monitoring your access controls, and in the event of a break-in, including a cyber-trained individual to assist with the analysis can be critical. A typical security person may not recognize the cyber assets that may have been compromised in such a break-in. Keylogger devices and similar black boxes can often be left behind to collect information and then accessed physically later or via wireless technologies. Finally, a SIEM solution can be very helpful in identifying credentials that are being used at odd times, or on systems that are atypical or not a part of an employee’s normal workflow, says Grate.
“When you design and construct control systems, always consider physically securing access points, using software methods to disable externally accessible network ports, and putting software and alarms in place to monitor and alert you when things that are not supposed to be used are being used,” advises Pflantz. “Get over the feeling that you are not trusting your coworkers or employees. You are mitigating unnecessary risks.”
“Access accounts disabled, retrieval of computer hardware, removal from all ancillary systems —immediately upon exiting personnel,” says Process and Data Automation’s McCausland. “Having a plan, and then sticking to it, can save you from diabolical ex-employees or current workers who feel slighted.”
Following these suggestions may prevent an intentional food adulteration and/or food safety event.
“Cyber Security & IP Cameras: Everyone’s Concern,” SDM Magazine, January 2018.
“Are your control networks safe from cyberattacks?” FE, March 2016.
Triconex Hatman virus—ICS CERT.
ICS-CERT (ICS Cyber Emergency Response Team), https://ics-cert.us-cert.gov/.
NIST “Cybersecurity Framework,” NIST website.
“How food processors can protect against cybersecurity breaches,” FE Manufacturing News, January 2018.
For more information:
ABB Food & Beverage, https://new.abb.com/food-beverage
Bedrock Automation, https://bedrockautomation.com
ICONICS, Inc., www.iconics.com
PREMIER System Integrators, www.premier-system.com
Process and Data Automation, http://processanddata.com
Rockwell Automation, www.rockwellautomation.com
Siemens Industrial Security Services, www.siemens.com