Just when IT and OT professionals were feeling a little more comfortable in preventing and tackling ransomware/malware attacks, they have something new to worry about—and potentially just as insidious—the Log4j vulnerability. While this potential threat doesn’t pose much of a problem for most Microsoft Windows systems, nevertheless it is worrisome for any device running certain Apache Foundation software—and that can affect everything from cell phones and tablets to Linux servers to OT devices—and potentially to Windows Servers running Apache software.

Log4j has been a big issue for suppliers of ICS (industrial control systems) equipment. But there’s not a single OT vendor that doesn’t have the Log4j software somewhere in their product line, says Eric Byres P.Eng., CTO at aDolus Technology and ISA Fellow. “Plus this is a trivial exploit, so the hackers are loving it,” adds Byres.

Byres is inventor of the Tofino Security firewall technology for industrial controllers and has provided guidance to government security agencies and major energy companies on critical infrastructure protection, chaired the ISA SP-99 Security Technologies Working Group and testified to the U.S. Congress on the security of industrial control systems in national critical infrastructures.

What is Log4j?

Log4j is a Java-based logging library used in a variety of consumer and enterprise services, websites, applications and OT products. These vulnerabilities, especially Log4Shell, are severe, according to the Cybersecurity & Infrastructure Security Agency (CISA).

Log4j is open source software provided by the Apache Software Foundation. It records events—errors and routine system operations—and communicates diagnostic messages about them to system administrators and users. An analogous example can be found in the Windows system Event Viewer, but since it’s proprietary to Microsoft, it isn’t affected by the Log4j vulnerability.

Unfortunately, the Log4Shell uses a feature in Log4j that allows users to write custom code for formatting a log message. Besides tracking user names (real names) and creating log messages, Log4j allows third-party servers to submit code that can perform actions on a targeted computer, which allows hackers to steal sensitive information, take control of a targeted system and slip malicious content to users contacting the hacked server.

Fortunately, Microsoft is largely absolved from this vulnerability. “This isn’t really a Microsoft-related problem, as Log4j is an Apache Foundation product for Java,” says Byres. “So I’d expect that Microsoft products like Active Directory or their operating systems will not be affected since their use of Java is limited. However, there are Microsoft services like SQL Server 2019 Big Data Clusters that are affected. The real issues that we are seeing are the applications running on Microsoft products that may use Log4j and thus, be susceptible to these vulnerabilities.”

What about internet routers and DNS servers? “My initial guess is that most of the core internet routing hardware or DNS servers will not be affected,” says Byres. “The risk is likely with management server applications that monitor and supervise these critical devices—in other words, anything consuming logs from other devices.”

“We are seeing that 90% of the OT vendors have at least one affected product.”

Unfortunately, vulnerabilities turn into attacks

The vulnerability is almost too widespread to comprehend. “With the FACT platform, we have access to a database of about 45 million software packages and components used in the OT space,” says Byres. “We are seeing that 90% of the OT vendors have at least one affected product, and some, like Siemens, have hundreds.”

Tenable, a cybersecurity firm, expects to see several waves of iteration on this exploit, resulting in more aggressive damage that may be impossible to stop. According to telemetry data from Tenable, as of December 2021, only 70% of organizations have even scanned for the vulnerability. Of the assets that have been assessed, Log4Shell has been found in approximately 10%—including a wide range of servers, web applications, containers and IoT devices. Log4Shell is pervasive across all industries and geographies.

This zero-day bug has been in a proof-of-concept, but there have already been attacks. “Unfortunately, there are lots of active attacks occurring,” says Byres. “The government security agencies like CISA and private services like Sophos and CloudFlare are all reporting active exploitation of the vulnerability. It is clear that both adversaries and security researchers are attempting to identify vulnerable hosts on the internet and take them over. Even the ransomware community is seeing this vulnerability as a golden opportunity. They will focus on getting access to their victims now and then launch the ransomware attacks at their convenience later.”

“Unfortunately, your readers should probably be very worried. The vulnerable Log4j software was initially released in 2014 so there have been seven years of shipping software that could be vulnerable.”

What CISA advises

In Alert (AA21-356A), “Mitigating Log4Shell and Other Log4j-Related Vulnerabilities,” CISA details further steps that vendors and organizations with IT and/or cloud assets should take to reduce the risk posed by these vulnerabilities.

These steps include:

• Identifying assets affected by Log4Shell and other Log4j-related vulnerabilities
•  Upgrading Log4j assets and affected products to the latest version as soon as patches are available and remaining alert to vendor software updates
• Initiating hunt and incident response procedures to detect possible Log4Shell exploitation

Easier said than done?

While it’s great that CISA provides these steps, actually taking them could be a real challenge.

“Mitigation is impossible if you don’t even know if you’ve got the vulnerability,” says Byres. “Unfortunately, most OT companies won’t know what products contain Log4j. And if they don’t know that the software they’ve deployed on the plant floor contains the Log4j software, they won’t know to patch or block evil traffic until it is too late and they are compromised. As Jen Easterly, the head of CISA said, this vulnerability is ‘one of the most serious I’ve seen in my entire career, if not the most serious.’”

A major obstacle is that Log4j has been embedded in software products for some time. “Unfortunately, your readers should probably be very worried,” adds Byres. “The vulnerable Log4j software was initially released in 2014 so there have been seven years of shipping software that could be vulnerable. As we scan our database of software bills of materials (SBOMs) for ICS products, we see that Log4j software is widely deployed in OT products by many OEMs.”

Is this vulnerability like SolarWinds?

Some people have been comparing the Log4j vulnerability to SolarWinds, but they are not really comparable, says Byres. “One was a deliberate and sophisticated attack, and the other is a widespread vulnerability. They may be [similar] when we consider long-term impact.”

The Log4j issue is far less sophisticated and is more of a commodity attack opportunity, which is good, because it’s not as stealthy, i.e. it wasn’t hidden into a vendor’s legitimate product and upgrade pipeline, says Byres. The bad news is that it is easy to execute, so many more threat actors will take advantage of it. The other bad news is it will invisibly live on within many products, and many of those products will go without fixes until they are retired.

A patch can’t undo an already compromised host, says Byres. Similar to the SolarWinds case, patches to software containing Log4j don’t completely address the risk. They may remove the attack vector, provided that a company hasn’t already been compromised, but many companies are not in a position to know if they were compromised.

Like SolarWinds, the log4j vulnerability highlights the challenge of third-party risk management that the industry is facing today. Asset owners and governments will certainly demand more visibility into what exactly they are buying when they purchase a software or hardware “solution.” Suppliers will be scrambling to provide this information, and it will be an interesting few months, Byres adds.

Help is available

At the time of writing this report, tools are becoming available, and expert help is available from aDolus Technology (https://adolus.com/vulnerabilities/log4j/) and Tenable (www.tenable.com), which both specialize in OT security. CISA has released an Apache Log4j scanner to find vulnerable applications, according to Sergiu Gatlan on Bleeping Computer.com. “This scanning solution builds upon similar tools, including an automated scanning framework for the CVE-2021-44228 bug (dubbed & Log4Shell) and developed by cybersecurity company FullHunt,” says Gatlan. More about the scanner can be found on GitHub at https://github.com/cisagov/log4j-scanner/tree/master/log4-scanner#features.

Resources

“Apache Log4j Vulnerability Guidance,” Cybersecurity & Infrastructure Security Agency (CISA), https://www.cisa.gov/uscert/apache-log4j-vulnerability-guidance