Software supply chain attacks are on the rise and growing in severity. Part of this is due to the interconnectedness of DevOps and CI/CD tooling as well as the rise of GitOps. The “everything as code” movement makes lateral movement across all systems easier once a single tool is breached. This blog defines what lateral movement is, why it’s so easy to move across the software development life cycle (SDLC) from one tool to another, and what you can do to harden your entire software supply chain to prevent future breaches.
What Is Lateral Movement?
Lateral movement in the SDLC occurs when an attacker gains access to a user account then uses those login credentials to move into additional tools and steal advanced user privileges. Once attackers gain access to the original breached tool, they move through systems, looking for new opportunities and gaining increased privileges. Malicious actors are able to move deeper into their target’s software delivery pipeline in search of other high-value data or assets, even manipulating code to target downstream customers.
An attack that involves lateral movement has several goals. Attackers use lateral movement to:
- Expand into additional tools and applications
- Identify privileged user credentials
- Map critical systems and high-value data
This type of attack is difficult to detect because malicious actors use valid credentials to move through systems. Attackers are able to evade detection and remain in an organization’s pipeline, often long after the initial breach is detected. This allows attackers to deepen and expand the attack, moving from a single entry point to total SDLC compromise, code tampering, or gaining access to sensitive IP and customer data. Often with these attacks, the original victim is not the final target, but rather a stepping stone to move through a network.
Lateral Movement and the Codecov Breach
The Codecov breach is an example of malicious actors using lateral movement to execute a supply chain attack. The breach began in January 2021, but was not discovered until mid-April—more than 10 weeks after attackers gained access to their systems.
The breach started with an error in one of Codecov’s Docker images. Attackers were able to steal credentials from their Docker image, which allowed them to introduce malicious code into the company’s Bash Uploader script. By adding just a single line among more than 1,800 lines of code in the Bash Uploader script, attackers were able to move laterally into Codecov’s CI server to send customers’ credentials to a remote server. From there, attackers used stolen Git credentials to access the private repositories of Codecov customers. The attackers scanned those repositories for any secrets or sensitive data and then exploited those secrets to breach customers’ tools and applications. Companies impacted by this software supply chain breach included Google, IBM, HP, Hashicorp, Confluent, and many others.
Ultimately, a Codecov customer discovered the breach when they noticed that the Bash Uploader had a different hash value than what was published on Codecov’s website. Unfortunately, malicious actors were able to move laterally through CI systems to maximize the impact of the exploit for weeks before it was discovered and disclosed.
The Trouble with DevOps and GitOps
The rise in DevOps and GitOps practices has made lateral movement across the SDLC easier than ever. Why are these practices, which are aimed at improving software development, making pipelines less secure?
DevOps
DevOps is a set of best practices that automate and integrate application development processes between software development and IT teams. It includes source code version control, compliance, CI/CD tooling, and more. DevOps’s goalis to shorten development timelines while delivering high-quality software. The outcome of this automation is that development and deployment tools are highly connected. Attackers can piggyback on that interconnectedness to further their ends.
GitOps
Similarly, GitOps is a set of best practices for infrastructure automation. Think of it as the infrastructure equivalent of DevOps. Much like the SDLC has been automated to meet the demands of modern software development, today’s infrastructure needs to be both flexible and automated to manage the cloud resources required for continuous deployment. GitOps automates the process of provisioning infrastructure, storing configuration files as code. Infrastructure as Code (IaC) allows IT to manage and provision computer data centers through machine-readable definition files rather than physically configuring hardware. Like DevOps, GitOps is all about interconnection and automation.
Interconnected Tools and Lateral Movement
As DevOps and GitOps practices become more commonplace, development tools are now deeply interconnected. Once attackers have breached a single system, automated pipelines make it easy for them to move laterally across the SDLC and remain undetected for long periods of time. With everything as code, malicious actors can sweep through an entire system and do more damage with less effort. As the Codecov breach shows, one poorly configured file can expose your entire system—not to mention your customers’ systems—and cause irreparable damage.
Defending against software supply chain attacks requires downstream customers to be aware that they are uniquely vulnerable. Many third-party software solutions require both privileged access to and frequent communication with their customers’ networks. Their exposure could mean your exposure.
Preventing Lateral Movement to Secure Your SDLC
SolarWinds. Codecov. Kaseya. Software supply chain attacks are only increasing in frequency and intensity. The SDLC provides a broad attack surface from which to execute these attacks. Interconnected automated tools allow for lateral movement with devastating effects. Protecting development pipelines and infrastructure is imperative for all organizations.
To reduce attack surfaces, organizations should implement and enforce SDLC-wide governance and security policies. Guaranteeing that policies like least privilege are in place and hardening authentication through multi-factor authentication and single sign-on controls reduce the risk of all security issues.
Organizations also need to scan for hardcoded secrets and misconfigured IaC code regularly. Hardcoded secrets can be found throughout the SDLC—in proprietary code, logs, containers, production environments, and IaC code—and provide access to your pipeline. Attackers who have breached one system are able to use hardcoded secrets to move even deeper into their target’s infrastructure. Similarly, an IaC misconfiguration, in which private resources have public visibility, can provide another entry point for malicious actors to move through your pipeline. Often simple errors lead to disastrous breaches. The goal for any organization should be to reduce attackers’ pathways to prevent breaches whenever possible, but at the very least to minimize their impact.
Lastly, software supply chain attack surfaces are vast and interconnected, and software development pipelines are constantly evolving. To prevent lateral movement and potentially even greater destruction, organizations would benefit from a solution that offers complete visibility of their entire software supply chain and automatically remediates defects to prevent future breaches.
Originally published: April 19, 2022
