Cycode Labs have uncovered a new vulnerability, CVE-2024-31989, with a critical score of 9.1. The vulnerability affects Kubernetes clusters equipped with Argo CD, a widely used GitOps continuous delivery tool for Kubernetes. The vulnerability exploits the Argo CD server’s elevated permissions, creating an attack vector for malicious actors to escalate their privileges and potentially take complete control over the Kubernetes cluster.
Let’s Break It Down
Before we delve into the vulnerability, let’s provide some background on the systems in play.
GitOps is an emerging paradigm that leverages Git repositories to manage and automate the deployment of infrastructure and applications, offering a declarative approach to Kubernetes cluster management. It involves using Infrastructure as Code (IaC) files to document your infrastructure configurations while also being the source of truth for the operational infrastructure. You can read more about the GitOps concept here.
Argo CD is an open-source tool built around the GitOps principle. It continuously monitors Git repositories for any changes in application configurations and seamlessly applies these updates to the Kubernetes cluster. Given its role, Argo CD is granted high-level access to the Kubernetes cluster and may also have permission to access private Git repositories to pull infrastructure configurations. This extensive access makes it a prime target for attackers. Argo CD has great documentation, which you can explore further here.
The identified vulnerability in Argo CD exploits its extensive permissions, providing attackers with a pathway to escalate their privileges and potentially gain control over the Kubernetes cluster. Specifically, the attack manipulates the data stored in Argo CD’s Redis caching server, which, by default, lacks password protection and can be accessed by any pod within the cluster if proper network policies are not in place.
The attack involves altering application state manifests stored in the Redis server, which Argo CD uses to determine the desired state of infrastructure in the cluster. The integrity of these manifests is typically verified by a hash generated by Argo CD. However, this hash can be recalculated without a secret key, enabling attackers to modify the manifest data and present it as legitimate. This breach leads Argo CD Server to accept the altered manifest as a valid cache entry, unintentionally triggering an unauthorized update to the cluster’s state.
By exploiting this flaw, attackers can change the application manifest to deploy malicious pods with high privileges, execute code on the host node, and access sensitive information, including Kubernetes Secrets. Moreover, the attack allows for the removal of all evidence.
Why Research Argo CD?
There are several reasons why we looked into Argo CD.
It’s a rising star, attracting major customers as its popularity grows. Argo CD is a project under the Cloud Native Computing Foundation (CNCF). It is dedicated to building and managing continuous delivery workflows in Kubernetes. Argo CD enjoys widespread popularity, being used by major companies such as TikTok, Spotify, Mercedes Benz, and more. The 2023 Argo CD Users Survey reveals an impressive Net Promoter Score (NPS) of 76, with 93% of users deploying Argo CD in their organization’s production environments.
Argo CD, inherently designed with high cluster permissions, is a prime target for attackers seeking privilege escalation. Its operation requires elevated privileges in the cluster to deploy infrastructure and monitor the cluster’s state.
Are You Vulnerable?
To ensure you are not exposed to this attack, take the following steps:
- Ensure that your Argo CD application is updated to version 2.11.1, 2.10.10, 2.9.15, and 2.8.19.
- Verify that the Network Policy rule named “argocd-redis-network-policy” is present and enabled within your cluster. This rule is intended to limit access to the Redis server.
kubectl -n argocd get networkpolicy argocd-redis-network-policy
- Ensure a Container Network Interface (CNI) is correctly configured and operational in your Kubernetes cluster. This step is essential for enforcing the Argo CD Network Policy rules. For further details, please refer to this support article.
- Use controller-based secrets management tools instead of injecting secrets via Argo CD plugins. This keeps secrets from being exposed in the Redis server. Learn more.
Note: For enhanced security, it is advisable to deploy Argo CD on a dedicated cluster isolated from other applications.
The Research Story
Let’s Test the System
We began by deploying an Argo CD application on a new Amazon EKS cluster using the default configuration. We then created a simple demo application consisting of two pod deployments and one service.
Everybody Loves Redis
After experimenting with the system for a while, our curiosity shifted toward understanding the Kubernetes components of the Argo CD application. We noticed the deployment includes two familiar pods, Redis and Dex, so we decided to start with them. 
We connected to the Redis Server without the need for a password by using port forwarding to its pod. We found many keys that seem to reflect data about our deployed application. When we tried to view their content, it appeared compressed, likely in gzip format.
Curious about the Redis content, we referred to the documentation. It indicated that Redis serves as the cache layer, reducing requests sent to the Kube API and the Git provider. We also observed an interesting Secret Management page on the documentation website, stating the following:
Storing secrets in Redis? That’s interesting. There are two possibilities where secrets can end up in the cache: Users commit manifests with secrets or Argo CD Admins configure plugins to generate manifests with secrets. Both of these cases can have a severe impact.
Who can access the Redis server? To test this, we created a low-privilege pod in another namespace, simulating a compromised pod within the cluster. It could potentially be a webshell, a malicious package, or any other low-access point to the cluster. We resolved the Redis server’s address using the local kube-DNS server and attempted to connect to it on port 6379. Surprisingly, it worked!
Knowing that the Redis server doesn’t require a password by default, we proceeded with the following:
Using the Argo CD source code as a reference, we developed a Go program to decompress the gzipped contents and uncover their values. As expected, it was cached information about the applications managed by Argo CD, including their manifests and details about the Kubernetes cluster we ran in.
This confirms that any pod in the cluster could potentially access the Redis server and secrets. But what if there are no secrets? Could we still take over the cluster using the Redis server?
Cluster Takeover via Redis Server
To answer this, we needed to identify which keys in the Redis server affect application behavior and understand how Argo CD decides when to update or generate a resource. We used a Redis feature called the profiler to observe pods’ interaction with the Redis server. 
We utilized Redis’s profiler to analyze the server’s events as we observed that the Application Controller pod is accessing a key named ‘mfst|app.kubernetes.io/instance|myapp|42…|1.8.3.gz’. This key appears to contain the manifest of the application Argo CD monitors. It’s logical that the Application Controller accesses this key since it is in charge of matching the user-defined application state with the live state in the Kubernetes cluster.
We thought we should try to make the Application Controller load our manipulated application by modifying the manifest value. However, once we changed the manifest, the Argo CD Repo Server quickly reverted the change, restoring the original value.