Cycode Labs have uncovered a new vulnerability, CVE-2024-31989, with a critical score of 9.1. The vulnerability affects Kubernetes clusters equipped with Argo CD, a widely used GitOps continuous delivery tool for Kubernetes. The vulnerability exploits the Argo CD server’s elevated permissions, creating an attack vector for malicious actors to escalate their privileges and potentially take complete control over the Kubernetes cluster.
Let’s Break It Down
Before we delve into the vulnerability, let’s provide some background on the systems in play.
GitOps is an emerging paradigm that leverages Git repositories to manage and automate the deployment of infrastructure and applications, offering a declarative approach to Kubernetes cluster management. It involves using Infrastructure as Code (IaC) files to document your infrastructure configurations while also being the source of truth for the operational infrastructure. You can read more about the GitOps concept here.
Argo CD is an open-source tool built around the GitOps principle. It continuously monitors Git repositories for any changes in application configurations and seamlessly applies these updates to the Kubernetes cluster. Given its role, Argo CD is granted high-level access to the Kubernetes cluster and may also have permission to access private Git repositories to pull infrastructure configurations. This extensive access makes it a prime target for attackers. Argo CD has great documentation, which you can explore further here.
The identified vulnerability in Argo CD exploits its extensive permissions, providing attackers with a pathway to escalate their privileges and potentially gain control over the Kubernetes cluster. Specifically, the attack manipulates the data stored in Argo CD’s Redis caching server, which, by default, lacks password protection and can be accessed by any pod within the cluster if proper network policies are not in place.
The attack involves altering application state manifests stored in the Redis server, which Argo CD uses to determine the desired state of infrastructure in the cluster. The integrity of these manifests is typically verified by a hash generated by Argo CD. However, this hash can be recalculated without a secret key, enabling attackers to modify the manifest data and present it as legitimate. This breach leads Argo CD Server to accept the altered manifest as a valid cache entry, unintentionally triggering an unauthorized update to the cluster’s state.
By exploiting this flaw, attackers can change the application manifest to deploy malicious pods with high privileges, execute code on the host node, and access sensitive information, including Kubernetes Secrets. Moreover, the attack allows for the removal of all evidence.
Why Research Argo CD?
There are several reasons why we looked into Argo CD.
It’s a rising star, attracting major customers as its popularity grows. Argo CD is a project under the Cloud Native Computing Foundation (CNCF). It is dedicated to building and managing continuous delivery workflows in Kubernetes. Argo CD enjoys widespread popularity, being used by major companies such as TikTok, Spotify, Mercedes Benz, and more. The 2023 Argo CD Users Survey reveals an impressive Net Promoter Score (NPS) of 76, with 93% of users deploying Argo CD in their organization’s production environments.
Argo CD, inherently designed with high cluster permissions, is a prime target for attackers seeking privilege escalation. Its operation requires elevated privileges in the cluster to deploy infrastructure and monitor the cluster’s state.
Are You Vulnerable?
To ensure you are not exposed to this attack, take the following steps:
- Ensure that your Argo CD application is updated to version 2.11.1, 2.10.10, 2.9.15, and 2.8.19.
- Verify that the Network Policy rule named “argocd-redis-network-policy” is present and enabled within your cluster. This rule is intended to limit access to the Redis server.
kubectl -n argocd get networkpolicy argocd-redis-network-policy
- Ensure a Container Network Interface (CNI) is correctly configured and operational in your Kubernetes cluster. This step is essential for enforcing the Argo CD Network Policy rules. For further details, please refer to this support article.
- Use controller-based secrets management tools instead of injecting secrets via Argo CD plugins. This keeps secrets from being exposed in the Redis server. Learn more.
Note: For enhanced security, it is advisable to deploy Argo CD on a dedicated cluster isolated from other applications.
The Research Story
Let’s Test the System
We began by deploying an Argo CD application on a new Amazon EKS cluster using the default configuration. We then created a simple demo application consisting of two pod deployments and one service.
Everybody Loves Redis
After experimenting with the system for a while, our curiosity shifted toward understanding the Kubernetes components of the Argo CD application. We noticed the deployment includes two familiar pods, Redis and Dex, so we decided to start with them. 
We connected to the Redis Server without the need for a password by using port forwarding to its pod. We found many keys that seem to reflect data about our deployed application. When we tried to view their content, it appeared compressed, likely in gzip format.
Curious about the Redis content, we referred to the documentation. It indicated that Redis serves as the cache layer, reducing requests sent to the Kube API and the Git provider. We also observed an interesting Secret Management page on the documentation website, stating the following:
Storing secrets in Redis? That’s interesting. There are two possibilities where secrets can end up in the cache: Users commit manifests with secrets or Argo CD Admins configure plugins to generate manifests with secrets. Both of these cases can have a severe impact.
Who can access the Redis server? To test this, we created a low-privilege pod in another namespace, simulating a compromised pod within the cluster. It could potentially be a webshell, a malicious package, or any other low-access point to the cluster. We resolved the Redis server’s address using the local kube-DNS server and attempted to connect to it on port 6379. Surprisingly, it worked!
Knowing that the Redis server doesn’t require a password by default, we proceeded with the following:
Using the Argo CD source code as a reference, we developed a Go program to decompress the gzipped contents and uncover their values. As expected, it was cached information about the applications managed by Argo CD, including their manifests and details about the Kubernetes cluster we ran in.
This confirms that any pod in the cluster could potentially access the Redis server and secrets. But what if there are no secrets? Could we still take over the cluster using the Redis server?
Cluster Takeover via Redis Server
To answer this, we needed to identify which keys in the Redis server affect application behavior and understand how Argo CD decides when to update or generate a resource. We used a Redis feature called the profiler to observe pods’ interaction with the Redis server. 
We utilized Redis’s profiler to analyze the server’s events as we observed that the Application Controller pod is accessing a key named ‘mfst|app.kubernetes.io/instance|myapp|42…|1.8.3.gz’. This key appears to contain the manifest of the application Argo CD monitors. It’s logical that the Application Controller accesses this key since it is in charge of matching the user-defined application state with the live state in the Kubernetes cluster.
We thought we should try to make the Application Controller load our manipulated application by modifying the manifest value. However, once we changed the manifest, the Argo CD Repo Server quickly reverted the change, restoring the original value.
Upon re-examining the manifest to understand how the Repo Server detects our changes, we noticed the “cacheEntryHash” key, reminiscent of a checksum. According to the source code, this hash is intended to validate the manifest. However, it is generated without any private secret signing it.
We developed a program to replicate the fnv64a hashing algorithm in order to regenerate the checksum hash for our manipulated manifest.
We Can Exploit It!
Afterward, the modification to the manifest in Redis was successful. During the next state validation by the Application Controller, it detected that it was ‘out of sync’ and deployed our changes to the Kubernetes application. As a proof of concept, we deployed a privileged pod using a configuration from BadPods project on GitHub.
The newly deployed pod possesses elevated capabilities. It also mounts the host node’s filesystem to our pod. This grants us numerous options to escalate privileges and execute code on the host node, effectively achieving the goal we had set out.
We decided to add our public SSH key to the node’s authorized keys, which allows us to connect to the node with an SSH shell. 
Responsible Disclosure Timeline:
We disclosed the vulnerabilities to Argo CD in January 2024. Argo CD investigated and addressed the issues while keeping us updated.
- January 18, 2024 – Cycode Labs reported the vulnerability to the ArgoCD team through GitHub Security Advisories.
- January 25, 2024 – The Argo CD team confirmed the existence of the security issue.
- April 9, 2024 – The Argo CD team created CVE-2024-31989 for this vulnerability and began working on a solution.
- May 21, 2024 – Argo CD team released versions 2.11.1, 2.10.10, 2.9.15, and 2.8.19. These include an initialized password for the Redis server, partially addressing the security issue, as connecting to the Redis database without prior information will not be possible.
Argo CD Response:
The Argo CD maintainers are deeply committed to addressing security issues when they are raised. While many users mitigate this vulnerability with Redis already by isolating it from external actors, we recognize that not every Argo CD admin will implement the same measures. The patch we have released will make it easier for admins to protect their Redis instances with our default installation manifests and our official Helm chart. Cycode demonstrated exceptional professionalism by closely collaborating with us. They actively participated in all stages to ensure the fix was available to our entire user base before disclosing it. It was a pleasure collaborating with their team with the common goal of providing a safer Argo CD for everyone.
How Cycode Can Help
Cycode is a powerful platform that can help you identify and prevent misconfiguration or exploits like this one:
- Redis Pods without Password – Cycode’s advanced scanning capabilities enable the detection of all Redis pods within your cloud infrastructure that do not have a password set. This policy identifies the Argo CD Redis server and generates a violation on your system.
- Network Policy Enforcement—Cycode’s platform empowers you to create a custom policy rule to verify the presence of Kubernetes network policies. For example, you can verify the presence of the Argo CD network policy rule that restricts access to the Redis server. By ensuring the implementation of network policy rules, you enhance the security of your Kubernetes environment and mitigate the risk of unauthorized access to sensitive resources.
Cycode ASPM
Cycode is the leading Application Security Posture Management (ASPM) platform, providing peace of mind to its customers. Our complete ASPM platform scales and standardizes developer security without slowing down the business, delivering safe code, faster. Cycode delivers cyber resiliency through unmatched visibility, risk-driven prioritization, and just-in-time remediation of code vulnerabilities at scale. Cycode’s Risk Intelligence Graph (RIG), the brain behind the platform, provides traceability across the entire SDLC through natural language. As a purpose-built platform for developer security, Cycode delivers visibility, prioritization, and remediation of vulnerabilities across the entire SDLC.
Want to learn more about Cycode? Book a demo now.
