Exposed credentials are one of the most abused methods for gaining initial access. Breaches such as the Zendesk breach have been started by employees’ compromised credentials. GitHub suffered a massive breach due to stolen OAuth tokens. The CodeCov Bash uploader script was altered by malicious actors who obtained credentials due to an error in Codecov’s Docker image creation process. The post-breach playbook often emphasizes the need to revoke all forms of access tokens to prevent unauthorized access.
It’s a widely acknowledged practice that security professionals, organizations, and even government agencies like the Cybersecurity and Infrastructure Security Agency (CISA) recommend.
Our team examined different methods that may allow potential threat actors to remain hidden and undetected, even after the standard procedures were executed. The following article will dive into the world of “Shadow Tokens.” By examining the behavior of three popular systems, including one vulnerability we discovered, we will show that shadow tokens can be a powerful mechanism for potential attackers to sustain access to a range of systems and accounts quietly while concealing their presence and evading detection.
Maintaining the Foothold
Once a threat actor gains initial access to a user’s account, regardless of the method employed, one of their primary objectives is to ensure the persistence of their foothold. Threat actors operate with a deep-seated fear of being detected and losing access to their target accounts. To avoid such consequences, threat actors are cautious in their actions, carefully avoiding those that might raise alarms and threaten their continued access.
What threat actors (probably) won’t do
- Create a new access token: Threat actors typically avoid creating new access tokens because this action may trigger an immediate alert to the account owner and system administrators. It would signal a clear change in the account activity, potentially leading to an investigation of suspicious behavior.
- Change the user’s password: Changing the user’s password is another action attackers usually avoid. Password changes are highly visible and likely alert the account owner of unauthorized access. The account owner would probably take swift action to secure their account, including reporting the intrusion to system administrators.
Avoiding actions that might raise suspicion and stay under the radar will determine between a successful attack and one that will fail. To achieve this goal, threat actors must explore new ways to remain undetected for an extended period.
Instead, potential attackers may seek to exploit related services with extended sign-in sessions.
Services such as mobile apps and GUI interfaces are designed to provide user convenience but can also be leveraged by attackers to maintain access without triggering immediate suspicion, remaining hidden and undetected for an extended period. This strategic approach enables threat actors to operate without being noticed while keeping their initial access.
Unmasking Shadow Tokens: 3 Real-World Examples
We’ve examined how threat actors can establish persistence on three primary services widely utilized by developers: Docker Desktop, GitHub Desktop, and Google mobile apps.
- Docker Desktop: DockerHub is one of the most popular container registry services developers use worldwide. DockerHub also has a desktop app that offers a convenient GUI interface for Docker services.
- GitHub Desktop: GitHub is the leading platform for software development and version control among developers. Similar to DockerHub, GitHub also has a desktop app that provides a GUI interface that simplifies Git actions, such as pushing and pulling new code.
- Google mobile apps: apps like Drive, Docs, and Gmail allow users to use Google services directly from their mobile devices.
Gaining persistence to DockerHub account by abusing undocumented API
After gaining an initial foothold on a user account, a threat actor may want to push malicious images to DockerHub and infect the production environment with malicious code or leak private images.
When a threat actor attempts to carry out these actions on a 2FA-enforced user account using only the user’s username and password, they will encounter an error message directing them to create a Personal Access Token.
Upon generating a DockerHub Personal Access Token (PAT), users will receive an email notification with additional information about the newly created token, including the owner’s username and a short description. This email serves a dual purpose: confirming the successful generation of the token and serving as a security alert.
To further manage their PATs, DockerHub provides a security panel, which contains additional information such as the last usage of each token, its creation date, the current status (active or inactive), and greater visibility of all tokens across different systems.
Hence, threat actors are cautious not to create new access tokens, as this action would trigger immediate notifications to the user, alerting them to potential malicious activity within their DockerHub account.
We uncovered an undocumented API that malicious actors could exploit to generate long-lived personal access tokens. The alarming aspect of this vulnerability is that it bypasses any notification mechanisms and remains hidden from the user’s view in the security panel, making these access tokens undetectable and impossible to revoke. Those tokens will still be valid even if a user takes precautionary measures such as changing passwords or enabling two-factor authentication (2FA).
Research Flow and Discovery
Our journey began with a clear goal in mind: to achieve complete user persistence within DockerHub, all while avoiding detection. The initial step involved mapping out the services that could authenticate with DockerHub accounts. One such entry point was “Docker Desktop”.
We started analyzing Docker Desktop and its REST APIs to learn how its GUI communicates with DockerHub itself.
While investigating, we noticed a single API that returned a new Personal Access Token – without email notification, without being displayed in the security panel, nothing.
After receiving this access token – we checked its privileges by sending a POST request to: hub.docker.com/v2/users/login
In response, we got a new JSON Web Token (JWT) with full admin privileges. By leveraging this API, we were able to craft a shadow token that allowed us to gain undetectable and irrevocable access tokens.
The potential damage can be huge – such a method could allow a long-lasting supply-chain attack that compromises the artifacts deployed to the users’ production environment!
Upon discovering this issue, we promptly provided DockerHub with comprehensive details that addressed this vulnerability. Before the resolution, utilizing this API did not result in any record within the security panel, thus allowing the creation of shadow tokens. However, following the resolution, DockerHub has implemented an enhanced security dashboard featuring a Source label for each Personal Access Token (PAT), allowing users to track every PAT, including those originating from the desktop-generate API.
These sources are categorized as follows: MANUAL for PATs created manually and AUTO-GENERATED for those generated on behalf of Docker Desktop.
Before the vulnerability fix:
After the vulnerability fix:
Gaining persistence to GitHub using OAuth apps
GitHub allows users to create various types of tokens, each with a distinct set of permissions tailored to their needs. Like DockerHub, users receive a notification confirming the new token creation once they generate a token – making it less appealing for threat actors who want to stay undetected.
In addition to tokens, GitHub OAuth apps enable third-party applications to obtain designated permissions (scopes) on users’ accounts. GitHub Desktop is a popular example that facilitates repository management through a desktop application. These third-party apps follow the Authorization Code Flow. Let’s continue with the GitHub Desktop app as an example:
- Users are redirected to continue the process in their web browser after selecting the option to sign in using GitHub Desktop.
- The user is presented with a list of permissions the web application will grant upon logging in and providing consent.
- Once the user grants consent, they are redirected to the application using an
x-github-client://protocol handler with a one-time authorization code. - The application forwards the user’s authorization code, Client ID, and Client Secret to the GitHub OAuth authorization server.
- In response, GitHub returns a GitHub OAuth token (
gho_…) and the granted scopes, allowing the third-party application to access the approved resources on the user’s GitHub account.
The default granted scope for the GitHub desktop is full permission to repositories, user information, and workflows.
What makes this aspect appealing to threat actors is how they can modify these scopes to any other OAuth scopes. Imagine a threat actor with a specific objective: to establish persistent access within GitHub while maintaining a low profile. In this scenario, their goals extend beyond merely accessing repositories; they aim to become organization admin and gain full access to the user’s current organizations. Malicious actors can craft their authorization request, altering the scopes to include the necessary permissions for their scheme.
Firstly, we can craft an OAuth temporary code by sending a POST request to /login/oauth/authorize with the desired scopes.
Afterward, we create a new GitHub OAuth token using the temporary code we got from the last response by sending a POST request to /login/oauth/access_token.
The existing GitHub OAuth app will be updated with organization admin permission.
While known and trusted OAuth apps may provide users with a sense of security, the reality is that malicious actors can adeptly exploit these very apps to achieve silent persistence. For example, such a scenario could escalate by using the newly created OAuth token as follows:
- List all organizations with list organization API.
- Create a new private repository in the target organization with create repository API.
- Invite a new repository collaborator using repository collaborator API with admin privileges on the repository.
By inviting a threat actor-controlled account as an outside collaborator to this newly created repository, the threat actor can exfiltrate the organization’s GitHub Action secrets over time without being noticed by other members.
Gaining persistence in Google Workspace using OAuth apps
While investigating, we started to look at Google mobile apps and how they stay authenticated for an extended period. Let’s use the Gmail app as an example; to stay connected, it follows the Authorization Code Flow, the same as the GitHub OAuth flow. After completing the flow, Gmail sends a POST request to oauth2.googleapis.com/token, and the server responds with authentication details.
By using googleapis.com/oauth2/v1/tokeninfo, we can validate that the refresh token has an OAuthLogin scope with no expiration date.
Gmail then uses the Long-Lived refresh token to generate short-lived tokens with different OAuth scopes. Then, it sends a POST request to
oauthaccountmanager.googleapis.com/v1/issuetoken containing the refresh_token and the requested OAuth scopes. In response, a new token is returned with the requested permissions.
Threat actors can abuse this endpoint to request tokens for a wide range of Google services while impersonating a mobile app that is requesting a token. This OAuth-generated token may bypass policies that force users to reauthenticate their accounts or policies that destroy the current authenticated session.
DockerHub Disclosure
Our team diligently identified a potential security weakness within DockerHub’s infrastructure and initiated a responsible disclosure on June 7, 2023. Docker responded quickly, and through collaborative efforts with their support and security team, the vulnerability was successfully addressed by enhancing the security panel to document both manually created tokens and those generated by Docker Desktop.
Summary
Our daily interaction with a variety of platforms generates new tokens and authorizes a variety of services. Password managers assist us in storing these credentials securely, but system administrators and CISOs need to take proactive measures to prevent unauthorized access. It is important that they approach every day with the assumption that an account could have been compromised and used by malicious actors to gain long-lasting persistence.
To deny such access and possibly prevent malicious attacks, organizations could apply the next security measures:
- Scanning tokens across the entire software delivery pipeline – code, build, containers, cloud
- Revoking unused tokens
- Implementing regular password changes
- Enforcing 2FA for all users
- Continuously monitoring authorized OAuth apps and audit requested permissions
- Utilizing a password manager solution for secure access to privileged tokens
Learn more about our tool for secrets detection or book a demo.
Originally published: October 3, 2023
