Secrets Detection

It is important to identify any private or sensitive information that could potentially be used in an attack or data breach. This includes credentials (such as usernames or passwords that can grant a user or system access to resources or services), API keys or tokens (unique identifiers to authorize access to an API or web service), private keys or encryption keys (such as those used to encrypt/decrypt sensitive data or secure communication protocols), certificates (codes used to establish trust between two entities, such as between a server and a client), and private endpoint/webhook URLs.

Secrets can be exposed in a wide variety of places, including source code, configuration files (e.g., IaC files), CI/CD pipelines, developer productivity tools, collaboration tools, wikis, and generative AI tools. To minimize potential vulnerabilities, any secrets exposed in any non-private location must be identified, removed, and changed.

An effective secrets detection algorithm must exhibit high precision and high recall. High precision means a low number of false alerts. In other words, high precision means that a high percentage of identified secrets are actual secrets that are at risk of exposure. High recall means that a low number of secrets are missed. Given that even one undetected credential can introduce a large amount of risk, it is often considered preferable to have to investigate some false alerts to ensure that no real exposed secrets are overlooked.

Any time an exposed secret is discovered, it is advisable to immediately revoke/replace the secret to remove the risk of the secret being used in an attack or breach. This is especially important if the secret was exposed in a public platform (such as GitHub), because once posted it might never be possible to completely remove it. And, of course, do not repeat the mistake and include the new secret in an exposed manner.

There are many techniques that can prevent the exposure of secrets; when developers and DevOps professionals are aware of the dangers and available solutions, the incidence of exposed secrets drops sharply. Secrets can be stored in environment variables or separate files instead of hardcoding them (these files should be included in .gitignore to ensure that they are not synced to a repository). Another option is to encrypt all secrets, using a dedicated secrets management tool, and implement two-factor authentication (2FA) for any repositories that still might contain secrets. In all cases, automated scanning technology should be used to detect hardcoded secrets in source code and prevent them from being pushed to code repositories from where their leakage is more likely.

Developers often forget to remove hardcoded credentials, keys, private webhook URLs, and other sensitive secrets from their code when completing a development task. Scanning for exposed secrets upon code commit, for example, automates the process of identifying any exposed secrets, freeing up developers to focus on core development tasks.