Secrets Detection

It is important to identify any private or sensitive information that could potentially be used in an attack or data breach. This includes credentials (such as usernames or passwords that can grant a user or system access to resources or services), API keys or tokens (unique identifiers to authorize access to an API or web service), private keys or encryption keys (such as those used to encrypt/decrypt sensitive data or secure communication protocols), certificates (codes used to establish trust between two entities, such as between a server and a client), and private endpoint/webhook URLs.

Secrets can be exposed in a wide variety of places, including source code, configuration files (e.g., IaC files), CI/CD pipelines, developer productivity tools, collaboration tools, wikis, and generative AI tools. To minimize potential vulnerabilities, any secrets exposed in any non-private location must be identified, removed, and changed.

An effective secrets detection algorithm must exhibit high precision and high recall. High precision means a low number of false alerts. In other words, high precision means that a high percentage of identified secrets are actual secrets that are at risk of exposure. High recall means that a low number of secrets are missed. Given that even one undetected credential can introduce a large amount of risk, it is often considered preferable to have to investigate some false alerts to ensure that no real exposed secrets are overlooked.

Any time an exposed secret is discovered, it is advisable to immediately revoke/replace the secret to remove the risk of the secret being used in an attack or breach. This is especially important if the secret was exposed in a public platform (such as GitHub), because once posted it might never be possible to completely remove it. And, of course, do not repeat the mistake and include the new secret in an exposed manner.

There are many techniques that can prevent the exposure of secrets; when developers and DevOps professionals are aware of the dangers and these solutions, the incidence of exposed secrets should drop sharply. One example is storing secrets in environment variables or separate files instead of hardcoding them, and then using a .gitignore file to ensure that files containing secrets are not synced to a repository. Other possibilities are encrypting all secrets, using a dedicated secrets management tool, and implementing two-factor authentication (2FA) for any repositories that still might contain secrets. Of course, automated scanning technology can detect exposed secrets and prevent disaster when mistakes are made.

Developers often forget to remove hard-coded credentials, keys, private webhook URLs, and other sensitive secrets from their code when completing a development task. Scanning for exposed secrets upon code commit, for example, automates the process of identifying any exposed secrets, freeing up developers to focus on core development tasks.