Breaking Down False Positives in Secrets Scanning

It’s critical for secrets scanners to detect all secrets lingering within code repositories and other shared resources, lest they lead to the leakage of sensitive access information that can be exploited for cyberattacks.

But equally critical, especially from the perspective of delivering a great developer experience – is avoiding false positives during secrets scanning. False positives lead to frustration for coders, since they force them to waste time investigating risks that don’t actually exist, and loss of trust in their security tools.

This is why it’s critical to understand what causes false positives during secrets scanning and what organizations can do about it. Keep reading for guidance as we explain how to balance effective secrets detection with developer experience and productivity.

What is secrets scanning?

Secrets scanning is the process of automatically detecting secrets – such as passwords, access tokens, and API keys – within an application’s codebase, log files, and other shared resources. Secrets scanners work by analyzing source code, configuration files, and other resources for strings that resemble sensitive access credentials.

Secrets scanning is a key component of a secure software development lifecycle because it helps ensure that sensitive access credentials don’t appear within code or other resources that may become accessible to third parties. According to the Thales Group, 83% of organizations experience at least one security incident per year due to hardcoded secrets (meaning secrets that are embedded within software resources rather than being handled through environment variables, secure secrets management tools, or other solutions). Secrets scanning is critical for mitigating this risk.

The scourge of false positives during secrets scanning

The ability to discover secrets of all types, wherever they may exist, is critical for maximizing security and ensuring secrets detection accuracy.

However, the ability to detect secrets reliably should not come at the expense of avoiding false positives – meaning instances where a secrets scanning tool identifies a string as a secret when in reality the string doesn’t contain sensitive data.

False positives during secrets scanning are problematic because they waste valuable time, harm the developer experience, and erode trust between development and security teams. Specifically, they lead to issues such as:

• Developer frustration: Having to address an alert about a non-existent risk can frustrate developers. Chasing down non-issues is not how most coders want to spend their time.
• Delayed software releases: Alerts about secrets force developers to pause operations while they investigate the issue. As a result, false positives can unnecessarily delay software release schedules.
• Lack of confidence in security tools: When secrets scanners generate high rates of false positives, they can leave developers wondering whether the tools are also prone to false negatives (meaning issues where a risk does exist but the tool misses it). In turn, they can undercut developers’ faith in security tools and processes.
• Harm to the DevSecOps model: The ability of organizations to practice DevSecOps – meaning the seamless integration of security into the software development lifecycle – hinges on minimizing friction between development processes and security processes. The opposite happens when secrets scanners waste developers’ time by generating false positive alerts.

In short, high rates of false positives during secrets scanning can erode developer productivity and reduce developer job satisfaction. It’s bad for developers, and bad for the organization as a whole.

What causes false positive alerts in secret scanners?

Now that we’ve discussed the pitfalls of false positives in secrets scanning, let’s look at what causes the issue. In most cases, the problem stems from one or more of the following factors.

Overly aggressive detection rules

Secrets scanners work by using pattern-matching techniques to determine which strings might be a password, access key, encryption token, or other type of credential. If the rules that govern pattern-matching are too aggressive, a scanner might end up incorrectly flagging non-sensitive data as a secret.

For example, strings that contain a seemingly random set of characters may be a password. But they could also be something like a UUID, which is not sensitive in most cases. If a scanner is configured to generate an alert for any series of random characters, it is likely to end up flagging many strings that are not actually passwords or other secrets.

Lack of context

Secrets scanning tools that don’t factor in context effectively can generate false positives. As a basic example, consider a configuration file that includes a comment like the following:

# Configure database access in this format: user:password@host:1234/database_url

This comment doesn’t present a risk because it doesn’t actually contain sensitive data. It just provides an example of how users would configure database access during an actual deployment. However, if the secrets scanner is not able to distinguish between a comment and an actual value, it may miss this nuance, leading to a false positive.

Simplistic detection techniques

Scanners that lack the ability to perform sophisticated analysis may experience high false-positive rates.

For instance, imagine a scanner that treats any string preceded by the phrase pass: as a password. This would make sense in some contexts because pass: does often precede passwords within configuration files. But there are plenty of instances where that’s not the case. The label could be part of application logic that reads an environment variable that is not actually hardcoded, for example.

Better secrets scanners rely on more accurate detection techniques to separate non-issues from those that actually pose a risk.

Redundant alerts

In some cases, scanning tools may generate multiple alerts for the same discovered secret. This isn’t a false positive, per se, if a hardcoded secret actually exists. But it can still harm the developer experience by needlessly increasing the total volume of alerts that developers have to address.

As an example, imagine a scanner that performs multiple passes on the same file, using a different detection technique each time. Imagine, too, that it generates a new alert each time it discovers a secret, even if the secret was already flagged by a previous alert. A better tool would consolidate the alerts so that developers only have to respond once to each discovered secret.

How to select an accurate secrets detection tool

To avoid the risk of undercutting the developer experience due to false positives, look for secrets scanning tools that provide the following:

• Customizable detection rules: The more development and security teams can customize detection rules, the more easily they can avoid false positive scans.
• Context-aware scanning: Scanners that evaluate the complete context in which a potential secret appears, rather than relying on cruder techniques like pattern-matching alone, tend to generate fewer false positives.
• Advanced detection algorithms: The best secrets scanners do more than just look for certain types of strings. They employ sophisticated algorithms that evaluate a number of factors before labeling something a secret.
• Smart alert management: The ability to track and consolidate alerts helps to avoid redundant notifications.

These are the guiding principles behind Checkmarx’s Secrets Detection, available as part of the cloud-native Checkmarx One application security platform. Learn more about Checkmarx Secrets Detection here.