Speed is the name of the game in the modern software development life cycle (SDLC). But in the race to deliver features, it’s easy to overlook small mistakes that can have catastrophic consequences, like accidentally committing API keys, tokens, or passwords to your source code repos.
For DevOps engineers, this isn’t a hypothetical risk. It’s a daily reality that can lead to data breaches, compromised infrastructure, or service abuse. So how does this happen, and how can we prevent it? Fortunately, there are well-established methods to detect, remove, and prevent these leaks with tools and workflows that don’t get in the way of productivity.
This guide breaks down how to detect and remove leaked secrets from code repos without getting in the way of productivity and the fast-paced SDLC we’ve come to know and, well, deal with.
Why Secrets Leaks Happen, and Why They’re So Dangerous
Before jumping into detection, it’s important to understand the scope of the problem.
Leaked secrets in code repos are typically authentication credentials. Things like API keys, OAuth tokens, SSH private keys, cloud service credentials (AWS, Azure, GCP), database usernames/passwords, and webhook secrets that get left behind for any number of reasons. Some of the main ones include:
- Hardcoding for testing convenience
- Sharing across teams via config files
- Misconfigured .gitignore files
- CI/CD logs or pipelines storing secrets in plaintext
Once committed, even in private repos, secrets can be accessed by insiders, external collaborators, or attackers who gain access via lateral movement. If pushed to a public repo, they can be discovered within minutes by threat actors using automated scanners.
And when those secrets grant access to undocumented APIs or shadow services, the breach surface grows exponentially, without you even knowing it.
Step-by-Step: How to Detect Leaked Secrets in Code Repositories
Catching leaked secrets early is critical, but it’s not always straightforward. Secrets can hide in obscure corners of your repositories, and they might not even look like secrets at first glance. Some are disguised in config files, embedded in code for convenience, or lost in layers of CI/CD automation.
For DevOps teams, detection must be continuous, automated, and deeply integrated into your workflows. Below are three essential steps to uncover leaked API keys, tokens, and passwords before they can be exploited.
Step 1: Scan the Entire Repository, Including History
One of the most common mistakes is scanning only the latest version of a file. But secrets are often buried deeper. They can be in a commit from six months ago, a deprecated branch, or a forked copy that nobody remembered existed. That’s why it’s essential to scan your entire repository history.
Look for secrets detection tools that support full Git history scanning. These tools can analyze the entire repo, including all tags and branches, using pattern matching, entropy analysis, and even leverage AI models to surface both obvious and subtle leaks. Once scanning is complete, you’ll get a comprehensive view of where secrets are hiding, along with risk context and remediation options.
To operationalize this, scan every active and archived repo in your org. Schedule automated scans to run regularly – daily or weekly – and monitor changes over time. This is your first line of defense against secrets that have already been exposed.
Step 2: Integrate Secrets Detection into Your CI/CD Pipeline
Even with regular scans, things slip through the cracks, especially in fast-moving teams. That’s why detection must be embedded directly into your CI/CD processes. When secrets scanning is part of your pull request checks, build gates, and developer workflows, you catch problems before they hit production.
Start with pre-commit hooks to prevent secrets from ever reaching version control. Tools like pre-commit, gitleaks, or Checkmarx’s AI Secure Coding Assistant provide real-time, AI-powered feedback in a developer’s IDE—building on the Checkmarx IDE plugin to offer enhanced guidance through an AI subscription. For GitHub users, the open-source Vorpal tool surfaces these results natively in pull requests. Then, enforce scanning in your CI/CD pipeline using GitHub Actions, GitLab CI, Jenkins, or Bitbucket Pipelines. This allows you to block builds if sensitive data is detected and automate alerts to the relevant teams.
The result is a shift-left approach to security where secrets issues are resolved as they’re introduced, not after they’ve shipped.
Step 3: Inventory and Prioritize Detected Secrets
Finding secrets is one thing. Understanding which ones matter is another. Not every high-entropy string is a real credential, and not every real secret is high-risk. That’s why context and prioritization are key.
Start by validating whether each detected secret is active. Is it a live AWS key? An expired test token? A placeholder in a sample config? Tools that offer contextual and reachability analysis help you filter out noise and focus on what’s exploitable.
Then, tag each secret with its type (e.g., API key, database password), location (source file, config, test folder), and sensitivity (dev vs. production). This gives you a clear map of your exposure and helps you assess business impact. With a complete inventory, you can decide what to rotate, what to revoke, and what to monitor more closely going forward.
How to Remove Leaked Secrets and Remediate
Detecting a leaked secret is only the beginning. What you do next, and how quickly, can determine whether it turns into a security incident or just a close call. Once a secret is exposed, it must be treated as compromised. The priority is to eliminate its usefulness to an attacker and prevent further exposure. This part of the process is all about speed, precision, and clean-up.
Step 1: Revoke and Rotate Secrets Immediately
Once you’ve identified an active secret in your codebase, the first step is to revoke and replace it without delay. Waiting even for a few hours can be the difference between staying secure and watching a breach unfold in real time.
For API keys and tokens, log into the provider’s admin interface (AWS, Slack, GitHub, Stripe, etc.) and immediately disable or regenerate the credential. For database credentials, change the passwords, update the configurations across all services that rely on them, and restart any dependent connections to apply the change. If SSH keys were exposed, remove them from all authorized_keys files and replace them with new ones across affected systems.
Revoking the secret is only part of the response. You also need to check your logs for signs of suspicious activity: unauthorized access, failed authentication attempts, unusual API usage, etc.. If any is found, begin incident response procedures.
At this stage, use your secrets manager to handle the rotation securely and consistently. A tool like AWS Secrets Manager or HashiCorp Vault ensures new credentials are safely stored, distributed, and versioned, and that old ones can’t be reused.
Step 2: Purge the Secret from Git History
Once the active secret is neutralized, you need to clean your repository. Simply deleting the secret from a config file won’t cut it. Git’s history is persistent, and so are the risks.
After detecting leaked secrets with tools like Checkmarx Secrets Detection, use tools like git filter-repo (preferred for performance and flexibility) or BFG Repo-Cleaner to scrub the secret from the entire history of your repository. This includes all branches, tags, and remotes. After cleaning, you’ll need to force push the updated history and coordinate with your team to re-clone or reset their local repositories.
This process is invasive, but necessary. As part of the cleanup, document who had access to the compromised secret before rotation and audit any exposed systems. If that secret was used in production, consider rebuilding or redeploying affected services to ensure no residual access paths remain.
Think of this as an opportunity to reset trust not just in credentials, but in your team’s ability to respond rapidly and effectively.
Step 3: Move to Secure Secrets Storage
The final step is about prevention. Once you’ve removed the hardcoded secret and rotated it, make sure the new one doesn’t follow the same path to exposure. That means removing all secrets from source code entirely and relying instead on a centralized secrets manager.
Secrets management tools offer everything you need for long-term hygiene: encryption at rest and in transit, granular access control, automatic rotation policies, and detailed audit logs. By using a service like Azure Key Vault, Doppler, or HashiCorp Vault, you can reduce the burden on developers while improving security posture across the board.
This also means refactoring your code and deployment process. Applications should never read secrets directly from files under version control. Instead, they should pull credentials from environment variables, injected secrets at runtime, or secrets stores integrated with your deployment toolchain (like Kubernetes secrets or cloud-native services).
Finally, define strict policies for access control. Segment secrets by role and environment, limiting access to only what’s absolutely necessary. Development credentials should never be reused in production, and audit trails should be monitored regularly.
Prevent Future Leaks: Build a Culture of Secrets Hygiene
Stopping leaks is about habits just as much, if not more, than it is about tools. To keep secrets out of code for good, you need a culture shift backed by smart automation and developer-friendly workflows.
Start with developer enablement. Integrate Checkmarx AI Secure Coding Assistant, which brings real-time secrets detection and secure coding guidance directly into popular IDEs like VS Code, JetBrains, and Eclipse. Engineers get actionable alerts before they even hit commit. Make secrets security part of onboarding, and embed these checks in your team’s code review workflows.
Automate policy enforcement. Use Checkmarx Policy Management to define organization-wide guardrails as code. Policies can automatically block commits or CI/CD builds that include hardcoded secrets, misconfigured environment files, or out-of-policy vulnerabilities—preventing risky code from progressing through the pipeline.
Clean up your infrastructure hygiene. Use tightly scoped service accounts. Split secrets by environment, and never reuse prod keys in dev. And always scrub secrets from logs, telemetry, and error messages. If a secret isn’t visible, it can’t leak.
Code-Based API Discovery with Checkmarx API Security and DAST
In modern software development, APIs are integral, but they can also introduce significant security risks—especially when undocumented or outdated APIs (often called shadow or zombie APIs) are left unmanaged.
Checkmarx addresses these challenges with a dual approach:
- Checkmarx API Security uncovers APIs directly from your source code, even if they’re undocumented.
- Checkmarx DAST (Dynamic Application Security Testing) scans live APIs at runtime to find vulnerabilities missed by static analysis alone.
Key Capabilities of Checkmarx API Security:
- Automatic API Discovery: Scans source code to identify all API endpoints, providing complete visibility into your API landscape.
(Learn more)
- Shadow and Zombie API Detection: Flags undocumented or outdated APIs that attackers could exploit.
(Introducing Checkmarx API Security)
- Prioritized Remediation: Correlates vulnerabilities with specific APIs, helping teams focus on the highest business risks.
Enhancing API Security with Checkmarx DAST:
- Runtime Vulnerability Detection: DAST tests live APIs (REST, SOAP, gRPC) to uncover authentication gaps, misconfigurations, and other runtime issues.
(Checkmarx DAST Overview)
- Unified Risk Reporting: Combines findings from static (API Security) and dynamic (DAST) scans into a single view for easier prioritization and remediation.
(Ultimate Guide to API Security)
By combining static code-based discovery with dynamic runtime testing, Checkmarx provides full lifecycle API protection—helping you detect hidden risks, fix vulnerabilities faster, and harden your APIs from development through production.
The Hidden Cost of Secrets Sprawl
To put this issue into perspective, I’ll leave you with the eye-opening story of Bill Demirkapi, featured in WIRED. As a college student, he uncovered more than 100,000 hardcoded secrets – including AWS keys, tokens, and passwords – just by scanning public GitHub repositories.
His scans revealed secrets left behind by major organizations, schools, healthcare providers, and financial services. Many were still active, and some granted full access to sensitive systems. What’s most alarming? These secrets were often buried deep in repo history or pushed accidentally by inexperienced developers.
In Demirkapi’s words, “Hardcoded secrets are the equivalent of leaving your house key under the welcome mat and announcing it on social media.”
This story underscores just how easy it is to miss the warning signs and how important it is to take secrets management seriously at every step of the SDLC.
Stop Leaks Before They Become Breaches
Detecting and removing leaked secrets is both an engineering and security challenge, but it’s one DevOps and AppSec teams are well-positioned to manage. With the right tools and workflows, you can secure pipelines without slowing delivery.
Checkmarx One provides a platform approach to secrets and API security, including:
Secrets Detection:
- Pre-commit scanning to catch exposed credentials before code is committed.
- Git repository scanning (including full history) to detect secrets already introduced.
- CI/CD integration to block builds containing leaked credentials.
API Security:
- Code-based API discovery to find shadow, zombie, and undocumented APIs from source code.
- Contextual mapping between secrets and exposed APIs to assess exploitability.
- Prioritized remediation based on sensitivity and business risk.
DAST for APIs and Web Applications:
- Dynamic scanning of live APIs (REST, SOAP, gRPC) to uncover runtime vulnerabilities.
- Vulnerability validation across both code (static) and runtime (dynamic) phases.
- Unified reporting that consolidates SAST, SCA, API Security, and DAST findings.
Application Security Posture Management (ASPM):
- Centralized risk management across all AppSec activities.
- Real-time visibility into secrets exposure, API risk, open-source vulnerabilities, code weaknesses, and runtime issues.
- Streamlined governance and compliance reporting.
By combining static analysis, dynamic scanning, and unified risk context across your SDLC, Checkmarx enables you to detect leaked secrets early, secure APIs, and reduce the risk of breaches—without impacting developer velocity.
With these capabilities, your team can shift security left without slowing development down, proactively preventing secret sprawl, improving API security, and hardening your DevOps pipeline end to end.
Take the next step: Explore Checkmarx Secrets Detection today.
Don’t Spill Any Secrets
Checkmarx Secrets Detection stops breaches before they begin without slowing down developers.