Human-in-the-Loop Cybersecurity: Designing HITL Controls for AI and AppSec

What Is Human-in-the-Loop (HITL) in Cybersecurity?

Human-in-the-loop (HITL) is an approach in which a human actively participates in an automated or AI-driven process rather than delegating every decision to the machine. Humans may review recommendations, resolve uncertainty, approve high-impact actions, provide feedback, or take over when automation reaches a defined boundary.

In cybersecurity, human-in-the-loop AI combines the speed and scale of automation with human security judgment. An AI system might analyze thousands of alerts, identify a likely vulnerability, propose a remediation, or recommend an incident-response action. A human then becomes part of the decision process when additional business context, accountability, or authorization is required.

HITL should not mean that a person manually reviews every automated action. That would eliminate much of the value of automation. Effective HITL security instead defines where human judgment materially changes the risk of a decision.

For example, an AI system may automatically dismiss duplicate low-risk alerts but require a security analyst to approve disabling a production account. An AppSec agent may identify and propose a code fix automatically but route a security-sensitive architectural change through developer review.

The goal is therefore not maximum human involvement. It is appropriate human authority at the right point in the security workflow.

Who Needs Human-in-the-Loop Cybersecurity?

Human-in-the-loop cybersecurity is most relevant when automated or AI-driven systems can make decisions or take actions with meaningful security, operational, financial, or compliance consequences.

It is particularly important for AppSec and security teams using AI to prioritize or remediate vulnerabilities. Automation can reduce repetitive analysis, but security professionals may still need to review ambiguous findings, validate business context, approve risk exceptions, or evaluate changes that could affect production applications.

Development teams using AI coding assistants and coding agents also need clear human-control boundaries. AI can generate or modify code, introduce dependencies, change configurations, and propose remediation. Human review becomes more important as the change becomes more security-sensitive, complex, privileged, or difficult to reverse.

Security operations and incident-response teams can use automation to correlate alerts, gather evidence, and recommend containment. Human intervention remains valuable when actions could disable critical accounts, isolate production systems, block legitimate business traffic, or materially affect operations.

DevOps and platform engineering teams need HITL where agents or automation interact with repositories, CI/CD pipelines, infrastructure, cloud environments, credentials, or production systems. These workflows require clear boundaries around what can happen automatically and what requires additional authorization.

CISOs, AppSec leaders, and governance teams need human oversight where organizations must demonstrate accountability for AI-assisted security decisions, risk acceptance, policy exceptions, and high-impact remediation.

A useful rule is that the need for HITL increases when one or more of the following are true: the action is difficult to reverse, the blast radius is large, sensitive data or privileged systems are involved, AI confidence is insufficient, business context materially changes the decision, or regulations and internal policy require an accountable human decision-maker.

Core Principles of Human-in-the-Loop Security

An effective HITL security system combines automation and human judgment deliberately rather than adding manual approval at the end of an otherwise autonomous process.

Human-Machine Collaboration

Automation is strongest at high-volume work such as correlation, pattern detection, repetitive analysis, evidence gathering, and generating candidate actions. Humans add contextual judgment, organizational knowledge, accountability, and the ability to recognize situations that fall outside the automation’s assumptions.

The strongest HITL workflows therefore assign each task to the participant best suited to perform it rather than treating human and machine analysis as interchangeable.

Explicit Decision Authority

The workflow should define which decisions AI can make independently and which require a human.

Low-impact, reversible actions may be automated. Actions involving sensitive data, production systems, privileged identities, critical vulnerabilities, destructive commands, or broad business impact may require explicit approval.

Risk-Based Intervention

Human review should be triggered by risk, not by arbitrary workflow stages.

Relevant factors can include:

• confidence or uncertainty,
• blast radius,
• reversibility,
• asset criticality,
• regulatory impact,
• privilege level,
• exploitability,
• and whether the action affects production or sensitive data.

Explainability and Evidence

A reviewer needs enough evidence to make a meaningful decision.

A useful approval should explain what the system intends to do, why, which assets are affected, what evidence supports the recommendation, and what the likely impact will be.

Feedback and Continuous Improvement

Human decisions can improve future automation by revealing false positives, missing context, policy gaps, and recurring exceptions.

The feedback loop may improve rules, policies, playbooks, training data, or evaluation processes. It does not require that the AI model itself retrain continuously during operation.

Human-in-the-Loop vs. Human-on-the-Loop vs. Fully Automated Security

Human oversight can be designed at different levels.

Human-in-the-loop (HITL) means the automated system pauses at defined points and requires human input before proceeding.

Human-on-the-loop (HOTL) allows the automated system to operate independently while humans supervise the process and can intervene, stop, or override it.

Fully automated security allows the system to detect, decide, and act without human intervention during normal operation.

The right model depends on risk.

Routine, reversible actions with well-understood outcomes may be suitable for full automation. High-volume security workflows may work better with humans supervising exceptions. High-impact or irreversible actions often benefit from explicit HITL approval.

A mature security program will usually use all three models, rather than selecting one approach for every workflow.

How Human-in-the-Loop Cybersecurity Works

A well-designed HITL cybersecurity workflow usually follows six steps.

1. Automation Detects or Analyzes an Event

The security system identifies an alert, vulnerability, suspicious behavior, policy violation, risky code change, or proposed agent action.

2. The System Builds Context

Automation gathers the evidence needed to make a decision: vulnerability data, identity context, affected applications, reachability, threat intelligence, code changes, prior activity, or the actions an AI agent intends to perform.

3. Risk and Policy Determine the Decision Path

Rules decide whether the system can proceed automatically or whether human input is required.

A low-risk event may be handled autonomously. An uncertain or high-impact decision enters a human-review queue.

4. The Human Reviews the Evidence

The reviewer may approve the proposed action, reject it, modify it, request additional evidence, or escalate it.

The important point is that the human should review evidence and consequences, not simply an AI-generated recommendation.

5. The System Executes Within Defined Permissions

Approved actions should remain constrained by least privilege and technical policy. Human approval should not convert an overprivileged system into an unrestricted one.

6. The Decision and Outcome Are Recorded

The system records what was proposed, what evidence was shown, who approved or rejected it, what action occurred, and what happened afterward.

That information improves auditability and helps teams refine policies and automation over time.

Human-in-the-Loop Cybersecurity and AppSec Use Cases

Security Operations and Alert Triage

AI can correlate high volumes of alerts, collect evidence, summarize incidents, and rank likely threats. Analysts can then focus on ambiguous or high-impact cases rather than manually processing every signal.

Incident Response

Automation can recommend containment or remediation steps, while human approval remains appropriate for actions such as isolating critical systems, disabling important accounts, blocking business traffic, or changing production configurations.

Threat Hunting

AI can generate hypotheses and search large datasets quickly. Human threat hunters provide adversarial thinking, environmental context, and validation before a hypothesis becomes an incident conclusion.

Vulnerability Triage and Remediation

In AppSec, automation can analyze findings, reachability, exploitability, code context, and policy to help prioritize which vulnerabilities deserve attention. Humans remain valuable when the decision depends on business context, architecture, risk acceptance, or potentially disruptive remediation. Checkmarx has previously described this combination as one of the important roles of humans in AI-powered AppSec.

AI-Generated Code and Coding Agents

Coding agents can generate or modify software much faster than traditional development workflows. HITL can provide review boundaries around sensitive code changes, new dependencies, infrastructure modifications, or privileged commands.

Human approval should complement, not replace, SAST, SCA, secret detection, policy enforcement, testing, and repository controls.

AI Agents and Tool-Connected Workflows

Human approval becomes particularly important when an AI agent can call APIs, execute commands, modify files, access sensitive data, or interact with production systems.

OWASP’s current agentic AI guidance recommends explicit human confirmation for high-impact actions while combining it with allowlisted actions, context validation, and other technical controls.

How to Implement HITL in Enterprise AppSec

For enterprise AppSec organizations, the objective should not be to insert a person into every automated security process. At Checkmarx-customer scale, that would recreate the manual bottlenecks AI and automation are supposed to reduce.

A more practical model is to decide which AppSec decisions can be automated, which need human supervision, and which require explicit human approval.

1. Inventory the Security Decisions Being Automated

Start with decisions and actions rather than tools.

Typical AppSec examples include:

• classifying a security finding;
• determining whether a vulnerability is reachable or exploitable;
• changing a vulnerability’s risk state;
• prioritizing remediation;
• proposing a code fix;
• changing application code;
• approving a dependency upgrade;
• enforcing a CI/CD policy;
• granting a policy exception;
• accepting application risk;
• or allowing an AI coding agent to continue after a security issue is found.

This helps teams distinguish analysis from authority. An AI system may be allowed to analyze a vulnerability automatically without automatically receiving authority to modify production-bound code.

2. Assign an Automation Model Based on Risk

Use three control modes instead of treating HITL as a binary choice.

How to Implement HITL in Enterprise AppSec

For enterprise AppSec organizations, the objective should not be to insert a person into every automated security process. At Checkmarx-customer scale, that would recreate the manual bottlenecks AI and automation are supposed to reduce.

A more practical model is to decide which AppSec decisions can be automated, which need human supervision, and which require explicit human approval.

1. Inventory the Security Decisions Being Automated

Start with decisions and actions rather than tools.

Typical AppSec examples include:

• classifying a security finding;
• determining whether a vulnerability is reachable or exploitable;
• changing a vulnerability’s risk state;
• prioritizing remediation;
• proposing a code fix;
• changing application code;
• approving a dependency upgrade;
• enforcing a CI/CD policy;
• granting a policy exception;
• accepting application risk;
• or allowing an AI coding agent to continue after a security issue is found.

This helps teams distinguish analysis from authority. An AI system may be allowed to analyze a vulnerability automatically without automatically receiving authority to modify production-bound code.

2. Assign an Automation Model Based on Risk

Use three control modes instead of treating HITL as a binary choice.

The goal is to reserve human attention for decisions where it creates measurable security value.

A Practical HITL Model for Enterprise AppSec

3. Define the Decision Owner

A HITL control is ineffective if everyone can approve but nobody owns the decision.

For enterprise AppSec, ownership may differ by decision:

• developers approve the code they merge;
• AppSec reviews vulnerability risk and security exceptions;
• platform teams own pipeline and infrastructure controls;
• application owners supply business context;
• security leadership defines enterprise risk thresholds.

The workflow should make the accountable role explicit.

4. Give Humans Independent Security Evidence

This is one of the most important principles for Checkmarx’s ICP.

Do not ask a reviewer to approve an AI recommendation using only the AI’s own explanation.

A useful AppSec evidence package can include:

• the original finding;
• affected code;
• data or execution path;
• reachability and exploitability context;
• existing mitigating controls;
• application criticality;
• relevant policy;
• proposed code diff;
• dependency path or package information;
• verification results.

The reviewer should be able to understand why the security decision is justified, not merely whether the AI sounds confident.

This matches the direction of Checkmarx Triage Assist, which provides reviewable analysis using reachability, exploitability, code context, risk context and relevant mitigating controls. AI-driven state changes remain visible and traceable in Checkmarx One.

5. Keep Technical Boundaries Even After Approval

Human approval should not convert an AI agent into an unrestricted administrator.

Continue to enforce:

• least privilege;
• limited tools and scopes;
• branch and repository protections;
• policy controls;
• allowed actions;
• credential boundaries;
• and separation between development and production environments.

The principle is:
Human approval authorizes a specific action, not unlimited autonomy.

6. Put Human Decisions Where Teams Already Work

HITL adds friction when reviewers must leave their normal workflow to understand and approve an AI action.

For AppSec organizations, useful decision points are typically:

• IDEs for developer-led remediation;
• pull requests for code review;
• Risk Orchestration for centralized vulnerability decisions;
• CI/CD for policy and exception workflows;
• security dashboards for program-level governance.

Checkmarx’s current product model already reflects multiple levels of human involvement rather than one universal HITL model. Developer Assist supports developer-led workflows in which developers review and accept proposed changes, while supported autonomous coding-agent workflows can perform detection, remediation and verification with different levels of automation.

Triage and Remediation Assist provide an even clearer example: AI Triage can operate through defined automated workflows, while Remediation Assist produces suggested fixes that remain subject to developer review and approval; proposed fixes are not automatically merged.

7. Measure Whether Human Oversight Is Actually Working

Do not measure HITL success by the number of approvals completed.

Measure whether human involvement improves security decisions.

Useful metrics include:

• time to security decision;
• approval and rejection rates;
• human override or reversal rate;
• false approvals;
• false rejections;
• remediation acceptance;
• first-attempt fix success;
• time to remediation;
• number of findings waiting for human review;
• approval latency by risk tier;
• and recurring reasons humans override automation.

Checkmarx Triage and Remediation Assist already provides a useful model for this approach: Checkmarx One records AI-driven changes and provides usage analytics around triage, remediation, adoption and outcomes rather than treating automation volume alone as success.

Benefits of Human-in-the-Loop Security

The strongest advantage of HITL is that organizations do not have to choose between manual cybersecurity and unrestricted automation.

Automation can process far more data than human teams can review manually, while humans can focus on decisions where context, ambiguity, accountability, or impact make judgment valuable.

HITL can also improve:

Risk prioritization. Humans can incorporate business impact that may not exist in technical telemetry.

Accountability. Sensitive decisions can retain a clearly identified decision owner.

Safe automation adoption. Teams can automate gradually while retaining stronger intervention points for unfamiliar or high-risk workflows.

Continuous improvement. Human overrides and corrections reveal where automation, detection, or policies need refinement.

The benefit is not simply “humans catch AI mistakes.”

The more important outcome is a security architecture in which automation has clearly defined authority and escalation boundaries.

HITL Security Risks and Limitations

Human involvement improves many security workflows, but it should not be treated as an inherently secure control.

Scalability and Latency

Requiring approval too frequently creates bottlenecks and eliminates the benefits of machine-speed security.

Approval Fatigue

If people receive too many approvals, they may begin to accept recommendations automatically. At that point, the workflow technically contains a human but no longer gains meaningful human judgment.

Automation Bias

Reviewers may over-trust an AI-generated explanation because the system appears authoritative or has been correct in the past.

Incomplete Context

A human can only make a good decision from the information available. A confident recommendation based on incomplete security context can lead both the AI and reviewer to the wrong conclusion.

Adversarial Manipulation of Human Approval

HITL itself can become an attack surface.

Checkmarx Zero demonstrated Lies-in-the-Loop, also documented by OWASP as HITL Dialog Forging, in which attackers manipulate the information presented during an AI agent’s approval workflow so that a harmful operation appears benign.

This changes an important assumption:

“A human approval is only as trustworthy as the evidence and interface on which that approval is based. HITL should therefore be designed as a security control – not merely added as a checkbox.”

Darren Meyer, — Senior Security Researcher, Checkmarx Security Research

Best Practices for Secure HITL Implementation

Define Intervention Points by Risk

Document which actions can occur automatically, which require supervision, and which require explicit approval.

The higher the blast radius, privilege level, irreversibility, or data sensitivity, the stronger the human-control boundary should usually be.

Make Every Approval Specific

Avoid vague prompts such as “Allow this agent to continue?”

A useful approval should identify:

• the proposed action,
• affected resource,
• requested permission,
• relevant inputs,
• expected impact,
• and whether the action is reversible.

Provide Evidence Independent of the AI Narrative

Do not rely exclusively on the AI agent to describe why its own action is safe.

Where possible, support approval decisions with independent security signals such as scanner findings, policy results, verified diffs, dependency information, access context, or deterministic system telemetry.

This is particularly important in AppSec, where security engines and contextual risk evidence can validate an AI recommendation separately from the model’s explanation.

Maintain Least Privilege

Human approval should not give the AI unlimited authority.

Agents should have only the tools, scopes, credentials, and resources required for their task, even after approval.

Protect the Approval Channel

Approval dialogs, command previews, code diffs, and other evidence should be treated as security-sensitive interfaces. They should clearly distinguish trusted system information from content that an attacker or AI system can influence.

Preserve Auditability

Record the recommendation, evidence, human decision, resulting action, and final outcome.

This supports incident investigation, compliance, and continuous improvement.

Test the Human Control Itself

Red-team the HITL workflow.

Test whether manipulated explanations, excessive information, hidden commands, confusing interfaces, or repeated approvals can cause reviewers to make unsafe decisions.

Measure Decision Quality, Not Approval Throughput

A fast approval process is not necessarily an effective one.

Useful metrics include human override rates, false approvals, false rejections, decision time by risk tier, recurrence of overridden recommendations, and outcomes after approved actions.

Human-in-the-Loop in Application Security

AppSec is particularly well suited to risk-based HITL because the volume of machine-generated findings and code changes is increasing faster than human teams can review everything manually.

The goal should not be to put a developer or AppSec engineer in front of every vulnerability.

Instead:

• automation can identify and explain issues,
• risk context can determine which findings matter,
• AI can propose remediation,
• policy can constrain what happens automatically,
• and humans can retain authority where business context or high-impact changes require judgment.

Checkmarx’s current agentic AppSec direction reflects this broader shift toward machine-speed analysis and remediation backed by platform context. Developer Assist brings security guidance into developer workflows, while Triage & Remediation Assist uses contextual signals to prioritize findings and generate review-ready remediation.

But the security lesson from HITL is broader than any product workflow:

Do not rely on human approval as the only safeguard.

Strong AppSec HITL combines human review with independent security analysis, policy controls, least privilege, traceability, and evidence that helps the reviewer understand the actual security consequence of the decision.

How Checkmarx Supports Governed Agentic AppSec

Checkmarx’s agentic AppSec model illustrates why modern security does not have to choose between unrestricted automation and fully manual review. Different security activities can use different levels of automation based on their risk, repeatability, and need for human judgment.

Across Checkmarx One, AI-assisted workflows can help developers and AppSec teams analyze risk, apply policies, generate remediation, and surface portfolio insights while keeping enterprise controls, traceability, and accountable decision-making in the surrounding security program. The Checkmarx One Assist family is explicitly designed around different roles across the inner, middle, and outer loops of software delivery.

Triage and Remediation Assist provides a particularly clear example. Eligible SAST and SCA findings can receive automated AI-assisted triage based on defined criteria.

When remediation is required, Checkmarx can provide contextual suggested fixes, and supported GitHub projects can receive a separate fix pull request for developer review and approval. Automated workflows currently apply to AI Triage rather than AI Remediation, and suggested changes are not automatically merged.

This illustrates an important HITL principle for enterprise AppSec: the right amount of human involvement depends on the action. Evidence gathering and repeatable analysis can operate at machine speed, while higher-impact risk and code decisions preserve human accountability.

Conclusion

Human-in-the-loop cybersecurity is not simply a compromise between manual security and automation. It is a way to deliberately assign decision authority between humans and machines according to risk.

AI can provide speed, scale, correlation, and automation. Humans provide context, accountability, judgment, and the ability to question assumptions the automation may not recognize.

But human involvement should not be mistaken for an automatic security guarantee. Reviewers can be overloaded, biased by AI recommendations, given incomplete evidence, or even deliberately deceived.

The most effective HITL systems therefore secure both sides of the loop: the automated system that proposes or performs the action and the human decision process that governs it.

For cybersecurity and AppSec teams moving toward agentic AI, that is the more useful objective: not keeping a human involved everywhere, but keeping humans meaningfully in control where it matters most.