Goals & Objectives

Have you defined clear goals and objectives for your AppSec program that are aligned with the risk appetite of your organization?

Guidance

Goals and objectives are often used interchangeably, but there are significant differences. A goal is a broad, overarching idea or vision of where you want to reach. Objectives are the steps or milestones that towards reaching the identified goals. Objectives are important because they keep the goal alive and are the smaller, time-bound steps to help getting towards the overall goal.
To be part of the corporate governance function of an organisation, the goals, and objectives of the AppSec program should be tied to the risk appetite of the organisation. For this purpose, the risk appetite of the organization's executive leadership needs to be captured and aligned with the goals and objectives of the AppSec program. The organization's leadership should vet and approve the set of goals.

AppSec Policies

Do you have an AppSec policy and standards in place?

Guidance

The AppSec policy is intended to govern application security activities for applications developed by the organization. By definition, a policy is a statement of intent and is implemented by standards, processes, and procedures.

Strategic KPIs

Do you have strategic Key Performance Indicators (KPI) for your AppSec program?

Guidance

KPI stands for Key Performance Indicator. It is a measurable value that demonstrates how effectively an organisation is achieving key business objectives. The KPI is the most important metric that allows you to know how well you are working toward your goals. The strategic KPIs will allow you to track the progress of your objectives toward the goals of your AppSec program, e.g. towards managing risk and/or compliance.

Education & Guidance

Have you defined the education and guidance strategy for your AppSec program?

Guidance

The education and guidance strategy includes the training that is required for the various roles of the stakeholders within the AppSec program. This includes the training and guidance for stakeholders such as the development organisation as well as the AppSec auditors, AppSec testing infrastructure management and maintenance, as well as AppSec program management. Typical goals of the education and guidance strategy are to understand the risks from application level attacks, common vulnerability types and how to avoid them, how to remediate vulnerabilities that were identified, how to use and deploy the required tools but also what are typical goals and objectives of an AppSec program. Suitable education and guidance will help avoiding vulnerabilities, improve the security culture, of your organization, and help remediating the identified vulnerabilities faster and more consistently.

Application Inventory with Risk Rating

Do you have an updated inventory of your applications, and do you perform a risk rating of those applications?

Guidance

An application inventory is the inventory of applications developed by the organisation. This inventory should include the risk rating for each application. This inventory is essential to allocate the right priority and strategy for security measures and security testing strategy for each application.
The risk rating process is defined as the process to qualify and classify applications regarding their business risk and therefore the need for protection on the application layer. The result of the process is a categorization of the application in the application inventory.

Security Testing Tools in use (Depth of coverage)

Do you scan applications with automated security testing tools?

Guidance

Goals and objectives, security plans, and processes such as vulnerability lifecycle ultimately determine the security testing tools that should be used as part of an AppSec program. These can include different types of application security testing tools such as Static Application Security Testing (SAST), Software Composition Analysis (SCA), Infrastructure as Code (IaC) security, etc.

Architecture, Deployment Model, and Sizing

Is the architecture for application security testing deployed and does it meet your requirements?

Guidance

This question relates to the deployment model and architecture for the technical deployment of the security testing solution. The deployment models can include SaaS/public cloud multi-tenant, single-tenant on private cloud, or on-prem. It also includes hybrid deployment models of the aforementioned.
The deployment model and architecture need to be planned and decided carefully depending on the scale of the security testing required, as well as confidentiality or compliance requirements which may include regulations from different regions in multinational corporations.
The architecture planning also includes questions such as data retention.

SDLC Integration & Scanning Approach

Do you trigger application security scans automatically as part of the software development or DevOps lifecycle?

Guidance

An optimal scanning strategy requires automation to achieve consistent and predictable results of the analysis process. Integration points refer to how the application security testing is integrated into the software development lifecycle (SDLC). Potential integration points for scan automation can be source control management (SCM) integration, build pipeline (CI/CD) integration, or scheduled scans.

Result Review & Remediation Process

Does your development organization take action based on results from automated application security testing?

Guidance

Taking action from security testing includes:
- Reviewing results consistently,
- Taking effective remediation action, and
- Automation to ensure that an automated build or deployment process does not continue in the case of serious vulnerabilities detected or that issues or risks are reported automatically.

Planning in General

Do you have a roll-out plan, adoption plan, resource plan, and training plan for your AppSec program in place?

Guidance

- Roll-out plan: We define roll-out plan as the plan to implement the solution until a business as usual (BAU) state is reached and includes considerations such as whether a pilot phase is planned and how to move between different stages until the BAU state is reached.
- Adoption plan: The adoption plan refers to scale up to the whole application estate of the organization that is scope for the AppSec program.
- Resource plan: The plan for the human resources needed for the AppSec program including the capacity needs and the roles & responsibilities.
- Training plan: The plan for the training that is required for stakeholders in the AppSec program.

Security Testing Tools in use (Depth of coverage)

Do you scan applications with automated security testing tools?

Goals and objectives, security plans, and processes such as vulnerability lifecycle ultimately determine the security testing tools that should be used as part of an AppSec program. These can include different types of application security testing tools such as Static Application Security Testing (SAST), Software Composition Analysis (SCA), Infrastructure as Code (IaC) security, etc.

SDLC Integration & Scanning Approach

Do you trigger application security scans automatically as part of the software development or DevOps lifecycle?

An optimal scanning strategy requires automation to achieve consistent and predictable results of the analysis process. Integration points refer to how the application security testing is integrated into the software development lifecycle (SDLC). Potential integration points for scan automation can be source control management (SCM) integration, build pipeline (CI/CD) integration, or scheduled scans.

Architecture, Deployment Model, and Sizing

Is the architecture for application security testing deployed and does it meet your requirements?

This question relates to the deployment model and architecture for the technical deployment of the security testing solution. The deployment models can include SaaS/public cloud multi-tenant, single-tenant on private cloud, or on-prem. It also includes hybrid deployment models of the aforementioned.
The deployment model and architecture need to be planned and decided carefully depending on the scale of the security testing required, as well as confidentiality or compliance requirements which may include regulations from different regions in multinational corporations.
The architecture planning also includes questions such as data retention.

Application Inventory with Risk Rating

Do you have an updated inventory of your applications, and do you perform a risk rating of those applications?

An application inventory is the inventory of applications developed by the organisation. This inventory should include the risk rating for each application. This inventory is essential to allocate the right priority and strategy for security measures and security testing strategy for each application.
The risk rating process is defined as the process to qualify and classify applications regarding their business risk and therefore the need for protection on the application layer. The result of the process is a categorization of the application in the application inventory.

Application Triage & Optimization

Do you have a triage and optimization process for your applications (also known as application onboarding)?

When applications get first included in the application security process they require to be initially integrated into the process. We call this the application triage and optimization process which typically includes performing an initial scan (baseline scan), an initial result review, performing the SDLC integration, and making sure that the checks/queries of application security tests perform as expected for the application (optimization). Furthermore, the application development team needs to be familiarized with the process of application security testing. A security architecture assessment/threat assessment in this triage and optimization process can be included depending on the risk rating of the application.
The triage and optimization should be reviewed at certain points, e.g., after major architectural changes to the application, and also in certain regular intervals, e.g., annually, to make sure all the automated testing still works as expected.

Tactical KPIs

Did you define and are you tracking tactical Key Performance Indicators (KPI) for your AppSec program?

A KPI is a measurable value that demonstrates how effectively an organisation is achieving or progressing towards its objectives of the AppSec program.
The tactical KPIs are intended to track progress on performing processes such as the vulnerability lifecycle and security testing plan or how far the adoption as progressed as opposed to the strategic KPIs which are mainly intended to track progress on the overall goals such as managing risk. Tactical KPIs are more in the responsibility of the AppSec management whereas strategic KPIs are more in the domain of executive level leadership (CISO).

Result Review & Remediation Process

Does your development organization take action based on results from automated application security testing?

Taking action from security testing includes:
- Reviewing results consistently,
- Taking effective remediation action, and
- Automation to ensure that an automated build or deployment process does not continue in the case of serious vulnerabilities detected or that issues or risks are reported automatically.

Goals & Objectives

Have you defined clear goals and objectives for your AppSec program that are aligned with the risk appetite of your organization?

Goals and objectives are often used interchangeably, but there are significant differences. A goal is a broad, overarching idea or vision of where you want to reach. Objectives are the steps or milestones that towards reaching the identified goals. Objectives are important because they keep the goal alive and are the smaller, time-bound steps to help getting towards the overall goal.
To be part of the corporate governance function of an organisation, the goals, and objectives of the AppSec program should be tied to the risk appetite of the organisation. For this purpose, the risk appetite of the organization's executive leadership needs to be captured and aligned with the goals and objectives of the AppSec program. The organization's leadership should vet and approve the set of goals.

AppSec Policies

Do you have an AppSec policy and standards in place?

The AppSec policy is intended to govern application security activities for applications developed by the organization. By definition, a policy is a statement of intent and is implemented by standards, processes, and procedures.

Education & Guidance

Have you defined the education and guidance strategy for your AppSec program?

The education and guidance strategy includes the training that is required for the various roles of the stakeholders within the AppSec program. This includes the training and guidance for stakeholders such as the development organisation as well as the AppSec auditors, AppSec testing infrastructure management and maintenance, as well as AppSec program management. Typical goals of the education and guidance strategy are to understand the risks from application level attacks, common vulnerability types and how to avoid them, how to remediate vulnerabilities that were identified, how to use and deploy the required tools but also what are typical goals and objectives of an AppSec program. Suitable education and guidance will help avoiding vulnerabilities, improve the security culture, of your organization, and help remediating the identified vulnerabilities faster and more consistently.

Planning in General

Do you have a roll-out plan, adoption plan, resource plan, and training plan for your AppSec program in place?

- Roll-out plan: We define roll-out plan as the plan to implement the solution until a business as usual (BAU) state is reached and includes considerations such as whether a pilot phase is planned and how to move between different stages until the BAU state is reached.
- Adoption plan: The adoption plan refers to scale up to the whole application estate of the organization that is scope for the AppSec program.
- Resource plan: The plan for the human resources needed for the AppSec program including the capacity needs and the roles & responsibilities.
- Training plan: The plan for the training that is required for stakeholders in the AppSec program.

Security Testing Tools in use (Depth of coverage)

Do you scan applications with automated security testing tools?

Goals and objectives, security plans, and processes such as vulnerability lifecycle ultimately determine the security testing tools that should be used as part of an AppSec program. These can include different types of application security testing tools such as Static Application Security Testing (SAST), Software Composition Analysis (SCA), Infrastructure as Code (IaC) security, etc.

SDLC Integration & Scanning Approach

Do you trigger application security scans automatically as part of the software development or DevOps lifecycle?

An optimal scanning strategy requires automation to achieve consistent and predictable results of the analysis process. Integration points refer to how the application security testing is integrated into the software development lifecycle (SDLC). Potential integration points for scan automation can be source control management (SCM) integration, build pipeline (CI/CD) integration, or scheduled scans.

Architecture, Deployment Model, and Sizing

Is the architecture for application security testing deployed and does it meet your requirements?

This question relates to the deployment model and architecture for the technical deployment of the security testing solution. The deployment models can include SaaS/public cloud multi-tenant, single-tenant on private cloud, or on-prem. It also includes hybrid deployment models of the aforementioned.
The deployment model and architecture need to be planned and decided carefully depending on the scale of the security testing required, as well as confidentiality or compliance requirements which may include regulations from different regions in multinational corporations.
The architecture planning also includes questions such as data retention.

Application Inventory with Risk Rating & Security Testing Plan

Are you aware of a classification of applications according to risk?

The risk rating process is defined as the process to qualify and classify applications regarding their business risk and therefore the need for protection on the application layer. The result of the process is a categorization of the application in the Application Inventory. This inventory is essential to allocate the right priority and strategy for security measures and security testing strategy for each application.
After applications have been categorized by risk, a security plan according to the risk category should be devised. The security plan is the planned procedure for ensuring that the security of an application is within acceptable risk levels. It is governed by the application security policy, but it is more detail including which security testing tools to use. Usually, different security plans are drawn up for each application risk category or application type.

Application Triage & Optimization

Do you have a triage and optimization process for your applications (also known as application onboarding)?

When applications get first included in the application security process they require to be initially integrated into the process. We call this the application triage and optimization process which typically includes performing an initial scan (baseline scan), an initial result review, performing the SDLC integration, and making sure that the checks/queries of application security tests perform as expected for the application (optimization). Furthermore, the application development team needs to be familiarized with the process of application security testing. A security architecture assessment/threat assessment in this triage and optimization process can be included depending on the risk rating of the application.
The triage and optimization should be reviewed at certain points, e.g., after major architectural changes to the application, and also in certain regular intervals, e.g., annually, to make sure all the automated testing still works as expected.

Vulnerability Lifecycle

Do you have a defined vulnerability life cycle process?

The vulnerability lifecycle describes the existence of a vulnerability from its detection to its closure. The typical steps of the vulnerability lifecycle include: detecting a vulnerability (usually by a security testing tool); reviewing it for relevance and urgency; assigning to a resource for it to be remediated (in case of a relevant finding); ensuring effectiveness of fix; and lastly closing the vulnerability if the fix is effective.
Vulnerability lifecycles can be differentiated for different criticalities of issues or applications. The process definition includes roles and responsibilities of different stakeholders involved.

Result Review & Remediation Process

Do you take action based on results from automated application security testing?

Taking action from security testing includes:
- Reviewing results consistently,
- Taking effective remediation action, and
- Automation to ensure that an automated build or deployment process does not continue in the case of serious vulnerabilities detected or that issues or risks are reported automatically.

Tactical KPIs

Are you aware of Key Performance Indicators (KPIs) for the AppSec program and do you have access to the data?

A KPI is a measurable value that demonstrates how effectively an organisation is achieving or progressing towards its objectives of the AppSec program. Tactical KPIs are intended to track progress on performing processes such as the vulnerability lifecycle and security testing plan.

Goals & Objectives; AppSec Policies

Are you aware of defined goals and objectives for your AppSec program and that an AppSec policy exists?

Goals and objectives are often used interchangeably, but there are significant differences. A goal is a broad, overarching idea or vision of where you want to reach. Objectives are the steps or milestones that towards reaching the identified goals. Objectives are important because they keep the goal alive and are the smaller, time-bound steps to help getting towards the overall goal.
The AppSec policy is intended to govern application security activities for applications developed by the organization. By definition, a policy is a statement of intent and is implemented by standards, processes, and procedures.

Education & Guidance

Did you receive AppSec training and guidance?

AppSec Training should be repeatable, consistent, and available to anyone involved with the software development lifecycle. Training should include security standards such as the OWASP Top 10, and should include concepts such as Least Privilege, Defence-in-Depth, Fail Safe/Secure, Complete Mediation, Session Management, Open Design, and Psychological Acceptability. Training should include a sign-off or an acknowledgment from attendees. There should be regular refresher trainings and with updated training content, at least annually. Training should be provided as part of employees' onboarding process.

Planning in General

Are you aware of a roll-out plan, adoption plan, and/or resource plan for your AppSec program?

- Roll-out plan: We define roll-out plan as the plan to implement the solution until a business as usual (BAU) state is reached and includes considerations such as whether a pilot phase is planned and how to move between different stages until the BAU state is reached.
- Adoption plan: The Adoption plan refers to scale up to the whole application estate of the organisation that is scope for the AppSec program.
- Resource plan: The plan for The human resources needed for the AppSec program including the capacity needs and the roles & responsibilities.