Skip to main content

Maintenance Settings

Data Retention Management

In order to properly manage data storage consumption, CxSAST allows for the manual purging of old scan data. An administrator can define the desired storage policy by date range or by defining a minimal number of scans to retain overriding the date range.

For more information about data retention management settings, please refer to Data Retention Management for v9.0.0.

Data Retention Management

In order to properly manage data storage consumption, CxSAST allows for the manual purging of old scan data. An administrator can define the desired storage policy by date range or by defining a minimal number of scans to retain overriding the date range.

Notice

Warning - Scanned data is purged from the file system as well as the database, therefore, once deleted cannot be reversed. See Data Retention Purged Data, below.

Using our CxSAST (REST) API for Data Retention, this process can be automated (v8.8.0 and up).

Notice

Data retention settings apply globally to all projects within the system. This global configuration can be overridden for a specific project, either during the project creation or by editing the project's setting through the Data Retention tab (see Creating and Configuring a CxSAST Project and Viewing Project Details.

Specific scans may be marked as “Locked” to avoid automated purging of important scan data.

Notice

Locked scans cannot be deleted, and will be skipped in the data retention process. If you would like to delete all scans within the range defined for deletion, it is highly important to ensure that no locked scans are included within this range. If the range does include locked scans, unlock the scans before executing the Data Retention command. Refer to Unlocking Scans (v9.0.0 to v9.2.0) or Unlocking Scans respectively.

Defining Data Retention Settings

To define the data retention settings, do the following:

Select Settings > Application Settings > Data Retention. The Data Retention window is displayed.

6436176966.png

The Data Retention window includes the following settings:

Scans to keep:

  • Keep last successful scans - Set the requested number of scans to be kept. This setting leaves only the specified number of recent successful last scans and deletes all other scans. For example, if the value is set to 10, it will keep the last 10 successful scans for each project.

Scans to delete:

  • Select date range to delete scans - Enter a start and an end date. This setting deletes all scans within a predefined time range.

  • Retention duration limit (hours) - Set a limit to the amount of time the operation should take. If set to 10, then after 10 hours the operation automatically stops, regardless of whether the operation is complete.

Click Start. The following message appears:

6436176987.png

If you are unsure whether you have backed up your database, or if the range you defined for deletion includes locked scans, click Cancel to postpone the deletion.

If you want to continue, click Yes, delete it. The following message is displayed "Data retention is now in progress" and the progress of the data retention process is represented in the Stages panel.

6436176978.png

Once the data retention process is complete, status information about last deletion is displayed in the Last Executed Data Retention panel.

6436176981.png

Data Retention Purged Data

Scanned data is purged from the file system as well as the database, therefore, once deleted cannot be reversed. Note that data retention process also deletes failed and cancelled scans.

The following data is purged as part of the data retention:

Database Tables

Selected data from the following tables is purged as part of the data retention:

  • All Scans

  • TaskScans

  • CancelledScans

  • TaskScanEnvironment

  • ScanReports

  • FailedScans

  • PathResults

  • NodeResults

File System

  • CxSRC folder – This folder holds the extracted source files which are being scanned.

    Files and folders inside the CxSrc folder are deleted as part of data retention except for the following scenario:

    In case the exact same sources (ZIP, remote location..) are uploaded to the same existing scan, the extracted folder will be excluded from further data retention cleaning tasks.

  • CxReports folder - This folder holds the following:

    • Reports requested by the customer and created in the CxSAST reports page. These reports are deleted as part of the data retention

    • Eclipse IDE reports created after each developer scan request. These reports are not deleted as part of the data retention.

Unlocking Scans

One of the most common reasons for having no scans deleted is that one or more of the scans are locked. This can be modified by unlocking the scans as follows:

1. Go to Projects & Scans > Projects.

6436177047.png

2. Select the requested project. If many projects exist, filter the project list as follows:

  • Click

    6436177059.png

    Filters on the right.

  • Type one or more identifying criteria for the project, such as the project name, owner or team.

  • Press <Enter> to only see the projects listed that match the filter criteria you entered.

3. Under Scans List, click 6436177056.png View Projects Scans. The scans run by the selected project appear listed.

4. Locate the locked scan. Locked scans are labeled Locked in the LOCKED column as illustrated below. Unlocked scans do not have an entry in that column.

6436177053.png

5. Click 6436177050.png to unlock the scan.

To lock a scan:

  • Go to the desired scan in the list and click

    6436177050.png

    . The Locked indicator appears in the LOCKED column.

CxSAST Data Retention Control with CLI

CxSAST Data Retention Control – Start / Stop (Windows)

Given the URL of a specific Checkmarx web interface, starts a data retention by deleting either all scans within a specified date range or all but the last X scans for each project. This command can also be used for stopping a currently running data retention process.

Notice

The “CxManagement.ps1” script should be requested via Checkmarx Support.

Syntax

CxManagement.ps1 -StartRetention -serviceUrl "http://domain.mysite.com/" -username "admin" -pass "[email protected]"

Parameters

Parameter

Description

serviceUrl

The URL of a Checkmarx web service.

username

The Checkmarx user name of a user with permissions to run a data retention process (Server Administrator).

pass

The password of the specified Checkmarx user.

StopRetention

Enable this switch to stop the Data Retention process

StartRetention

Enable this switch to start the Data Retention process.

ByNumOfScans

A switch defining that all of the scans in the system, except for the most recent X scans in each project, will be deleted. The number of recent scans to be kept (X) is specified by the numOfScansToKeep parameter.

numOfScansToKeep

When the ByNumOfScans switch is enables, defines how many recent scans are kept in each project when data retention is carried out.

ByDateRange

A switch that defines that all of the scans within a specified date range will be deleted.

startDate

An optional inclusive lower limit of the date range of scans to delete. Only considers dates, ignores hours.

endDate

A mandatory inclusive upper limit of the date range of scans to delete. Only considers dates, ignores hours.

retentionDurationLimit

An optional parameter that allows to limit the duration of the data retention process. Specified only in round hours (integers), and applies to all scans performed after this parameter was set.

The duration limit does not override the data retention schedule. Scans are deleted in bulks of X (configured in the database with a default value of 3), and the data retention process stops only upon the completion of the last bulk that started before the duration limit has been reached. For example: if the duration limit was set to 4 hours, and the last bulk started 3:50 hours after this parameter was set, the retention process will stop only when this bulk completes, even if the completion takes place well past 4:00 hours.

Error Codes

  • 1: argument error

  • 2: http error

  • 4: server error

Remarks

  • It is only possible to define either StartRetention -or- StopRetention.

  • It is only possible to define one type of retention at a time.

  • The duration limit is not an immediate limit, scans are deleted in bulks of X (configured in the database with a default value of 3) and the data retention will only stop at the end of a bulk.

Examples

To delete all but the last 5 scans of each project:

CxManagement.ps1 -StartRetention -serviceUrl "http://domain.mysite.com/" -username "admin" -pass "[email protected]" -ByNumOfScans -numOfScansToKeep 5

To delete all but the last 5 scans of each project, limit the duration of the data retention process to 2 hours:

CxManagement.ps1 -StartRetention -serviceUrl "http://domain.mysite.com/" -username "admin" -pass "[email protected]" -ByNumOfScans -numOfScansToKeep 5 -retentionDurationLimit 2

To delete all of the scans performed before October 10, 2015:

CxManagement.ps1 -StartRetention -serviceUrl "http://domain.mysite.com/" -username "admin" -pass "[email protected]" -ByDateRange -endDate "2015-10-10"

To delete all of the scans between October 5 and October 10, 2015:

CxManagement.ps1 -StartRetention -serviceUrl "http://domain.mysite.com/" -username "admin" -pass "[email protected]" -ByDateRange -startDate "2015-10-05" -endDate "2015-10-10" 

To delete all of the scans between October 5 and October 10 2015, and limit the duration of the data retention process to 2 hours:

CxManagement.ps1 -StartRetention -serviceUrl "http://domain.mysite.com/" -username "admin" -pass "[email protected]" -ByDateRange -startDate "2015-10-05" -endDate "2015-10-10" -retentionDurationLimit 2

To stop the data retention process.

CxManagement.ps1 -StopRetention -serviceUrl "http://domain.mysite.com/" -username "admin" -pass [email protected]

CxSAST Data Retention Control – Start / Stop (Linux)

Given the URL of a specific Checkmarx web interface, starts a data retention by deleting either all scans within a specified date range or all but the last X scans for each project. This command can also be used for stopping a currently running data retention process.

Syntax

CxManagement.sh --StartRetention --ServerUrl http://domain.mysite.com/ --Username admin --Password [email protected]
CxManagement.sh -start -s http://domain.mysite.com/ -u admin -p [email protected]

Parameters

Parameter

Description

-s, --ServiceUrl

The URL of a Checkmarx web service.

-u, --Username

The Checkmarx user name of a user with permissions to run a data retention process (Server Administrator).

-p, --Password

The password of the specified Checkmarx user.

-stop, --StopRetention

Enable this switch to stop the Data Retention process

-start, --StartRetention

Enable this switch to start the Data Retention process.

-byScans, --ByNumOfScans

A switch defining that all of the scans in the system, except for the most recent X scans in each project, will be deleted. The number of recent scans to be kept (X) is specified by the numOfScansToKeep parameter.

-n, --numOfScansToKeep

When the ByNumOfScans switch is enables, defines how many recent scans are kept in each project when data retention is carried out.

-byDates, --ByDateRange

A switch that defines that all of the scans within a specified date range will be deleted.

-f, --FromDate

An optional inclusive lower limit of the date range of scans to delete. Only considers dates, ignores hours.

-t, --ToDate

A mandatory inclusive upper limit of the date range of scans to delete. Only considers dates, ignores hours.

-l, --RetentionDurationLimit

An optional parameter that allows to limit the duration of the data retention process. Specified only in round hours (integers), and applies to all scans performed after this parameter was set.

The duration limit does not override the data retention schedule. Scans are deleted in bulks of X (configured in the database with a default value of 3), and the data retention process stops only upon the completion of the last bulk that started before the duration limit has been reached. For example: if the duration limit was set to 4 hours, and the last bulk started 3:50 hours after this parameter was set, the retention process will stop only when this bulk completes, even if the completion takes place well past 4:00 hours.

Error Codes

  • 1: argument error

  • 2: http error

  • 4: server error

Remarks

  • It is only possible to define either StartRetention -or- StopRetention.

  • It is only possible to define one type of retention at a time.

  • The duration limit is not an immediate limit, scans are deleted in bulks of X (configured in the database with a default value of 3) and the data retention will only stop at the end of a bulk.

Examples

To delete all but the last 5 scans of each project:

CxManagement.sh --StartRetention --ServerUrl http://domain.mysite.com/ --Username admin --Password [email protected] --ByNumOfScans --NumOfScansToKeep 5
CxManagement.sh -start -s http://domain.mysite.com/ -u admin -p [email protected] -byScans -n 5

To delete all but the last 5 scans of each project, limit the duration of the data retention process to 2 hours:

CxManagement.sh --StartRetention --ServerUrl http://domain.mysite.com/ --Username admin --Password [email protected] -- yNumOfScans --NumOfScansToKeep 5 --RetentionDurationLimit 2
CxManagement.sh -start -s http://domain.mysite.com/ -u admin -p [email protected] -byScans -n 5 -l 2

To delete all of the scans performed before October 10, 2015:

CxManagement.sh --StartRetention --ServerUrl http://domain.mysite.com/ --Username admin --Password [email protected] --ByDateRange --ToDate 2015-10-10
CxManagement.sh -start -s http://domain.mysite.com/ -u admin -p [email protected] -byDates -t 2015-10-10

To delete all of the scans between October 5 and October 10, 2015:

CxManagement.sh --StartRetention --ServerUrl http://domain.mysite.com/ --Username admin --Password [email protected] --ByDateRange --FromDate 2015-10-05 --ToDate 2015-10-10
CxManagement.sh -start -s http://domain.mysite.com/ -u admin -p [email protected] -byDates -f 2015-10-05 -t 2015-10-10

To delete all of the scans between October 5 and October 10 2015, and limit the duration of the data retention process to 2 hours:

CxManagement.sh --StartRetention --ServerUrl http://domain.mysite.com/ --Username admin --Password [email protected] --ByDateRange --FromDate 2015-10-05 --ToDate 2015-10-10 --RetentionDurationLimit 2
CxManagement.sh -start -s http://domain.mysite.com/ -u admin -p [email protected] -byDates -f 2015-10-05 -t 2015-10-10 -l 2

To stop the data retention process:

CxManagement.sh --StopRetention --ServerUrl http://domain.mysite.com/ --Username admin --Password [email protected]
CxManagement.sh -stop -s http://domain.mysite.com/ -u admin -p [email protected]sw0rd