Consider a DevOps process in which a project is built and uses ASTs nightly. Developers submit code to a central Git repository, the project’s master code set. Jenkins will orchestrate an overnight build. The overnight build typically incorporates many AST tools for vulnerability assessment. The result could be tens to hundreds of thousands of reported defects from each of a dozen or more AST tools in use. This volume of defects will overwhelm issue tracking systems, which leads to the use of expensive vulnerability management systems. In the end, many thousands of defects require triage.
TODAY’S DevOps AND DevSecOps PROCESSES
Consider a mid-sized mature project of 300 KLOC of source code. The AST results for such a project can be 50,000 - 250,000 defect reports from a single AST, with one-half to one-third being designated as severe by the AST. Organizations using ASTs must triage the reported defects to find the highest priority, most likely true defects. This triage must be context-sensitive - determining what is risky and important for a specific project. This monumental task is compounded by the fact that most defect reports are false positives, not important to an application, and/or difficult to exploit. A development team has a choice between spending months of time doing detailed triage or doing a quick triage, often missing important defects, or just being overwhelmed and ignoring the found defects.
In addition to triage issues of cost and time, manual triage is highly inaccurate. There are numerous sources of inaccuracies, among them:
Incomplete or incorrect defect descriptions
▷
Limited information on defect consequences
▷
Limited ability to determine a consequence’s severity for a particular application’s context
▷
No information on attack patterns or the ease of exploitation
▷
No information of the likelihood that the defect is a false positive
▷
The result is that application security teams must err on the side of caution - wasting significant resources, while missing many crucial defects or security issues.
The result is that often the time and cost of AST result triage causes security resolution to be performed separately, outside the daily DevOps practice.“ In 2019, the number of Gartner end-user client conversations on DevSecOps and AST increased by 50% over 2018. While most clients do not have a full or even majority DevOps team, many techniques out of the DevOps method are easily adapted to existing coding disciplines.”⁷
