SAST: a tale of inefficiency

Deploying SAST tooling can be a daunting task: after getting your hands dirty integrating repositories you don’t own in build pipelines you don’t own either, you end up with hundreds or even thousands of “security items”.

Browsing through the dozens of Critical and hundreds of High ones, you can’t most of the time spot quickly any obviously exploitable finding. The Signal to Noise ratio is more than disappointing, even after a first pass of fine tuning.

But your organization considers it as mandatory for compliance reasons (or has been told so).

To avoid damaging the trust relationship you patiently built with the Security champions, you don’t want to throw that kind of work at them. Yet your security team does not have the bandwidth to address that properly, let alone the motivation drawbacks of burning cycles on insignificant topics.

What if there was a way to prove the business value first ?
And do the heavy lifting later if it is worth it ? While still demonstrating that you do really care about the security of your codebase.

Having many SAST rules is tempting to improve coverage (and have great benchmark results), but Less Is More wins in terms of Signal to Noise. What if we could identify the few SAST rules detecting real vulnerabilities ? Ideally without generating too many false positives. And only start with those ?

Introducing TSAS: using SAST the other way round

Considering security fixes in git as the source of truth for real vulnerabilities.

TSAS is a Python CLI tool to figure out from a set of GitHub commits containing security fixes which standard SAST rules would have detected those vulnerabilities.

A SAST rule is considered relevant if in any file impacted by the security fix:

it reports a finding in the latest commit just before the fix
it does not report it anymore once the fix is included

Those relevant rules are then prioritised taking into account:

number of occurrences (if it has worked several times in the past, it is likely it will help in the future)
how many alerts are triggered if activated today on the codebase (i.e. how noisy)

There is also a feature to automatically discover security commits: it is far from perfect, but should be good enough to identify some relevant rules to start with.

Detailed information is available in the GitHub public repository

Any feedback or question ?

Let’s get in touch via tsas@appsecmatters.com or using the TSAS repository.