Source Code Auditing Research

Jon Heffley, Dr. Pascal Meunier

Overview: Sardonix.org has recently been started to “facilitate program audits and provide a core repository for reviewing and submitting them.”

Dr. Meunier and I were interested to find out if the code auditors that they listed were effective for auditing (open source) programs, specifically for security problems.  However, after evaluating the code auditors listed by the website, it was clear that only one code auditor was efficient at evaluating certain security problems.

Problem Statement:  Can source code auditing software identify common vulnerabilities and be used to evaluate software security?

Hypothesis:

Source code auditing software can identify common vulnerabilities and can be used to evaluate software security.

Null Hypothesis:

Source code auditing software can’t identify common vulnerabilities or be used to evaluate software security.

Results Obtained (thus far):

Not all code auditors are for third party evaluation

Not all code auditors focus on security

Not all code auditors are worth one’s time

Pscan is the most effective at pointing out specific kinds of errors

How I have contributed to the overall project:

Scratching surface

Trying to get an idea of what kind of code auditors out there

Evaluated the efficiency/effectiveness of types of auditors

Evaluated the efficiency/effectiveness of individual auditors

My task for the past semester has been learning about and evaluating open source code auditors and how they can benefit security.  This research has been performed by me under the guide of my mentor Dr. Pascal Meunier.

My research revolved around the Sardonix.org website, which is a website to “facilitate program audits and provide a core repository for reviewing and submitting them.”  From this website I was able to download many different open source code auditors, and found one commercial code auditor that I was able to use under the evaluation license.  These auditors audited C/C++, and some tested for other languages.  Because so many applications are developed in C/C++, benchmarking our results with applications in this language

One of the things I learned was that not all code auditors were intended for third parties, that is to say, parties other than the developers of the software, to use.  They require annotation to describe to the auditor what the code should or shouldn’t be doing, or attention to specific details in the project.

Another kind of code auditor is one that I placed into a category I have dubbed, “function spotter.”  All they do is find where certain functions defined by the C library, functions that are known for being problematic, are located, and reference that line as an “error.”  Because it does not discriminate how the functions are used or find any context to it, it will give an error for every single usage of these functions, which include but are not limited to, printf(), syslog(), etc.

Some auditors will check for header files and audit them as well alert you to problems inside of them.  This can be a burden, because it will sometimes ask you to look for coding problems within the standard library’s source code.

The most satisfactory code auditor looked for a particular kind of vulnerability (Format String vulnerability) and was able to most of them (as benchmarked by the Common Vulnerabilities and Exposures (CVE)).  It did this by looking for particular kinds of functions (variable arguments function) defined within the software package, and by looking at the specific arguments to see if it fit the common way that the function is misused.  Although it is fairly simple to find, it is an all too common vulnerability in software packages.  While the auditor may miss a few instances of this vulnerability, these ways are not usually ways that the vulnerability occurs.  While it might give false positives and false negatives, because it gives significantly less false positives, it is the most efficient code auditor.