Research Assistants: Sailesh Krishnamurthy
Sponsors: NSF, Bell Communications Research, Center for Advanced Studies at IBM; Toronto.
Our research is concerned with issues of relative cost-effectiveness of various testing methods, and in par ticular, the effectiveness of data flow and mutation testing. Our work, experimental in nature, answers ques tions such as: Which of these two test adequacy criteria is more difficult to satisfy? What is the incremental cost of using mutation testing after data flow testing has been used? What predictions can be made about the reli ability of a program after it has been tested using data flow and/or mutation testing? How does one reduce the cost of using mutation testing without adversely affecting its error exposing capability?
To answer the above questions we are presently conducting experiments using a suite of commonly used applications. This work is being done using ATAC, a proprietary data flow testing tool for C programs released by Bellcore for use in our research. In addition we are also using MOTHRA and PROTEUM, two tools for mu tation testing.
Another issue that becomes critical when doing the research described above and also limits the utility of mutation testing has to do with performance. A tremendous amount of computing power is required for rou tine mutation testing, and this can become a serious bottleneck when large programs are to be tested.
This has led to our research on Compiler Integrated Testing (CIT) in which we focus on issues that arise when one needs to integrate mutation and data flow testing into a compiler. CIT is a new and novel concept that, once fully developed, is likely to completely change the way testing tools are currently designed.