Research Assistant: S. Krishnamirthy
Sponsor: Bell Communications Research
Estimation of software reliability is important for several well known reasons. During the past two decades several new models for estimating software reliability have been proposed. Popular reliability estimation models include the Musa-Okumoto model, the Goel-Okumoto model, and the Jelinski-Moranda model. One common characteristic of all of these models is that they uniformly treat software as a black box. Thus, for example, these models do not account for the "thoroughness" with which the code may have been tested. We have noticed several startling effects of using such models. For example, one may predict high reliability for software artifacts that contain several lines of code that were never exercised during testing.
Our research is focused on the development of white-box models for predicting software reliability. Such models seek to explicitly incorporate the testing method used during the testing phase, as well as the structure of the software being tested. As a consequence, our models will lead to a lower reliability figure when only 70% of a given piece of code is exercised and a correspondingly larger reliability figure when 90% of the same code is exercised.
The white-box models proposed involve two types of testing methods: the traditional coverage based methods, such as data flow testing, and fault based methods, such as mutation testing. The proposed methods rely on a phenomenon called the saturation effect, which refers to the limitations of a given testing method. For example, we have observed empirically that extensive amounts of functional testing over long periods of time do not reveal all program errors. Thus, the ability of functional testing saturates after a certain amount of effort has been expended. Due to this saturation effect, we have noticed that existing reliability models may in several cases lead to overestimates of reliability. Through our white-box models we expect to provide more accurate estimates of reliability.
CS Annual Report - 19 APR 1996