Integrating mutation testing into agile processes through equivalent mutant reduction via differential symbolic execution

Abstract

In agile programming, software development is performed in iterations. To ensure the changes are correct, considerable effort is spent writing comprehensive unit tests. Unit tests are the most basic form of testing and is performed on the smallest or smaller set of code.These unit tests have multiple purposes, the main one being that of acting as a safety net between product releases. However, the value of testing can be called into question if there is no measure of the quality of unit tests. Code coverage analysis is an automated technique which illustrates which statements are covered by tests. However, high code coverage might still not be good enough as whole branches or paths could still go completely untested which in turn leads to false sense of security. Mutation Testing is a technique designed to successfully and realistically identify whether a test suite is satisfactory. In turn, such tests lead to finding bugs within the code. The technique behind mutation testing involves generating variants of a system by modifying operators (called mutants) and executing tests against them. If the test suite is thorough enough, at least one test should fail against every mutant thus rendering that mutant killed. Unkilled mutants would require investigation and potential modification of the test suite.