Extracting monitors from JUnit tests

Abstract

A large portion of the software development industry relies on testing as the main technique for quality assurance while other techniques which can provide extra guarantees are largely ignored. A case in point is runtime verification which provides assurance that a system’s execution flow is correct at runtime. Compared to testing, this technique has the advantage of checking the actual runs of a system rather than a number of representative testcases. Based on experience with the local industry, one of the main reasons for this lack of uptake of runtime verification is the extra effort required to formally specify the correctness criteria to be checked at runtime — runtime verifiers are typically synthesised from formal specifications. One potential approach to counteract this issue would be to use the information available in tests to automatically obtain monitors. The plausibility of this approach is the similarity between tests and runtime verifiers: tests drive the system under test and check that the outcome is correct while runtime verifiers let the system users drive the system under observation but still has to check that the outcome is as expected.