Extracting runtime monitors from Gherkin specifications

Abstract

Gherkin is an industry-standard language for specifying business level scenarios using natural language. These Gherkin specifications can also be used as the foundation for the generation of runtime monitors, thus transitioning from testing into Runtime Verification. However, there a number of challenges in bridging the gap between Gherkin System Level testing and Runtime Verification. This project is concerned with the deduction of different levels of monitor abstraction from test scenarios which can subsequently check the behaviour of a system during runtime, ensuring that the customer-specified business logic remains consistent throughout the systems lifetime. The proposed solution entails a two phase process - generation and monitoring. Central to this solution is the use of a listener which records events taking place on the system under test. These recorded events will be used as the foundation for the generation of runtime monitors. Evaluation of the two different levels of monitoring focuses on their respective expressivity, robustness and incurred overheads. The conclusion drawn from evaluating the results was that different levels of listener abstraction may be suitable for specific scenarios, rather than having an unequivocally better listener.