Automated test data generation using search based and machine learning techniques

Abstract

In recent years, the E-Commerce sector has evolved into one of the most lucrative sectors in the modern industry. This resulted in progressively larger web systems that are designed to provide the customer with a wide variety of functionality and tools. However, this growth also resulted in higher maintenance and support costs that limit the effectiveness of manual testing tasks. To mitigate such issues, development practices started to include testing components during their life cycles to verify specific elements of the system. However, even this solution is somewhat restrictive, as these resources generally require continuous updating and modifications. This study proposes an automated approach to evaluate an aspect of software testing, specifically, the generation of test data. The objective is to develop a system that would be capable of exercising and verifying the behaviour of a specific E-Commerce web-system. To direct the traversal of this entity, a Genetic Algorithm was established. The principle behind this algorithm is to utilise a search-based evolutionary structure to iteratively discover and test a wide range of operations of the system. Although, this approach is prone to a number of limitations. Two main weaknesses consist of the risk that said design might not be able to identify desirable behaviour and that it might not produce an ideal solution to the specified problem. To compensate for these weaknesses, a Reinforcement Learning Agent was introduced. The scope of this entity was to learn and direct the traversal behaviour of the respective test data generation structure at the Reinsertion stage. This agent was set up as a Model-Free Monte Carlo implementation. In addition, a subsequent implementation was evaluated where the Reinforcement Learning Agent started off from an educated state. The experiments were conducted on an established E-Commerce web architecture in an attempt to achieve comparable observations. From the generated results it was evident that the proposed design of the Genetic Algorithm with Reinforcement Learning Agent produced a higher coverage rate for the established regions, whilst also discovering a broader domain of the web-system. Furthermore, the educated agent proved to be the most resource efficient implementation, as it was able to reach its highest line coverage prior to majority of designs taken into consideration. These results exhibit encouraging outcomes for potential architectures on similar infrastructures. Nevertheless, further analysis should be taken into consideration for alternative attribute, interaction selection and specifications.