Neuroevolutional preference learner via architecture search

Abstract

Preference Learners that focus on games have been around for a little over a decade and they have made use of a number of different machine learning algorithms. Throughout this time neuroevolution has also been employed to help to improve the performance of these algorithms. This is evident from some of the earlier papers on preference learning, to some of the more recent ones. Architecture Search is a derivation of neuroevolution that directly alters the architecture of the neural network. It has seen several advancements throughout the years which have helped to improve its efficacy. The aim of this project is to apply Architecture Search to preference learning models and then to compare the resulting models to those from another study. The objectives for this project are: modifying a pre-existing dataset to be used for training, implementing the search space, exploration strategy, and evaluator for Evolutionary Neural Architecture Search, identifying the best models, and then comparing them to those from a similar study to determine the effectiveness of the new models. The custom dataset was created by extracting the images from the gameplay videos contained within the dataset and then by storing them in a custom dataset class. Subsequently, these were placed into data loaders and used to train the models. Evolutionary Neural Architecture Search was implemented through the use of NNI. It was employed to define the search space in the form of the model class with the list of allowed mutations. It was also used to define the exploration strategy which utilised the Regularized Evolution algorithm and the evaluator which contained the train and test methods. Afterwards, the models were trained through a series of NNI experiments, with models being created for each game using different parameter values. The data extracted from the models indicates that they generally perform better than those from the study being used for comparison purposes. The best models were established by identifying the models which had the highest values from a list of evaluation metrics which were collected during the test method. The new models were compared to those from the other study and it was found that the new models perform better in terms of the average accuracy scores achieved, but are less consistent in their training progress.