Optimising the meta-optimiser in machine learning problems

Abstract

The improvement of machine learning algorithm performance can be achieved via the search across the algorithm’s parameter space by a number of meta-heuristic based optimisation techniques or meta-optimisers. These meta-optimisers vary in their approach and have been shown to perform differently on various optimisation problems. This paper takes the concept of metaoptimisation further and explores the potential of fine tuning the meta-optimiser itself in order to improve the optimisation process. The study is centred around the use of a Simple Genetic Algorithm (SGA) based meta-optimisation approach and the measurement of the effect of changing its mutation and crossover rates on a set of machine learning algorithms and standard datasets. Although the results did not evidence any significant effect on the overall performance of the SGA as a machine learner meta-optimiser they showed significant effects on the SGA's efficiency. This discovery is beneficial for machine learning optimisation problems, particularly those that require high processing costs, have low computational budgets or both.