Enhancing stock price prediction models by using concept drift detectors

Abstract

Stock price movement prediction is faced with the problem that the distribution of certain underlying variables change over time. This phenomenon is defined as concept drift. Due to this phenomenon, stock price prediction models tend to give less accurate results, since the data distribution that the model has been trained on is no longer in-line with the current data distribution. In this paper an Adversarial Attentive Long Short-Term Memory (Adv-ALSTM) model is used together with a Hoeffding’s inequality based Drift Detection Method with moving Average-test (HDDMA) concept drift detector in order to make price movement predictions on 50 different stocks. Every time the HDDMA concept drift detector detects a concept drift, the model undergoes one of four possible retraining methods. The conducted experiments highlight the effectiveness of each of the proposed retraining methods, as well as how each of the methods mitigate the negative effects of concept drift in different ways. The best observed results were a 2.5% increase in accuracy and a 135.38% increase in Matthews Correlation Coefficient (MCC) when compared to the vanilla Adv-ALSTM model. These results validate the effectiveness of the proposed retraining methods, when applied to a model that has been trained on a financial dataset.