Towards gesture based assistive technology for persons experiencing involuntary muscle contractions

Abstract

This research investigates the viability of leveraging Machine Learning (ML) algorithms to develop gesture recognition systems that may benefit people who experience involuntary muscle contractions. This presents distinct challenges, such as the reduced ability to perform gestures accurately and repeatedly (flawed gestures) as well as the ability to provide sufficient data to pre-train models. This investigation revolves around three shortlisted gesture recognition algorithms which were evaluated in a controlled lab environment. The primary objective was to observe specific characteristics such as robustness under different simulated conditions, training requirements, as well as classification latency and accuracy. Results show distinct properties for each shortlisted algorithm. k- Nearest Neighbour (KNN) with Dynamic Time Warping (DTW), or KNN-DTW, is well suited where accurate gesture training is challenging due to frequent involuntary movements. Although this approach works well with one sample, the classification response time is significantly longer than KNN and Support Vector Machine (SVM). However, timing may not always be a priority, depending on the context of use. On the other hand, when real-time responses are necessary, KNN and SVM both offer a good level of performance. These, however, rely on training data to produce accurate classifications, in which case the user must be able to perform gestures in a reasonably repeatable manner. This work also presents a dataset of 1600 samples for four gesture classes, including a corresponding set of flawed gesture samples for each class.