A study on human pose classification using convolutional neural networks and tensor regression

Abstract

Folk dances are a source of a country’s history and traditions, and their documentation and analysis are important for their preservation. Human Pose Classification (HPC) covers the classification of human poses through body part detection from image, video, or measurement data. Folk dances are defined as a repeated sequence of main choreographic steps. The classification of the main choreographic steps of a dance falls under choreographic modeling, which is an application of HPC in dance. In this paper, we explore appropriate methods for choreographic modeling using image data, where we cover Convolutional Neural Networks (CNNs) and Tensor Regression (TR). CNNs are well-known in image classification since they are constructed more efficiently than Artificial Neural Networks (ANNs) for working with image data. TR is an extension of the regression problem using tensor representations, which would be more appropriate than classical regression for use with image data. We do a comparison study between CNNs and TR on a dance dataset, aiming to predict all poses over two trials. The first trial uses a standard training-test split across all dancers, while the second follows a leave-one-out approach, training on all but one dancer and testing on the excluded dancer. Both models correctly predicted all poses in the first trial, whereas the second trial proved more complicated, with fewer poses classified. CNNs yielded higher performance metrics compared to TR, where TR generally had worse results. However, CNNs contained significantly more parameters than TR, leading to signs of overfitting in the second trial.