Audio- and video-based human activity recognition systems in healthcare

Abstract

In recent years, human activity recognition (HAR) has gained importance in several domains such as surveillance, recognizing indoor and outdoor activities, and providing active and assisted living environments in smart homes and healthcare services. In all these scenarios, audio-, video- and image-based processing algorithms have been applied as well as systems using wearable sensors. This scoping review focuses on audio- and video-based activity recognition systems for healthcare applications. We provide a comprehensive overview of these systems and technologies and discuss their complexity, performance, robustness, stage of development, scalability as well as achievable privacy and security levels. Additionally, we present and discuss datasets that are designed for the evaluation of these activity recognition systems. Although a number of robust approaches have already been proposed, they still pose challenges when it comes to integrating them in larger systems or into clinical practice. We identify challenges for application of audio- and video-based HAR systems in real-word healthcare scenarios, draw recommendations and conclusions based on comparisons of existing approaches, and analyze future trends.