AI-based remote photoplethysmography : benchmarking on realistic datasets

Abstract

In this study, we investigate the performance of the Contrast-Phys AI model for remote photoplethysmography on datasets recorded in more realistic conditions, introducing a new dataset named CAMVISIM LAB for this purpose. The videos in CAMVISIM LAB were recorded in a lightly controlled lab environment, with participants lying 2 m from the camera and allowed to move their heads, speak, and act naturally. The Contrast-Phys model was evaluated on the CAMVISIM LAB data and the UBFC-RPPG and PURE datasets, as the latter two are among the most popular datasets for the performance assessment of rPPG methods. The initial evaluation of the heart rate estimation on CAMVISIM LAB was conducted by training a Contrast-Phys model from scratch using k-fold cross-validation, resulting in a mean absolute error (MAE) of 7.7 bpm. This was improved by using a model pre-trained on the UBFC dataset for rPPG estimation, which improved the result to 5.8 bpm. The error was further minimised by fine-tuning the model with the pretrained weights from UBFC and using our dataset for a second training run. This resulted in an improved MAE of 1.0 bpm when fine-tuning and freezing the initial layers of the Contrast-Phys model during training. This result outperformed the score achieved when using the same strategy on the PURE dataset.