Following the success of researchers at the MIT Media Lab estimating the average heart-rate using an RGB video feed obtained from a standard webcam (See: CardioCam), researchers at Affectiva explored the use of heart-rate as a signal for the more fine-grained task of emotion estimation. As research suggested that heart-rate variability (HRV) can be an important signal, Affectiva researchers focused on a new concept and developed a novel approach for accurately detecting individual heart-beats, not just average-heart-rate, through analysis of an RGB video. This approach, published at the 2015 IEEE Automatic Face and Gesture Recognition (FG 2015) conference, showed an improved ability to estimate heart-rate over existing state of the art-techniques and could prove relevant for those interested in the short-term fluctuations in heart-rate.
Abstract
A supervised machine learning approach to remote video-based heart rate (HR) estimation is proposed. We demonstrate the possibility of training a discriminative statistical model to estimate the Blood Volume Pulse signal (BVP) from the human face using ambient light and any off-the-shelf webcam. The proposed algorithm is 120 times faster than state of the art approach and returns a confidence metric to evaluate the HR estimates plausibility. The algorithm was evaluated against the state-of-the-art on 120 minutes of face videos, the largest video-based heart rate evaluation to date. The evaluation results showed a 53% decrease in the Root Mean Squared Error (RMSE) compared to state-of-the-art.