Abstract:
Remote photoplethysmography (rPPG) enables contactless monitoring of the blood volume pulse using a regular camera. Recent research focused on improved motion robustness, but the proposed blind source separation techniques (BSS) in RGB color space show limited success. We present an analysis of the motion problem, from which far superior chrominance-based methods emerge. For a population of 117 stationary subjects, we show our methods to perform in 92% good agreement (±1.96σ) with contact PPG, with RMSE and standard deviation both a factor of 2 better than BSS-based methods. In a fitness setting using a simple spectral peak detector, the obtained pulse-rate for modest motion (bike) improves from 79% to 98% correct, and for vigorous motion (stepping) from less than 11% to more than 48% correct. We expect the greatly improved robustness to considerably widen the application scope of the technology.
INTRODUCTION
PHOTOPLETHYSMOGRAPHY (PPG) is an optical technique to monitor various vital signs, like pulse rate, respiratory rate, and blood oxygenation, first described in the 1930s and highly popular because of its noninvasiveness. Essentially, PPG detects the optical absorption variations of the human skin due to the blood volume variations during the cardiac cycle.
Earlier work has shown that these variations can also be measured at a distance leading to remote-PPG (rPPG) [2], [3]. Publications have even shown successful rPPG using a regular color video camera in the ambient light conditions [4]–[7]. rPPG is a highly relevant development for cases where contact has to be prevented because of extreme sensitivity, e.g.,neonates, skin-damage, or when an increased unobtrusiveness is required/desired (surveillance, fitness).
The main concern with rPPG is robustness to subject motion. Recent research has brought some improvement, but significant motion renders current algorithms useless. At this point, we aim to contribute by analyzing how the motion enters the pulse signal, and deriving far superior motion robust rPPG algorithms from the analysis.
Essentially, all motion robust rPPG techniques profit from the fact that the variations in optical absorption of the human skin depend on the wavelength used. Motion of the skin relative to the sensor, on the other hand, mostly affects the light reflected or transmitted by the skin regardless the wavelength. Huelsbusch and Verkruijsse, in 2008, found that the PPG signal had different relative strength in the three color channels of a video camera pointed at the human skin [4], [8]. Huelsbusch first exploited this difference to achieve motion robustness by separating the noise and the PPG signal into two independent signals built as a linear combination of two color channels [8]. One combination approximated the clean pulse signal, the other the motion artifact, and the energy in the pulse signal was minimized to optimize the combination. Poh et al. extended this work proposing a linear combination of all three color channels defining three independent signals with independent component analysis (ICA) using non-Gaussianity as the criterion for independence [5]. Lewandowska et al. varied this concept defining three independent linear combinations of the color channels with principal component analysis (PCA) [6]. With both the blind source separation (BSS) techniques, the component that carries the pulse signal is a priori unknown. Commonly, the selection assumes that the pulse signal shows the strongest periodicity. Consequently, periodic motion as in a fitness setting will render the selection useless. Moreover, a
fairly long observation interval is required to have sufficient resolution in the frequency domain, which prohibits adaptation to quickly changing statistics.
In Section II of this paper, we shall analyze how motion enters the pulse signal. From this analysis, new techniques for rPPG emerge that shall be shown superior to all earlier methods both in signal-to-noise ratio (SNR) and motion robustness. In Section III, we provide the assessment details, the results of which are shown for 117 stationary subjects over a broad range of skin-types in Section IV, and for some subjects exercising in a gym to test motion robustness in Section V. Finally, we draw our conclusions in Section VI.
CONCLUSION
In this paper, we have analyzed rPPG using a color video camera. We showed why an attempt to achieve motion robustness with a ratio of two normalized color signals is problematic, due to the unpredictable normalization errors resulting from specular reflections at the skin surface, absent in contact PPG.
This put us on the track of using color difference signals in which this specular reflection component is eliminated, assuming white illumination. Elaborating this track, we derived a number of possible algorithms that separate the blood volume pulse signal from motion-induced distortions in a deterministic fashion.
We benchmarked these chrominance-based methods with algorithms that have been proposed in earlier research on this topic and are based on BSS with additional heuristics to select the proper signal from the resulting components.
We found an important advantage of our chrominance-based approach is that it eliminates this rather weak component selection heuristic, which can be expected to fail with periodic motion from exercise. As a related advantage, we found that the chrominance-based methods can more quickly adapt to changing conditions and can perform well with shorter latency. We also showed that the inherent drawback resulting from the “white illumination assumption,” can be successfully eliminated by a “skin-tone standardization.”
To evaluate the proposals, we have analyzed the accuracy of the alternative rPPG measurement techniques over a large population of 117 subjects. We have demonstrated that rPPG is providing pulse rates in 92% good agreement (±1.96σ) with a contact PPG sensor. Pearson’s correlation coefficient was very high, r = 1.00 for the best chrominance-based rPPG method and r = 0.97 for the worst, ICA-based method. The low error, RMSE = 0.4 (1.1) for the best (worst) method, between rPPG and the reference contact-PPG sensor shows interesting prospect for future applications.
Analyzing the measurements obtained for different skin types, we showed that pulse rate extraction using rPPG can be performed robustly regardless of the skin type, σ = 0.8 over all skin types for the best chrominance-based method. However, the signal-to-noise energy-ratio of the pulse signal decreased with the melanin content of the skin from roughly 9.5 dB for the lightest down to 4.5 dB for the darkest skin.
In a second experiment, we tested the motion robustness of several methods, moving the camera to the gym and analyzing the pulse rates obtained from a person exercising on a stationary bike and a stepping device, respectively.
This experiment confirmed the expected problem with the heuristics required by the BSS-based methods, as they could not reliably distinguish the pulse signal and the periodic motion distortion. This already occurred with quite modest motion, on the stationary bike, where our best chrominance-based method showed the pulse rate as the strongest spectral peak more than 98% of time, where ICA and PCA scored at best 79%. For more vigorous motion, on the stepping device, our best methods still showed the pulse rate as the highest spectral peak more than 48% of the time, while ICA and PCA completely failed with scores below 4% and 11%, respectively.
ACKNOWLEDGMENT
The authors would like to thank for the help and contributions of their Philips colleagues I. van Dijk, I. Kirenko, H. Kloosterman, A. van Leest, and W. Verkruijsse. They would also like to thank reviewers for their thorough feedback on the manuscript.