PUPIL-BASED HEART MONITORING: INFERRING HRV FROM SMARTPHONE UNLOCK BEHAVIOR

Abstract

Nowadays, heart disease is a fairly prevalent and serious condition that is really preventable with early intervention. As a result, everyday heart health monitoring has grown in significance. The majority of current mobile cardiac monitoring devices rely on either photo plethysmography (PPG) or seismocardiography (SCG). These techniques, however, are inconvenient and require extra equipment, making it impossible for patients to keep an eye on their hearts at any time or location. We explore exploiting the pupillary response when a user unlocks his or her phone using face recognition to infer the user's heart rate variability (HRV) at this period, allowing for heart monitoring, motivated by our finding of the relationship between pupil size and HRV. To do this, we provide PupilHeart, a computer vision-based mobile HRV monitoring platform that has a server side and a mobile terminal. When users unlock their phones using the front-facing camera, PupilHeart gathers information on pupil size changes on the mobile terminal. On the server side, the raw pupil size data is then pre-processed. In particular, PupilHeart finds time series properties linked to HRV using a one-dimensional convolutional neural network (1D-CNN). Furthermore, PupilHeart models the pupil and HRV by training a recurrent neural network (RNN) with three hidden layers. Every time a user unlocks their phone, PupilHeart uses this model to infer their heart rate variability (HRV) and determine their heart status. By enlisting 60 people, we develop PupilHeart and carry out field research and trials to completely assess its efficacy. Overall, the findings demonstrate that PupilHeart is capable of properly predicting the user's HRV.