Richard So | The Hong Kong University of Science and Technology (original) (raw)

Papers by Richard So

i-Perception 2023, Vol. 14(5), 1–16, 2023

Vection (illusion of self-motion) is known to be induced by watching large field-of-view (FOV) mo... more Vection (illusion of self-motion) is known to be induced by watching large field-of-view (FOV) moving scenes. In our study, we investigated vection induced by small FOV stimuli. Three experiments were conducted in 45 sessions to analyze vection provoked by moving scenes covering total FOVs as small as 10 square-degrees. Results indicated that 88% of the participants reported vection while watching two small patches of moving dots (1°horizontal by 5°vertical, each) placed on the left and right sides of the observers. This is less than a quarter of the total visual area of two Apple Watches viewed at a distance of 40 cm. Occlusion of the visual field between the two display patches significantly increased the levels of rated vection. Similarly, increasing the speed of the moving dots of the two display patches from about 5 to 25°/sec increased the levels of rated vection significantly. The location of the two patches in the horizontal visual field did not affect the vection perception significantly. When the two straight stripes of dots were moving in opposite directions, participants perceived circular vection. The observers connected the two stimuli in their minds and perceived them as parts of a single occluded background. The findings of this study are relevant to the design of mobile devices (e.g., smartphones) and wearable technology (e.g., smart watches) with small display areas.

Bookmarks Related papers MentionsView impact

Bioengineering 2023, 10(7), 851; 18 July 2023 , 2023

Remote Photoplethysmography (rPPG) is a contactless method that enables the detection of various ... more Remote Photoplethysmography (rPPG) is a contactless method that enables the detection of various physiological signals from facial videos. rPPG utilizes a digital camera to detect subtle changes in skin color to measure vital signs such as heart rate variability (HRV), an important biomarker related to the autonomous nervous system. This paper presents a novel contactless HRV extraction algorithm, WaveHRV, based on the Wavelet Scattering Transform technique, followed by adaptive bandpass filtering and inter-beat-interval (IBI) analysis. Furthermore, a novel method is introduced to preprocess noisy contact-based PPG signals. WaveHRV is bench-marked against existing algorithms and public datasets. Our results show that WaveHRV is promising and achieves the lowest mean absolute error (MAE) of 10.5 ms and 6.15 ms for RMSSD and SDNN on the UBFCrPPG dataset.

Bookmarks Related papers MentionsView impact

IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2023), 2023

Remote body temperature measurement using infrared thermography has been widely deployed worldwid... more Remote body temperature measurement using infrared thermography has been widely deployed worldwide to detect feverish persons, but the measurement accuracy is affected by various factors including ambient temperature and sensor-subject distance. We present a novel compensation model to address the undesirable interacting influence of ambient temperature and sensor-subject distance during remote facial temperature screening in real-world setting. We derived our model on site-data collected over 12 months and demonstrated the significant linear relationship between ambient temperature and the measured temperature from a thermal camera. In addition, the interaction between the effects of sensor-subject distance and ambient temperature on the measured temperature is significant. Our model can significantly reduce the measurement error (MAE) by 23.5% and is better than the best existing models. The model can also extend the detection distance by up to 46% with sensitivity and specificity over 90%.

Bookmarks Related papers MentionsView impact

IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2023), 2023

Remote photoplethysmography (rPPG) is a non-invasive and convenient approach for measuring human ... more Remote photoplethysmography (rPPG) is a non-invasive and convenient approach for measuring human vital signs using a camera. However, accurate measurement can be challenging due to the different illumination of the surrounding environment. In this study, we present a deep learning-based image enhancement model (IEM) inspired by the Retinex theory to improve the robustness of rPPG signal extraction and heart rate (HR) estimation in different lighting conditions. We fine-tuned the IEM with a timeshifted negative Pearson correlation between the PPG signal ground truth and the predicted rPPG signal from a pretrained 3D CNN (PhysNet). We evaluated our method using a publicly available dataset (BH-rPPG) of various lighting scenarios and our own private dataset. Our results demonstrate that our proposed model is generalizable and significantly improves rPPG extraction and HR estimation accuracies across a range of illumination intensities.

Bookmarks Related papers MentionsView impact

IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2023), 18-22 June 2023,, 2023

Remote photoplethysmography (rPPG) is an optical technique that enables both non-invasive and eff... more Remote photoplethysmography (rPPG) is an optical technique that enables both non-invasive and efficient measurement of vital signs from facial videos. However, the quality of rPPG measurements can be adversely affected by improper camera exposure control and bad lighting conditions. In this paper, we present a systematic study of camera exposure control settings, specifically gain and exposure time, in low-light environments. Our results indicate that manual adjustment of gain and exposure time can significantly improve the quality of rPPG measurements, enabling accurate vital sign measurement even in environments with illuminance levels as low as 25 lux. Furthermore, we demonstrate that the optimal brightness range for rPPG-based vital sign measurement depends on the sensitivity of the vital sign to the shape and peaks of the rPPG signal. These findings have important practical implications for the use of rPPG in healthcare and remote monitoring applications.

Bookmarks Related papers MentionsView impact

SID Display Week Symposium 2023, LA, 20-26 May 2023., 2023

VR HMD users can observe dynamic distortion (or global pupil swim). Our earlier study correlated ... more VR HMD users can observe dynamic distortion (or global pupil swim). Our earlier study correlated pupil swim to selected optic flow patterns and mathematically modeled discomfort. This study decomposed global pupil swim as a linear sum of orthogonal basis patterns for improved prediction of its perceptual effects for an improved perception model.

Bookmarks Related papers MentionsView impact

Healthcare , 2022

Blood pressure (BP) determines whether a person has hypertension and offers implications as to wh... more Blood pressure (BP) determines whether a person has hypertension and offers implications as to whether he or she could be affected by
cardiovascular disease. Cuff-based sphygmomanometers have traditionally provided both accuracy and reliability, but they require bulky equipment and relevant skills to obtain precise measurements. BP measurement from photoplethysmography (PPG) signals has become a promising alternative for convenient and unobtrusive BP monitoring. Moreover, the recent developments in remote photoplethysmography (rPPG) algorithms have enabled new innovations for contactless BP measurement. This paper illustrates the evolution of BP measurement
techniques from the biophysical theory, through the development of contact-based BP measurement from PPG signals, and to the modern innovations of contactless BP measurement from rPPG signals. We consolidate knowledge from a diverse background of academic research to highlight the importance of multi-feature analysis for improving measurement accuracy. We conclude with the ongoing challenges, opportunities, and possible future directions in this emerging field of research.

Bookmarks Related papers MentionsView impact

The IEEE/ACM international conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE) , 2022

Remote measurement of physiological signals through facial videos is an emerging and significant ... more Remote measurement of physiological signals through facial videos is an emerging and significant field of research. Through remote photoplethysmography (rPPG), RGB cameras can measure a person's heart rate variability (HRV) by analyzing subtle light variations on the skin. Fluctuations in HRV readings are caused by imbalances in the autonomic nervous system, such as experiencing a stressful event. This paper presents a novel method for HRV measurement from rPPG signals. We tested the model on 14 subjects participating in stress-inducing tasks. We compared our results against a contact-based ground truth device and demonstrated the potential for an off-the-shelf webcam to provide robust HRV measurement and subsequent stress estimation.

Bookmarks Related papers MentionsView impact

Contemporary Ergonomics , 1999

The prevalence rate of motion sickness occuπ巴nce and self-rated sickness susceptibility among Hon... more The prevalence rate of motion sickness occuπ巴nce and self-rated sickness susceptibility among Hong Kong Chinese aged between 19 to 29 are reported. The data are part of the results from a survey conducted at the Hong Kong University of Science and Technology. The sample size is 515 and the participants have been randomly selected from 5,600 undergraduate students. Results suggested that 14%±3% of the Hong Kong population ag巴d 19 to 29 would rate themselves as not susceptible to motion sickness while the prevalence rates for different levels of susceptibilities are: 'slightly susceptible ＇ ：﹒ 57%±4% ﹔ ' mod巴rately susc巴ptible' ： 22%±4% ﹔ ' very susceptible'; 6%±2%; and '巴xtremely susceptible': I %±0.08%. Effects of gender and sickness occu 汀•ences among different modes of transport are also reported.

Bookmarks Related papers MentionsView impact

The IEEE/ACM international conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), 2022

Remote photoplethysmography (rPPG) enables an effective way to measure human physiological signal... more Remote photoplethysmography (rPPG) enables an effective way to measure human physiological signals by capturing subtle light variations on the skin with an off-the-shelf camera. Due to its noncontact nature and compatibility with a widespread of existing hardware, rPPG offers significant potential in the future of digital health and wellness applications. We developed Vitals, a practical and cost-effective solution for remote vital signs measurement and health management that can be easily deployed to our everyday devices and embedded into our daily routines. In addition, the solution provides the feature to manage one's health information over time, which enables historical analysis of trends and further insights into health risk factors. Vitals demonstrates the promise of camera-based physiological monitoring and illustrates its potential as a tool for pervasive healthcare.

Bookmarks Related papers MentionsView impact

The 29th International Display Workshops (IDW '22), 14-16, December 2022, Fukuoka, Japan., 2022

Users of Virtual Reality (VR) expect to see space-stabilized scenes when they explore virtual env... more Users of Virtual Reality (VR) expect to see space-stabilized scenes when they explore virtual environments with combined head and eye movements. Like our experience in real world, users expect space-constancy during head and eye movements. This talk will present a study to demonstrate the breakdown of space-constancy during vestibule-ocular reflexes in VR.

Bookmarks Related papers MentionsView impact

The 21st IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2022), 2022

Today virtual reality (VR) technologies became more and more widespread and found strong applicat... more Today virtual reality (VR) technologies became more and more widespread and found strong applications in various domains. However the fear to experience motion sickness is still an important barrier for new VR users. Instead of moving physically, VR users experience virtual locomotion but their vestibular systems do not sense the self-motion that are visually induced by immersive displays. The mismatch in visual and vestibular senses causes sickness. Previous solutions actively reduce user’s field-of-view, introduce intruder in the view or alter their navigation. In this paper we propose a passive approach that partially simplify the virtual environment according to user navigation. One manual simplification approach has been proposed and prototyped to simplify the scene seen in the peripheral field of view. The optic flow is analyzed on the rendered images seen by users. The result shows that the simplification reduces the
perceived optic flow which is the main cause of the visually induced motion sickness (VIMS). This pilot study confirm the potential efficiency of reducing cybersickness through geometric simplification.

Bookmarks Related papers MentionsView impact

Journal of Clean Technologies and Environmental Policy, 2020

Despite the many advantages of third-generation biofuels, there are still numerous opportunities ... more Despite the many advantages of third-generation biofuels, there are still numerous opportunities to improve their production efficiency and streamline their commercialization. The unpredictability of cultivating biomass is a major challenge to consistent, efficient production. In particular, the cultivation of Spirulina platensis biomass for biofuel production is affected by various environmental factors such as light, temperature, pH and the nutrient concentration of water. Since controlling these factors is energy intensive, a biomass prediction model would be helpful in anticipating biomass production and in indicating necessary adjustments to the process to improve yield. In this case, earlier is clearly better. This study developed a machine learning-based early prediction model which identifies the earliest time during cultivation that the process parameters optical density and pH can accurately be used to predict biomass yield. In the case study, the early prediction model predicted the final biomass yield (on the 23 rd day) by the 8 th day of cultivation using ridge regression. Furthermore, an application of this model in pH control led to a 54.1% average improvement in biomass yield. This model may be used to monitor cultivation batches allowing problems (i.e., low yield) to be identified early. It can also be applied in process simulation and optimization to improve biomass yield. In summary, mathematical modelling can make the unpredictable biomass process more predictable, and improve production efficiency.

Bookmarks Related papers MentionsView impact

Open access: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022

Remote photoplethysmography (rPPG) is a contactless method to measure human vital signs by detect... more Remote photoplethysmography (rPPG) is a contactless method to measure human vital signs by detecting subtle skin color changes through a camera. Although many studies have used region of interest (ROI) selection tools to improve rPPG signal extraction, no study has investigated the influence of the ROI's surface orientation. We propose a novel 'angle map' representation of the face to study the effects of the surface orientation on the extracted rPPG signal. The angle map is generated by mapping each facial pixel to an angle of reflection (angle between the skin surface and the camera) calculated from the surface normal of the facial landmarks and the camera axis. Our results show that surface orientation significantly affects the correlation between the extracted rPPG signal and ground truth blood volume pulse (BVP). Regions with small angles of reflection contained stronger signals, which explains why areas near the cheeks and forehead are often chosen for rPPG signal extraction. Moreover, we applied a thresholding method to the angle map and demonstrated its potential for dynamic ROI selection, thereby optimising the rPPG signal extraction process.

Bookmarks Related papers MentionsView impact

Annual Conference of the European Association for Computer Graphics, 2020

Today virtual reality technologies is becoming more and more widespread and has found strong appl... more Today virtual reality technologies is becoming more and more widespread and has found strong applications in various domains. However, the fear to experience motion sickness is still an important barrier for VR users. Instead of moving physically, VR users experience virtual locomotion but their vestibular systems do not sense the self-motion that are visually induced by immersive displays. The mismatch in visual and vestibular senses causes sickness. Previous solutions actively reduce user's field-of-view and alter their navigation. In this paper we propose a passive approach that temporarily deforms geometrically the virtual environment according to user navigation. Two deformation methods have been prototyped and tested. The first one reduces the perceived optic flow which is the main cause of visually induced motion sickness. The second one encourages users to adopt smoother trajectories and reduce the cybersickness dose value. Both methods have the potential to be applied generically.

Bookmarks Related papers MentionsView impact

Envrionmental Research, 2022

In wastewater monitoring, detecting extremely high pollutant concentrations is necessary to prope... more In wastewater monitoring, detecting extremely high pollutant concentrations is necessary to properly calibrate the treatment process. However, existing hardware sensors have a limited linear range which may fail to measure extremely high levels of pollutants; and likewise, the conventional “soft” model sensors are not suitable for the highly-skewed data distributions either. This study developed a new soft sensor by using eXtreme Gradient Boosting (XGBoost) machine learning to ‘measure’ the wastewater organics (in terms of 5-day biochemical oxygen demand (BOD5)). The soft sensor was tested on influent and effluent BOD5 of two different wastewater treatment plants to validate the results. The model results showed that XGBoost can detect these extreme values better than conventional soft sensors. This new soft sensor can function using a sparse input matrix via XGBoost's sparsity awareness algorithm - which can address the limitation of the conventional soft sensor with the fallibility of supporting hardware sensors even.

Bookmarks Related papers MentionsView impact

Journal of Water Process Engineering, 2021

Software (soft) sensors have been developed by using mathematical modelling to translate easy-to-... more Software (soft) sensors have been developed by using mathematical modelling to translate easy-to-measure parameters or existing sensors into other important operating parameters. This review surveys the advancements of soft sensor development for water resource recovery facilities (WRRFs) with the intention of establishing a baseline for these soft sensor models. Although a variety of data-driven modelling approaches have been proposed, it is difficult to identify the state-of-the-art. This is because each study uses a unique WRRF dataset, which differ based on statistical attributes (e.g., range, distribution) and qualitative attributes (e.g., supporting on-line sensors, nature of the wastewater). This is a problem as certain methods may only be effective for datasets with specific attributes. Moreover, it makes direct comparison based on common performance measures inadequate and misleading. To address this, the current review summarized (1) the different supporting on-line sensors that have been used in soft sensor development; (2) the methods applied in soft sensor development as well as the specific problem addressed by these methods; and (3) model performance in relation to the source and size of the datasets.

Bookmarks Related papers MentionsView impact

IEEE CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2021

During the COVID-19 pandemic, temperature screening has emerged as a common practice in the infec... more During the COVID-19 pandemic, temperature screening has emerged as a common practice in the infection control pipeline. In particular, thermal imaging systems have risen in popularity for preliminary screening of individuals with elevated temperatures, especially in high throughput areas. However, remote temperature measurement is intrinsically complex and susceptible to unavoidable influences from the measuring environment. We study the effects of sensor-subject distance on remote temperature readings and present an infrared-based system for rapid temperature screening over long distances (2 m to 10 m). The system applies a state-of-the-art pose estimation algorithm to extract the face box locations, sensor-subject distances, and facial temperatures within a scene. For the use of infrared thermography in humans, we propose a thermal compensation model to correct the temperature of subjects measured at different distances and perform analyses to evaluate the trade-off between missing rate (elevated temperature does not trigger an alarm) and false alarm rate (normal temperature triggers an alarm). The experimental results show our system's promise to identify subjects with elevated temperatures and the potential to improve temperature screening protocols in different environments.

Bookmarks Related papers MentionsView impact

Frontier in Virtual Reality, 2021

Visually induced circular vection (CV) has been the subject of a wide range of functional brain a... more Visually induced circular vection (CV) has been the subject of a wide range of functional brain and behavioral research. Participants in MRI or PET studies on CV were mostly in a supine viewing position, while participants in behavioral studies on CV were mostly in an upright viewing position. This study examines the effects of viewing positions (upright and supine) on roll CV reported by 16 participants while watching random dots (92 × 60 degrees field-of-view) rotating at different angular velocities (2, 4, 8, 16, 32, 64 deg/s) for 30 s. Viewing positions affected roll CV durations differently depending on the stimulation velocities. At slower velocities (2, 4, and 8 deg/s), participants exhibited significantly longer roll CV sensations when they were sitting in an upright position as opposed to lying in a supine position. The onset of roll CV was also significantly earlier with participants in an upright position despite similar roll CV intensities in both viewing positions. Significant two-way interactions between effects of viewing positions and dot rotating velocities for some conditions were noted. Consistency between current findings and the hypothesis predicting a weaker roll CV in upright positions based upon perceived gravity by the otolith organs is discussed.

Bookmarks Related papers MentionsView impact

International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2020), 2020

This study proposes a bi-directional recurrent neural network (Bi-RNN) post-processing method for... more This study proposes a bi-directional recurrent neural network (Bi-RNN) post-processing method for speech enhancement (SE) at low signal-to noise ratios (SNR). Current speech enhancement solutions performed badly under low SNR situations. Loizou and Kim proposed a solution to reduce speech distortion errors in time-frequency (T-F) domain but it requires the knowledge of ground truth. As ground truth is unknown in real-life applications, the current study proposes to use a Bi-RNN to implement Loizou and Kim's solution as a post-processing method for SE engines. Our solutions do not require prior knowledge of ground truth. The effectiveness of the proposed method is investigated with a spectral subtraction (SS) SE engine, a non-negative matrix factorization (NMF) SE engine, and a deep neural network ideal ratio mask (DNN-IRM) SE engine, under matched/mis-matched noise and different SNR conditions. Experimental results demonstrate that the proposed post-processing method effectively improved both perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) for all of these SE engines, especially at low SNR conditions.

Bookmarks Related papers MentionsView impact

i-Perception 2023, Vol. 14(5), 1–16, 2023

Vection (illusion of self-motion) is known to be induced by watching large field-of-view (FOV) mo... more Vection (illusion of self-motion) is known to be induced by watching large field-of-view (FOV) moving scenes. In our study, we investigated vection induced by small FOV stimuli. Three experiments were conducted in 45 sessions to analyze vection provoked by moving scenes covering total FOVs as small as 10 square-degrees. Results indicated that 88% of the participants reported vection while watching two small patches of moving dots (1°horizontal by 5°vertical, each) placed on the left and right sides of the observers. This is less than a quarter of the total visual area of two Apple Watches viewed at a distance of 40 cm. Occlusion of the visual field between the two display patches significantly increased the levels of rated vection. Similarly, increasing the speed of the moving dots of the two display patches from about 5 to 25°/sec increased the levels of rated vection significantly. The location of the two patches in the horizontal visual field did not affect the vection perception significantly. When the two straight stripes of dots were moving in opposite directions, participants perceived circular vection. The observers connected the two stimuli in their minds and perceived them as parts of a single occluded background. The findings of this study are relevant to the design of mobile devices (e.g., smartphones) and wearable technology (e.g., smart watches) with small display areas.

Bookmarks Related papers MentionsView impact

Bioengineering 2023, 10(7), 851; 18 July 2023 , 2023

Remote Photoplethysmography (rPPG) is a contactless method that enables the detection of various ... more Remote Photoplethysmography (rPPG) is a contactless method that enables the detection of various physiological signals from facial videos. rPPG utilizes a digital camera to detect subtle changes in skin color to measure vital signs such as heart rate variability (HRV), an important biomarker related to the autonomous nervous system. This paper presents a novel contactless HRV extraction algorithm, WaveHRV, based on the Wavelet Scattering Transform technique, followed by adaptive bandpass filtering and inter-beat-interval (IBI) analysis. Furthermore, a novel method is introduced to preprocess noisy contact-based PPG signals. WaveHRV is bench-marked against existing algorithms and public datasets. Our results show that WaveHRV is promising and achieves the lowest mean absolute error (MAE) of 10.5 ms and 6.15 ms for RMSSD and SDNN on the UBFCrPPG dataset.

Bookmarks Related papers MentionsView impact

IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2023), 2023

Remote body temperature measurement using infrared thermography has been widely deployed worldwid... more Remote body temperature measurement using infrared thermography has been widely deployed worldwide to detect feverish persons, but the measurement accuracy is affected by various factors including ambient temperature and sensor-subject distance. We present a novel compensation model to address the undesirable interacting influence of ambient temperature and sensor-subject distance during remote facial temperature screening in real-world setting. We derived our model on site-data collected over 12 months and demonstrated the significant linear relationship between ambient temperature and the measured temperature from a thermal camera. In addition, the interaction between the effects of sensor-subject distance and ambient temperature on the measured temperature is significant. Our model can significantly reduce the measurement error (MAE) by 23.5% and is better than the best existing models. The model can also extend the detection distance by up to 46% with sensitivity and specificity over 90%.

Bookmarks Related papers MentionsView impact

IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2023), 2023

Remote photoplethysmography (rPPG) is a non-invasive and convenient approach for measuring human ... more Remote photoplethysmography (rPPG) is a non-invasive and convenient approach for measuring human vital signs using a camera. However, accurate measurement can be challenging due to the different illumination of the surrounding environment. In this study, we present a deep learning-based image enhancement model (IEM) inspired by the Retinex theory to improve the robustness of rPPG signal extraction and heart rate (HR) estimation in different lighting conditions. We fine-tuned the IEM with a timeshifted negative Pearson correlation between the PPG signal ground truth and the predicted rPPG signal from a pretrained 3D CNN (PhysNet). We evaluated our method using a publicly available dataset (BH-rPPG) of various lighting scenarios and our own private dataset. Our results demonstrate that our proposed model is generalizable and significantly improves rPPG extraction and HR estimation accuracies across a range of illumination intensities.

Bookmarks Related papers MentionsView impact

IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2023), 18-22 June 2023,, 2023

Remote photoplethysmography (rPPG) is an optical technique that enables both non-invasive and eff... more Remote photoplethysmography (rPPG) is an optical technique that enables both non-invasive and efficient measurement of vital signs from facial videos. However, the quality of rPPG measurements can be adversely affected by improper camera exposure control and bad lighting conditions. In this paper, we present a systematic study of camera exposure control settings, specifically gain and exposure time, in low-light environments. Our results indicate that manual adjustment of gain and exposure time can significantly improve the quality of rPPG measurements, enabling accurate vital sign measurement even in environments with illuminance levels as low as 25 lux. Furthermore, we demonstrate that the optimal brightness range for rPPG-based vital sign measurement depends on the sensitivity of the vital sign to the shape and peaks of the rPPG signal. These findings have important practical implications for the use of rPPG in healthcare and remote monitoring applications.

Bookmarks Related papers MentionsView impact

SID Display Week Symposium 2023, LA, 20-26 May 2023., 2023

VR HMD users can observe dynamic distortion (or global pupil swim). Our earlier study correlated ... more VR HMD users can observe dynamic distortion (or global pupil swim). Our earlier study correlated pupil swim to selected optic flow patterns and mathematically modeled discomfort. This study decomposed global pupil swim as a linear sum of orthogonal basis patterns for improved prediction of its perceptual effects for an improved perception model.

Bookmarks Related papers MentionsView impact

Healthcare , 2022

Blood pressure (BP) determines whether a person has hypertension and offers implications as to wh... more Blood pressure (BP) determines whether a person has hypertension and offers implications as to whether he or she could be affected by
cardiovascular disease. Cuff-based sphygmomanometers have traditionally provided both accuracy and reliability, but they require bulky equipment and relevant skills to obtain precise measurements. BP measurement from photoplethysmography (PPG) signals has become a promising alternative for convenient and unobtrusive BP monitoring. Moreover, the recent developments in remote photoplethysmography (rPPG) algorithms have enabled new innovations for contactless BP measurement. This paper illustrates the evolution of BP measurement
techniques from the biophysical theory, through the development of contact-based BP measurement from PPG signals, and to the modern innovations of contactless BP measurement from rPPG signals. We consolidate knowledge from a diverse background of academic research to highlight the importance of multi-feature analysis for improving measurement accuracy. We conclude with the ongoing challenges, opportunities, and possible future directions in this emerging field of research.

Bookmarks Related papers MentionsView impact

The IEEE/ACM international conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE) , 2022

Remote measurement of physiological signals through facial videos is an emerging and significant ... more Remote measurement of physiological signals through facial videos is an emerging and significant field of research. Through remote photoplethysmography (rPPG), RGB cameras can measure a person's heart rate variability (HRV) by analyzing subtle light variations on the skin. Fluctuations in HRV readings are caused by imbalances in the autonomic nervous system, such as experiencing a stressful event. This paper presents a novel method for HRV measurement from rPPG signals. We tested the model on 14 subjects participating in stress-inducing tasks. We compared our results against a contact-based ground truth device and demonstrated the potential for an off-the-shelf webcam to provide robust HRV measurement and subsequent stress estimation.

Bookmarks Related papers MentionsView impact

Contemporary Ergonomics , 1999

The prevalence rate of motion sickness occuπ巴nce and self-rated sickness susceptibility among Hon... more The prevalence rate of motion sickness occuπ巴nce and self-rated sickness susceptibility among Hong Kong Chinese aged between 19 to 29 are reported. The data are part of the results from a survey conducted at the Hong Kong University of Science and Technology. The sample size is 515 and the participants have been randomly selected from 5,600 undergraduate students. Results suggested that 14%±3% of the Hong Kong population ag巴d 19 to 29 would rate themselves as not susceptible to motion sickness while the prevalence rates for different levels of susceptibilities are: 'slightly susceptible ＇ ：﹒ 57%±4% ﹔ ' mod巴rately susc巴ptible' ： 22%±4% ﹔ ' very susceptible'; 6%±2%; and '巴xtremely susceptible': I %±0.08%. Effects of gender and sickness occu 汀•ences among different modes of transport are also reported.

Bookmarks Related papers MentionsView impact

The IEEE/ACM international conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), 2022

Remote photoplethysmography (rPPG) enables an effective way to measure human physiological signal... more Remote photoplethysmography (rPPG) enables an effective way to measure human physiological signals by capturing subtle light variations on the skin with an off-the-shelf camera. Due to its noncontact nature and compatibility with a widespread of existing hardware, rPPG offers significant potential in the future of digital health and wellness applications. We developed Vitals, a practical and cost-effective solution for remote vital signs measurement and health management that can be easily deployed to our everyday devices and embedded into our daily routines. In addition, the solution provides the feature to manage one's health information over time, which enables historical analysis of trends and further insights into health risk factors. Vitals demonstrates the promise of camera-based physiological monitoring and illustrates its potential as a tool for pervasive healthcare.

Bookmarks Related papers MentionsView impact

The 29th International Display Workshops (IDW '22), 14-16, December 2022, Fukuoka, Japan., 2022

Users of Virtual Reality (VR) expect to see space-stabilized scenes when they explore virtual env... more Users of Virtual Reality (VR) expect to see space-stabilized scenes when they explore virtual environments with combined head and eye movements. Like our experience in real world, users expect space-constancy during head and eye movements. This talk will present a study to demonstrate the breakdown of space-constancy during vestibule-ocular reflexes in VR.

Bookmarks Related papers MentionsView impact

The 21st IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2022), 2022

Today virtual reality (VR) technologies became more and more widespread and found strong applicat... more Today virtual reality (VR) technologies became more and more widespread and found strong applications in various domains. However the fear to experience motion sickness is still an important barrier for new VR users. Instead of moving physically, VR users experience virtual locomotion but their vestibular systems do not sense the self-motion that are visually induced by immersive displays. The mismatch in visual and vestibular senses causes sickness. Previous solutions actively reduce user’s field-of-view, introduce intruder in the view or alter their navigation. In this paper we propose a passive approach that partially simplify the virtual environment according to user navigation. One manual simplification approach has been proposed and prototyped to simplify the scene seen in the peripheral field of view. The optic flow is analyzed on the rendered images seen by users. The result shows that the simplification reduces the
perceived optic flow which is the main cause of the visually induced motion sickness (VIMS). This pilot study confirm the potential efficiency of reducing cybersickness through geometric simplification.

Bookmarks Related papers MentionsView impact

Journal of Clean Technologies and Environmental Policy, 2020

Despite the many advantages of third-generation biofuels, there are still numerous opportunities ... more Despite the many advantages of third-generation biofuels, there are still numerous opportunities to improve their production efficiency and streamline their commercialization. The unpredictability of cultivating biomass is a major challenge to consistent, efficient production. In particular, the cultivation of Spirulina platensis biomass for biofuel production is affected by various environmental factors such as light, temperature, pH and the nutrient concentration of water. Since controlling these factors is energy intensive, a biomass prediction model would be helpful in anticipating biomass production and in indicating necessary adjustments to the process to improve yield. In this case, earlier is clearly better. This study developed a machine learning-based early prediction model which identifies the earliest time during cultivation that the process parameters optical density and pH can accurately be used to predict biomass yield. In the case study, the early prediction model predicted the final biomass yield (on the 23 rd day) by the 8 th day of cultivation using ridge regression. Furthermore, an application of this model in pH control led to a 54.1% average improvement in biomass yield. This model may be used to monitor cultivation batches allowing problems (i.e., low yield) to be identified early. It can also be applied in process simulation and optimization to improve biomass yield. In summary, mathematical modelling can make the unpredictable biomass process more predictable, and improve production efficiency.

Bookmarks Related papers MentionsView impact

Open access: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022

Remote photoplethysmography (rPPG) is a contactless method to measure human vital signs by detect... more Remote photoplethysmography (rPPG) is a contactless method to measure human vital signs by detecting subtle skin color changes through a camera. Although many studies have used region of interest (ROI) selection tools to improve rPPG signal extraction, no study has investigated the influence of the ROI's surface orientation. We propose a novel 'angle map' representation of the face to study the effects of the surface orientation on the extracted rPPG signal. The angle map is generated by mapping each facial pixel to an angle of reflection (angle between the skin surface and the camera) calculated from the surface normal of the facial landmarks and the camera axis. Our results show that surface orientation significantly affects the correlation between the extracted rPPG signal and ground truth blood volume pulse (BVP). Regions with small angles of reflection contained stronger signals, which explains why areas near the cheeks and forehead are often chosen for rPPG signal extraction. Moreover, we applied a thresholding method to the angle map and demonstrated its potential for dynamic ROI selection, thereby optimising the rPPG signal extraction process.

Bookmarks Related papers MentionsView impact

Annual Conference of the European Association for Computer Graphics, 2020

Today virtual reality technologies is becoming more and more widespread and has found strong appl... more Today virtual reality technologies is becoming more and more widespread and has found strong applications in various domains. However, the fear to experience motion sickness is still an important barrier for VR users. Instead of moving physically, VR users experience virtual locomotion but their vestibular systems do not sense the self-motion that are visually induced by immersive displays. The mismatch in visual and vestibular senses causes sickness. Previous solutions actively reduce user's field-of-view and alter their navigation. In this paper we propose a passive approach that temporarily deforms geometrically the virtual environment according to user navigation. Two deformation methods have been prototyped and tested. The first one reduces the perceived optic flow which is the main cause of visually induced motion sickness. The second one encourages users to adopt smoother trajectories and reduce the cybersickness dose value. Both methods have the potential to be applied generically.

Bookmarks Related papers MentionsView impact

Envrionmental Research, 2022

In wastewater monitoring, detecting extremely high pollutant concentrations is necessary to prope... more In wastewater monitoring, detecting extremely high pollutant concentrations is necessary to properly calibrate the treatment process. However, existing hardware sensors have a limited linear range which may fail to measure extremely high levels of pollutants; and likewise, the conventional “soft” model sensors are not suitable for the highly-skewed data distributions either. This study developed a new soft sensor by using eXtreme Gradient Boosting (XGBoost) machine learning to ‘measure’ the wastewater organics (in terms of 5-day biochemical oxygen demand (BOD5)). The soft sensor was tested on influent and effluent BOD5 of two different wastewater treatment plants to validate the results. The model results showed that XGBoost can detect these extreme values better than conventional soft sensors. This new soft sensor can function using a sparse input matrix via XGBoost's sparsity awareness algorithm - which can address the limitation of the conventional soft sensor with the fallibility of supporting hardware sensors even.

Bookmarks Related papers MentionsView impact

Journal of Water Process Engineering, 2021

Software (soft) sensors have been developed by using mathematical modelling to translate easy-to-... more Software (soft) sensors have been developed by using mathematical modelling to translate easy-to-measure parameters or existing sensors into other important operating parameters. This review surveys the advancements of soft sensor development for water resource recovery facilities (WRRFs) with the intention of establishing a baseline for these soft sensor models. Although a variety of data-driven modelling approaches have been proposed, it is difficult to identify the state-of-the-art. This is because each study uses a unique WRRF dataset, which differ based on statistical attributes (e.g., range, distribution) and qualitative attributes (e.g., supporting on-line sensors, nature of the wastewater). This is a problem as certain methods may only be effective for datasets with specific attributes. Moreover, it makes direct comparison based on common performance measures inadequate and misleading. To address this, the current review summarized (1) the different supporting on-line sensors that have been used in soft sensor development; (2) the methods applied in soft sensor development as well as the specific problem addressed by these methods; and (3) model performance in relation to the source and size of the datasets.

Bookmarks Related papers MentionsView impact

IEEE CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2021

During the COVID-19 pandemic, temperature screening has emerged as a common practice in the infec... more During the COVID-19 pandemic, temperature screening has emerged as a common practice in the infection control pipeline. In particular, thermal imaging systems have risen in popularity for preliminary screening of individuals with elevated temperatures, especially in high throughput areas. However, remote temperature measurement is intrinsically complex and susceptible to unavoidable influences from the measuring environment. We study the effects of sensor-subject distance on remote temperature readings and present an infrared-based system for rapid temperature screening over long distances (2 m to 10 m). The system applies a state-of-the-art pose estimation algorithm to extract the face box locations, sensor-subject distances, and facial temperatures within a scene. For the use of infrared thermography in humans, we propose a thermal compensation model to correct the temperature of subjects measured at different distances and perform analyses to evaluate the trade-off between missing rate (elevated temperature does not trigger an alarm) and false alarm rate (normal temperature triggers an alarm). The experimental results show our system's promise to identify subjects with elevated temperatures and the potential to improve temperature screening protocols in different environments.

Bookmarks Related papers MentionsView impact

Frontier in Virtual Reality, 2021

Visually induced circular vection (CV) has been the subject of a wide range of functional brain a... more Visually induced circular vection (CV) has been the subject of a wide range of functional brain and behavioral research. Participants in MRI or PET studies on CV were mostly in a supine viewing position, while participants in behavioral studies on CV were mostly in an upright viewing position. This study examines the effects of viewing positions (upright and supine) on roll CV reported by 16 participants while watching random dots (92 × 60 degrees field-of-view) rotating at different angular velocities (2, 4, 8, 16, 32, 64 deg/s) for 30 s. Viewing positions affected roll CV durations differently depending on the stimulation velocities. At slower velocities (2, 4, and 8 deg/s), participants exhibited significantly longer roll CV sensations when they were sitting in an upright position as opposed to lying in a supine position. The onset of roll CV was also significantly earlier with participants in an upright position despite similar roll CV intensities in both viewing positions. Significant two-way interactions between effects of viewing positions and dot rotating velocities for some conditions were noted. Consistency between current findings and the hypothesis predicting a weaker roll CV in upright positions based upon perceived gravity by the otolith organs is discussed.

Bookmarks Related papers MentionsView impact

International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2020), 2020

This study proposes a bi-directional recurrent neural network (Bi-RNN) post-processing method for... more This study proposes a bi-directional recurrent neural network (Bi-RNN) post-processing method for speech enhancement (SE) at low signal-to noise ratios (SNR). Current speech enhancement solutions performed badly under low SNR situations. Loizou and Kim proposed a solution to reduce speech distortion errors in time-frequency (T-F) domain but it requires the knowledge of ground truth. As ground truth is unknown in real-life applications, the current study proposes to use a Bi-RNN to implement Loizou and Kim's solution as a post-processing method for SE engines. Our solutions do not require prior knowledge of ground truth. The effectiveness of the proposed method is investigated with a spectral subtraction (SS) SE engine, a non-negative matrix factorization (NMF) SE engine, and a deep neural network ideal ratio mask (DNN-IRM) SE engine, under matched/mis-matched noise and different SNR conditions. Experimental results demonstrate that the proposed post-processing method effectively improved both perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI) for all of these SE engines, especially at low SNR conditions.

Bookmarks Related papers MentionsView impact