Mark Hansen | Bristol UWE (original) (raw)

Papers by Mark Hansen

Automation in Construction, May 1, 2020

Enclosed spaces are common in built structures but pose a challenge to many forms of manual or ro... more Enclosed spaces are common in built structures but pose a challenge to many forms of manual or robotic surveying and maintenance tasks. Part of this challenge is to train robot systems to understand their environment without human intervention. This paper presents a method to automatically classify features within a closed void using deep learning. Specifically, the paper considers a robot placed under floorboards for the purpose of autonomously surveying the underfloor void. The robot uses images captured using an RGB camera to identify regions such as floorboards, joists, air vents and pipework. The paper first presents a standard mask regions convolutional neural network approach, which gives modest performance. The method is then enhanced using a two-stage transfer learning approach with an existing dataset for interior scenes. The conclusion from this work is that, even with limited training data, it is possible to automatically detect many common features of such areas.

Procedia Computer Science, 2010

A major consideration in state-of-the-art face recognition systems is the amount of data that is ... more A major consideration in state-of-the-art face recognition systems is the amount of data that is required to represent a face. Even a small (64 × 64) photograph of a face has 2 12 dimensions in which a face may sit. When large (> 1MB) photographs of faces are used, this represents a very large (and practically intractable) space and ways of reducing dimensionality without losing discriminatory information are needed for storing data for recognition. The eigenface technique, which is based upon Principal Components Analysis (PCA), is a well established dimension reduction method in face recognition research but does not have any biological basis. Humans excel at familiar face recognition and this paper attempts to show that modelling a biologically plausible process is a valid alternative approach to using eigenfaces for dimension reduction. Using a biologically inspired method to extract the certain facial discriminatory information which mirrors some of the idiosyncrasies of the human visual system, we show that recognition rates remain high despite 90% of the raw data being discarded.

Palmprints are of considerable interest as a reliable biometric, since they offer significant adv... more Palmprints are of considerable interest as a reliable biometric, since they offer significant advantages, such as greater user acceptance than fingerprint or iris recognition. 2D systems can be spoofed by a photograph of a hand; however, 3D avoids this by recovering and analysing 3D textures and profiles. 3D palmprints can also be captured in a contactless manner, which is critical for ensuring hygiene (something that is particularly important in relation to pandemics such as COVID-19), and ease of use. The gap in prior work, between low resolution wrinkle studies and high-resolution palmprint recognition, is bridged here using high-resolution non-contact photometric stereo. A camera and illuminants are synchronised with image capture to recover high-definition 3D texture data from the palm, which are then analysed to extract ridges and wrinkles. This novel low-cost approach, which can tolerate distortions inherent to unconstrained contactless palmprint acquisition, achieved a 0.1% equ

Proceedings of SPIE, Feb 8, 2017

We present a spine arch analysis method in dairy cows using overhead 3D video data. This method i... more We present a spine arch analysis method in dairy cows using overhead 3D video data. This method is aimed for early stage lameness detection. That is important in order to allow early treatment; and thus, reduce the animal suffering and minimize the high forecasted financial losses, caused by lameness. Our physical data collection setup is non-intrusive, covert and designed to allow full automation; therefore, it could be implemented on a large scale or daily basis with high accuracy. We track the animal's spine using shape index and curvedness measure from the 3D surface as she walks freely under the 3D camera. Our spinal analysis focuses on the thoracic vertebrae region, where we found most of the arching caused by lameness. A cubic polynomial is fitted to analyze the arch and estimate the locomotion soundness. We have found more accurate results by eliminating the regular neck/head movements' effect from the arch. Using 22-cow data set, we are able to achieve an early stage lameness detection accuracy of 95.4%.

We investigate two locomotion traits in dairy cows from overhead 3D video to observe lameness tre... more We investigate two locomotion traits in dairy cows from overhead 3D video to observe lameness trends. Detecting lameness-particularly at an early stage-is important in order to allow early treatment which maximizes detection benefits. The proposed physical setup is covert, non-intrusive and it facilitates full autonomy; therefore, it could be implemented on a large-scale or daily-basis with high accuracy. The algorithm automatically tracks features to key regions (i.e. spine, hook bones) using shape index and curvedness measure from the 3D map. The gait asymmetry trait is analysed in the form of a dynamic novel proxy derived from the pelvic height movements, as the animal walks. We have found this proxy sensitive to early lameness trends. The back arch trait is analysed using a fitted polynomial in the extracted spine region. The proposed methods in this paper could be implemented on other cattle breeds, equine or other quadruped animals for the purposes of locomotion assessment.

We present and experimentally demonstrate the potential effectiveness of a photometric stereo bas... more We present and experimentally demonstrate the potential effectiveness of a photometric stereo based high resolution system for capturing 3D handprints using visible light sources. The sub-surface vascular structures are also enhanced through the use of near-infrared light sources which offers a potentially useful technique to increase system security. In contrast to existing systems which locate specific minutiae features of a fingerprint, we propose to use a global/holistic approach based on the spatial frequencies of the handprint, and preliminary results on 11 subjects show the high potential of this approach for contactless biometric identification purposes

Proceedings of the 2019 3rd International Conference on Biometric Engineering and Applications, 2019

In recent decades, many 3D data acquisition methods have been developed to provide accurate and c... more In recent decades, many 3D data acquisition methods have been developed to provide accurate and cost-effective 3D captures of the human face. An example system, which can accommodate both research and commercial applications, is the Photoface device. Photoface is based on the photometric stereo imaging technique. To improve the recognition performance using Photoface captures, a novel landmarking algorithm is first proposed by thresholding surface normals maps. The development of landmarking algorithms specifically for photometric stereo captures enables region-based feature extraction and fills a gap in the 3D face landmarking literature. Nasal curves and spherical patches are then used respectively for recognition and are evaluated on the 3DE-VISIR database, which contains Photoface captures with expressions. The neutral vs. non-neutral matching results demonstrate high face recognition performance using spherical patches and a KFA classifier, achieving a R1RR of 97.26% when only 24 patches are selected for matching.

2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC), 2018

Weed control in pasture is a challenging problem that can be expensive and environmentally unfrie... more Weed control in pasture is a challenging problem that can be expensive and environmentally unfriendly. This paper proposes a novel method for recognition of broad-leaf weeds in pasture such that precision weed control can be achieved with reduced herbicide use. Both conventional machine learning algorithms and deep learning methods have been explored and compared to achieve high detection accuracy and robustness in real-world environments. In-pasture grass/weed image data have been captured for classifier training and algorithm validation. The proposed deep learning method has achieved 96.88% accuracy and is capable of detecting weeds in different pastures under various representative outdoor lighting conditions.

GigaScience, 2019

Background: Tracking and predicting the growth performance of plants in different environments is... more Background: Tracking and predicting the growth performance of plants in different environments is critical for predicting the impact of global climate change. Automated approaches for image capture and analysis have allowed for substantial increases in the throughput of quantitative growth trait measurements compared with manual assessments. Recent work has focused on adopting computer vision and machine learning approaches to improve the accuracy of automated plant phenotyping. Here we present PS-Plant, a low-cost and portable 3D plant phenotyping platform based on an imaging technique novel to plant phenotyping called photometric stereo (PS). Results: We calibrated PS-Plant to track the model plant Arabidopsis thaliana throughout the day-night (diel) cycle and investigated growth architecture under a variety of conditions to illustrate the dramatic effect of the environment on plant phenotype. We developed bespoke computer vision algorithms and assessed available deep neural network architectures to automate the segmentation of rosettes and individual leaves, and extract basic and more advanced traits from PS-derived data, including the tracking of 3D plant growth and diel leaf hyponastic movement. Furthermore, we have produced the first PS training data set, which includes 221 manually annotated Arabidopsis rosettes that were used for training and data analysis (1,768 images in total). A full protocol is provided, including all software components and an additional test data set. Conclusions: PS-Plant is a powerful new phenotyping tool for plant research that provides robust data at high temporal and spatial resolutions. The system is well-suited for small-and large-scale research and will help to accelerate bridging of the phenotype-to-genotype gap.

Procedia Computer Science, 2010

This paper is concerned with the acquisition of model data for automatic 3D face recognition appl... more This paper is concerned with the acquisition of model data for automatic 3D face recognition applications. As 3D methods become progressively more popular in face recognition research, the need for fast and accurate data capture has become crucial. This paper is motivated by this need and offers three primary contributions. Firstly, the paper demonstrates that four-source photometric stereo offers a potential means for data capture that is computationally and financially viable and easily deployable in commercial settings. We have shown that both visible light and less intrusive near infrared light is suitable for facial illumination. The second contribution is a detailed set of experimental results that compare the accuracy of the device to ground truth, which was captured using a commercial projected pattern range finder. Importantly, we show that not only is near infrared light a valid alternative to the more commonly exploited visible light, but that it actually gives more accurate reconstructions. Finally, we assess the validity of the Lambertian assumption on skin reflectance data and show that better results may be obtained by incorporating more advanced reflectance functions, such as the Oren-Nayar model.

We present a number of related novel methods for reducing the dimensionality of data for the purp... more We present a number of related novel methods for reducing the dimensionality of data for the purposes of 2D and 3D face recognition. Results from psychology show that humans are capable of very good recognition of low resolution images and caricatures. These findings have inspired our experiments into methods of effective dimension reduction. For experimentation we use a subset of the benchmark FRGCv2.0 database as well as our own photometric stereo ``Photoface'' database. Our approaches look at the effects of image resizing, and inclusion of pixels based on percentiles and variance. Via the best combination of these techniques we represent a 3D image using only 61 variables and achieve 95.75% recognition performance (only a 2.25% decrease from using all pixels). These variables are extracted using computationally efficient techniques instead of more intensive methods employed by Eigenface and Fisherface techniques and can additionally reduce processing time tenfold.

Applications of Machine Learning, 2019

Automatic identification and selective spraying of weeds (such as dock) in grass can provide very... more Automatic identification and selective spraying of weeds (such as dock) in grass can provide very significant long-term ecological and cost benefits. Although machine vision (with interface to suitable automation) provides an effective means of achieving this, the associated challenges are formidable, due to the complexity of the images. This results from factors such as the percentage of dock in the image being low, the presence of other plants such as clover and changes in the level of illumination. Here, these challenges are addressed by the application of Convolutional Neural Networks (CNNs) to images containing grass and dock; and grass, dock and white clover. The performance of conventionally- trained CNNs and those trained using ‘Transfer Learning’ was compared. This was done for increasingly small datasets, to assess the viability of each approach for projects where large amounts of training data are not available. Results show that CNNs provide considerable improvements over previous methods for classification of weeds in grass. While previous work has reported best accuracies of around 83%, here a conventionally-trained CNN attained 95.6% accuracy for the two-class dataset, with 94.9% for the three-class dataset (i.e. dock, clover and grass). Interestingly, use of Transfer learning, with as few as 50 samples per class, still provides accuracies of around 84%. This is very promising for agricultural businesses that, due to the high cost of collecting and processing large amounts of data, have not yet been able to employ Neural Network models. Therefore, the employment of CNNs, particularly when incorporating Transfer Learning, is a very powerful method for classification of weeds in grassland, and one that is worthy of further research.

Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

An estimated 20% of patients admitted to hospital wards are a ected by delirium. Early detection ... more An estimated 20% of patients admitted to hospital wards are a ected by delirium. Early detection is recommended to treat underlying causes of delirium, however workforce strain in general wards often causes it to remain undetected. This work proposes a robotic implementation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) to aid early detection of delirium. Interactive features of the assessment are performed by Human-robot Interaction while a Transformer-based deep learning model predicts the Richmond Agitation Sedation Scale (RASS) level of the patient from image sequences; thermal imaging is used to maintain patient anonymity. A user study involving 18 participants role-playing each of alert, agitated, and sedated levels of the RASS is performed to test the HRI components and collect a dataset for deep learning. The HRI system achieved accuracies of 1.0 and 0.833 for the inattention and disorganised thinking features of the CAM-ICU, respectively, while the trained action recognition model achieved a mean accuracy of 0.852 on the classi cation of RASS levels during cross-validation. The three features represent a complete set of capabilities for automated delirium detection using the CAM-ICU, and the results demonstrate the feasibility of real-world deployment in hospital general wards. CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI); • Computer systems organization → Robotics; • Applied computing → Health care information systems; • Computing methodologies → Vision for robotics.

2016 International Conference for Students on Applied Engineering (ISCAE), Oct 1, 2016

To date, there has not been a single method suitable for large-scale or regular-basis implementat... more To date, there has not been a single method suitable for large-scale or regular-basis implementation to analyze the locomotion of quadruped animals. Existing methods are not sensitive enough for detecting minor deviations from healthy gaits. That is important because these minor deviations could develop into a severe painful lameness condition. We introduce a dynamic novel proxy for early stage lameness by analyzing the height movements from an overhead-view 3D video data. These movements are derived from key regions (e.g. spine, hook joints, and sacroiliac joint). The features to these key regions are automatically tracked using shape index and curvedness threshold from the 3D map. Our system is fully automated, covert and non-intrusive. This directly affects the accuracy of the analysis as we are able to observe the animals without spooking them. We believe that our proposed method could be used on other animals, i.e. predator quadrupeds where human presence is difficult.

Automatic face recognition has been an active research area for the last four decades. This thesi... more Automatic face recognition has been an active research area for the last four decades. This thesis explores innovative bio-inspired concepts aimed at improved face recognition using surface normals. New directions in salient data representation are explored using data captured via a photometric stereo method from the University of the West of England's "Photoface" device. Accuracy assessments demonstrate the advantage of the capture format and the synergy offered by near infrared light sources in achieving more accurate results than under conventional visible light. Two 3D face databases have been created as part of the thesis-the publicly available Photoface database which contains 3187 images of 453 subjects and the 3DE-VISIR dataset which contains 363 images of 115 people with different expressions captured simultaneously under near infrared and visible light. The Photoface database is believed to be the first to capture naturalistic 3D face models. Subsets of these databases are then used to show the results of experiments inspired by the human visual system. Experimental results show that optimal recognition rates are achieved using surprisingly low resolution of only 10×10 pixels on surface normal data, which corresponds to the spatial frequency range of optimal human performance. Motivated by the observed increase in recognition speed and accuracy that occurs in humans when faces are caricatured, novel interpretations of caricaturing using outlying data and pixel locations with high variance show that performance remains disproportionately high when up to 90% of the data has been discarded. These direct methods of dimensionality reduction have useful implications for the storage and processing requirements for commercial face recognition systems. The novel variance approach is extended to recognise positive expressions with 90% accuracy which has useful implications for human-computer interaction as well as ensuring that a subject has the correct expression prior to recognition. Furthermore, the subject recognition rate is improved by removing those pixels which encode expression. ii Finally, preliminary work into feature detection on surface normals by extending Haar-like features is presented which is also shown to be useful for correcting the pose of the head as part of a fully operational device. The system operates with an accuracy of 98.65% at a false acceptance rate of only 0.01 on front facing heads with neutral expressions. The work has shown how new avenues of enquiry inspired by our observation of the human visual system can offer useful advantages towards achieving more robust autonomous computer-based facial recognition.

Computers in industry, 2018

Machine vision systems offer great potential for automating crop control, harvesting, fruit picki... more Machine vision systems offer great potential for automating crop control, harvesting, fruit picking, and a range of other agricultural tasks. However, most of the reported research on machine vision in agriculture involves a 2D approach, where the utility of the resulting data is often limited by effects such as parallax, perspective, occlusion and changes in background light - particularly when operating in the field. The 3D approach to plant and crop analysis described in this paper offers potential to obviate many of these difficulties by utilising the richer information that 3D data can generate. The methodologies presented, such as four-light photometric stereo, also provide advanced functionalities, such as an ability to robustly recover 3D surface texture from plants at very high resolution. This offers potential for enabling, for example, reliable detection of the meristem (the part of the plant where growth can take place), to within a few mm, for directed weeding (with all...

Computers in industry, 2018

Leaf venation extraction studies have been strongly discouraged by considerable challenges posed ... more Leaf venation extraction studies have been strongly discouraged by considerable challenges posed by venation architectures that are complex, diverse and subtle. Additionally, unpredictable local leaf curvatures, undesirable ambient illuminations, and abnormal conditions of leaves may coexist with other complications. While leaf venation extraction has high potential for assisting with plant phenotyping, speciation and modelling, its investigations to date have been confined to colour image acquisition and processing which are commonly confounded by the aforementioned biotic and abiotic variations. To bridge the gaps in this area, we have designed a 3D imaging system for leaf venation extraction, which can overcome dark or bright ambient illumination and can allow for 3D data reconstruction in high resolution. We further propose a novel leaf venation extraction algorithm that can obtain illumination-independent surface normal features by performing Photometric Stereo reconstruction a...

Computers in industry, 2018

Here we propose a low-cost automated system for the unobtrusive and continuous welfare monitoring... more Here we propose a low-cost automated system for the unobtrusive and continuous welfare monitoring of dairy cattle on the farm. We argue that effective and regular monitoring of multiple condition traits is not currently practicable and go on to propose 3D imaging technology able to acquire differing forms of related animal condition data (body condition, lameness and weight), concurrently using a single device. Results obtained under farm conditions in continuous operation are shown to be comparable or better than manual scoring of the herd. We also consider inherent limitations of using scoring and argue that sensitivity to relative change over successive observations offers greater benefit than the use of what may be considered abstract and arbitrary scoring systems.

Biosystems Engineering, 2017

Disclaimer UWE has obtained warranties from all depositors as to their title in the material depo... more Disclaimer UWE has obtained warranties from all depositors as to their title in the material deposited and as to their right to deposit such material. UWE makes no representation or warranties of commercial utility, title, or fitness for a particular purpose or any other warranty, express or implied in respect of any material deposited. UWE makes no representation that the use of the materials will not infringe any patent, copyright, trademark or other property or proprietary rights. UWE accepts no liability for any infringement of intellectual property rights in any material deposited but will remove such material from public view pending investigation in the event of an allegation of any such infringement.

Automation in Construction, May 1, 2020

Enclosed spaces are common in built structures but pose a challenge to many forms of manual or ro... more Enclosed spaces are common in built structures but pose a challenge to many forms of manual or robotic surveying and maintenance tasks. Part of this challenge is to train robot systems to understand their environment without human intervention. This paper presents a method to automatically classify features within a closed void using deep learning. Specifically, the paper considers a robot placed under floorboards for the purpose of autonomously surveying the underfloor void. The robot uses images captured using an RGB camera to identify regions such as floorboards, joists, air vents and pipework. The paper first presents a standard mask regions convolutional neural network approach, which gives modest performance. The method is then enhanced using a two-stage transfer learning approach with an existing dataset for interior scenes. The conclusion from this work is that, even with limited training data, it is possible to automatically detect many common features of such areas.

Procedia Computer Science, 2010

A major consideration in state-of-the-art face recognition systems is the amount of data that is ... more A major consideration in state-of-the-art face recognition systems is the amount of data that is required to represent a face. Even a small (64 × 64) photograph of a face has 2 12 dimensions in which a face may sit. When large (> 1MB) photographs of faces are used, this represents a very large (and practically intractable) space and ways of reducing dimensionality without losing discriminatory information are needed for storing data for recognition. The eigenface technique, which is based upon Principal Components Analysis (PCA), is a well established dimension reduction method in face recognition research but does not have any biological basis. Humans excel at familiar face recognition and this paper attempts to show that modelling a biologically plausible process is a valid alternative approach to using eigenfaces for dimension reduction. Using a biologically inspired method to extract the certain facial discriminatory information which mirrors some of the idiosyncrasies of the human visual system, we show that recognition rates remain high despite 90% of the raw data being discarded.

Palmprints are of considerable interest as a reliable biometric, since they offer significant adv... more Palmprints are of considerable interest as a reliable biometric, since they offer significant advantages, such as greater user acceptance than fingerprint or iris recognition. 2D systems can be spoofed by a photograph of a hand; however, 3D avoids this by recovering and analysing 3D textures and profiles. 3D palmprints can also be captured in a contactless manner, which is critical for ensuring hygiene (something that is particularly important in relation to pandemics such as COVID-19), and ease of use. The gap in prior work, between low resolution wrinkle studies and high-resolution palmprint recognition, is bridged here using high-resolution non-contact photometric stereo. A camera and illuminants are synchronised with image capture to recover high-definition 3D texture data from the palm, which are then analysed to extract ridges and wrinkles. This novel low-cost approach, which can tolerate distortions inherent to unconstrained contactless palmprint acquisition, achieved a 0.1% equ

Proceedings of SPIE, Feb 8, 2017

We present a spine arch analysis method in dairy cows using overhead 3D video data. This method i... more We present a spine arch analysis method in dairy cows using overhead 3D video data. This method is aimed for early stage lameness detection. That is important in order to allow early treatment; and thus, reduce the animal suffering and minimize the high forecasted financial losses, caused by lameness. Our physical data collection setup is non-intrusive, covert and designed to allow full automation; therefore, it could be implemented on a large scale or daily basis with high accuracy. We track the animal's spine using shape index and curvedness measure from the 3D surface as she walks freely under the 3D camera. Our spinal analysis focuses on the thoracic vertebrae region, where we found most of the arching caused by lameness. A cubic polynomial is fitted to analyze the arch and estimate the locomotion soundness. We have found more accurate results by eliminating the regular neck/head movements' effect from the arch. Using 22-cow data set, we are able to achieve an early stage lameness detection accuracy of 95.4%.

We investigate two locomotion traits in dairy cows from overhead 3D video to observe lameness tre... more We investigate two locomotion traits in dairy cows from overhead 3D video to observe lameness trends. Detecting lameness-particularly at an early stage-is important in order to allow early treatment which maximizes detection benefits. The proposed physical setup is covert, non-intrusive and it facilitates full autonomy; therefore, it could be implemented on a large-scale or daily-basis with high accuracy. The algorithm automatically tracks features to key regions (i.e. spine, hook bones) using shape index and curvedness measure from the 3D map. The gait asymmetry trait is analysed in the form of a dynamic novel proxy derived from the pelvic height movements, as the animal walks. We have found this proxy sensitive to early lameness trends. The back arch trait is analysed using a fitted polynomial in the extracted spine region. The proposed methods in this paper could be implemented on other cattle breeds, equine or other quadruped animals for the purposes of locomotion assessment.

We present and experimentally demonstrate the potential effectiveness of a photometric stereo bas... more We present and experimentally demonstrate the potential effectiveness of a photometric stereo based high resolution system for capturing 3D handprints using visible light sources. The sub-surface vascular structures are also enhanced through the use of near-infrared light sources which offers a potentially useful technique to increase system security. In contrast to existing systems which locate specific minutiae features of a fingerprint, we propose to use a global/holistic approach based on the spatial frequencies of the handprint, and preliminary results on 11 subjects show the high potential of this approach for contactless biometric identification purposes

Proceedings of the 2019 3rd International Conference on Biometric Engineering and Applications, 2019

In recent decades, many 3D data acquisition methods have been developed to provide accurate and c... more In recent decades, many 3D data acquisition methods have been developed to provide accurate and cost-effective 3D captures of the human face. An example system, which can accommodate both research and commercial applications, is the Photoface device. Photoface is based on the photometric stereo imaging technique. To improve the recognition performance using Photoface captures, a novel landmarking algorithm is first proposed by thresholding surface normals maps. The development of landmarking algorithms specifically for photometric stereo captures enables region-based feature extraction and fills a gap in the 3D face landmarking literature. Nasal curves and spherical patches are then used respectively for recognition and are evaluated on the 3DE-VISIR database, which contains Photoface captures with expressions. The neutral vs. non-neutral matching results demonstrate high face recognition performance using spherical patches and a KFA classifier, achieving a R1RR of 97.26% when only 24 patches are selected for matching.

2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC), 2018

Weed control in pasture is a challenging problem that can be expensive and environmentally unfrie... more Weed control in pasture is a challenging problem that can be expensive and environmentally unfriendly. This paper proposes a novel method for recognition of broad-leaf weeds in pasture such that precision weed control can be achieved with reduced herbicide use. Both conventional machine learning algorithms and deep learning methods have been explored and compared to achieve high detection accuracy and robustness in real-world environments. In-pasture grass/weed image data have been captured for classifier training and algorithm validation. The proposed deep learning method has achieved 96.88% accuracy and is capable of detecting weeds in different pastures under various representative outdoor lighting conditions.

GigaScience, 2019

Background: Tracking and predicting the growth performance of plants in different environments is... more Background: Tracking and predicting the growth performance of plants in different environments is critical for predicting the impact of global climate change. Automated approaches for image capture and analysis have allowed for substantial increases in the throughput of quantitative growth trait measurements compared with manual assessments. Recent work has focused on adopting computer vision and machine learning approaches to improve the accuracy of automated plant phenotyping. Here we present PS-Plant, a low-cost and portable 3D plant phenotyping platform based on an imaging technique novel to plant phenotyping called photometric stereo (PS). Results: We calibrated PS-Plant to track the model plant Arabidopsis thaliana throughout the day-night (diel) cycle and investigated growth architecture under a variety of conditions to illustrate the dramatic effect of the environment on plant phenotype. We developed bespoke computer vision algorithms and assessed available deep neural network architectures to automate the segmentation of rosettes and individual leaves, and extract basic and more advanced traits from PS-derived data, including the tracking of 3D plant growth and diel leaf hyponastic movement. Furthermore, we have produced the first PS training data set, which includes 221 manually annotated Arabidopsis rosettes that were used for training and data analysis (1,768 images in total). A full protocol is provided, including all software components and an additional test data set. Conclusions: PS-Plant is a powerful new phenotyping tool for plant research that provides robust data at high temporal and spatial resolutions. The system is well-suited for small-and large-scale research and will help to accelerate bridging of the phenotype-to-genotype gap.

Procedia Computer Science, 2010

This paper is concerned with the acquisition of model data for automatic 3D face recognition appl... more This paper is concerned with the acquisition of model data for automatic 3D face recognition applications. As 3D methods become progressively more popular in face recognition research, the need for fast and accurate data capture has become crucial. This paper is motivated by this need and offers three primary contributions. Firstly, the paper demonstrates that four-source photometric stereo offers a potential means for data capture that is computationally and financially viable and easily deployable in commercial settings. We have shown that both visible light and less intrusive near infrared light is suitable for facial illumination. The second contribution is a detailed set of experimental results that compare the accuracy of the device to ground truth, which was captured using a commercial projected pattern range finder. Importantly, we show that not only is near infrared light a valid alternative to the more commonly exploited visible light, but that it actually gives more accurate reconstructions. Finally, we assess the validity of the Lambertian assumption on skin reflectance data and show that better results may be obtained by incorporating more advanced reflectance functions, such as the Oren-Nayar model.

We present a number of related novel methods for reducing the dimensionality of data for the purp... more We present a number of related novel methods for reducing the dimensionality of data for the purposes of 2D and 3D face recognition. Results from psychology show that humans are capable of very good recognition of low resolution images and caricatures. These findings have inspired our experiments into methods of effective dimension reduction. For experimentation we use a subset of the benchmark FRGCv2.0 database as well as our own photometric stereo ``Photoface'' database. Our approaches look at the effects of image resizing, and inclusion of pixels based on percentiles and variance. Via the best combination of these techniques we represent a 3D image using only 61 variables and achieve 95.75% recognition performance (only a 2.25% decrease from using all pixels). These variables are extracted using computationally efficient techniques instead of more intensive methods employed by Eigenface and Fisherface techniques and can additionally reduce processing time tenfold.

Applications of Machine Learning, 2019

Automatic identification and selective spraying of weeds (such as dock) in grass can provide very... more Automatic identification and selective spraying of weeds (such as dock) in grass can provide very significant long-term ecological and cost benefits. Although machine vision (with interface to suitable automation) provides an effective means of achieving this, the associated challenges are formidable, due to the complexity of the images. This results from factors such as the percentage of dock in the image being low, the presence of other plants such as clover and changes in the level of illumination. Here, these challenges are addressed by the application of Convolutional Neural Networks (CNNs) to images containing grass and dock; and grass, dock and white clover. The performance of conventionally- trained CNNs and those trained using ‘Transfer Learning’ was compared. This was done for increasingly small datasets, to assess the viability of each approach for projects where large amounts of training data are not available. Results show that CNNs provide considerable improvements over previous methods for classification of weeds in grass. While previous work has reported best accuracies of around 83%, here a conventionally-trained CNN attained 95.6% accuracy for the two-class dataset, with 94.9% for the three-class dataset (i.e. dock, clover and grass). Interestingly, use of Transfer learning, with as few as 50 samples per class, still provides accuracies of around 84%. This is very promising for agricultural businesses that, due to the high cost of collecting and processing large amounts of data, have not yet been able to employ Neural Network models. Therefore, the employment of CNNs, particularly when incorporating Transfer Learning, is a very powerful method for classification of weeds in grassland, and one that is worthy of further research.

Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction

An estimated 20% of patients admitted to hospital wards are a ected by delirium. Early detection ... more An estimated 20% of patients admitted to hospital wards are a ected by delirium. Early detection is recommended to treat underlying causes of delirium, however workforce strain in general wards often causes it to remain undetected. This work proposes a robotic implementation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU) to aid early detection of delirium. Interactive features of the assessment are performed by Human-robot Interaction while a Transformer-based deep learning model predicts the Richmond Agitation Sedation Scale (RASS) level of the patient from image sequences; thermal imaging is used to maintain patient anonymity. A user study involving 18 participants role-playing each of alert, agitated, and sedated levels of the RASS is performed to test the HRI components and collect a dataset for deep learning. The HRI system achieved accuracies of 1.0 and 0.833 for the inattention and disorganised thinking features of the CAM-ICU, respectively, while the trained action recognition model achieved a mean accuracy of 0.852 on the classi cation of RASS levels during cross-validation. The three features represent a complete set of capabilities for automated delirium detection using the CAM-ICU, and the results demonstrate the feasibility of real-world deployment in hospital general wards. CCS CONCEPTS • Human-centered computing → Human computer interaction (HCI); • Computer systems organization → Robotics; • Applied computing → Health care information systems; • Computing methodologies → Vision for robotics.

2016 International Conference for Students on Applied Engineering (ISCAE), Oct 1, 2016

To date, there has not been a single method suitable for large-scale or regular-basis implementat... more To date, there has not been a single method suitable for large-scale or regular-basis implementation to analyze the locomotion of quadruped animals. Existing methods are not sensitive enough for detecting minor deviations from healthy gaits. That is important because these minor deviations could develop into a severe painful lameness condition. We introduce a dynamic novel proxy for early stage lameness by analyzing the height movements from an overhead-view 3D video data. These movements are derived from key regions (e.g. spine, hook joints, and sacroiliac joint). The features to these key regions are automatically tracked using shape index and curvedness threshold from the 3D map. Our system is fully automated, covert and non-intrusive. This directly affects the accuracy of the analysis as we are able to observe the animals without spooking them. We believe that our proposed method could be used on other animals, i.e. predator quadrupeds where human presence is difficult.

Automatic face recognition has been an active research area for the last four decades. This thesi... more Automatic face recognition has been an active research area for the last four decades. This thesis explores innovative bio-inspired concepts aimed at improved face recognition using surface normals. New directions in salient data representation are explored using data captured via a photometric stereo method from the University of the West of England's "Photoface" device. Accuracy assessments demonstrate the advantage of the capture format and the synergy offered by near infrared light sources in achieving more accurate results than under conventional visible light. Two 3D face databases have been created as part of the thesis-the publicly available Photoface database which contains 3187 images of 453 subjects and the 3DE-VISIR dataset which contains 363 images of 115 people with different expressions captured simultaneously under near infrared and visible light. The Photoface database is believed to be the first to capture naturalistic 3D face models. Subsets of these databases are then used to show the results of experiments inspired by the human visual system. Experimental results show that optimal recognition rates are achieved using surprisingly low resolution of only 10×10 pixels on surface normal data, which corresponds to the spatial frequency range of optimal human performance. Motivated by the observed increase in recognition speed and accuracy that occurs in humans when faces are caricatured, novel interpretations of caricaturing using outlying data and pixel locations with high variance show that performance remains disproportionately high when up to 90% of the data has been discarded. These direct methods of dimensionality reduction have useful implications for the storage and processing requirements for commercial face recognition systems. The novel variance approach is extended to recognise positive expressions with 90% accuracy which has useful implications for human-computer interaction as well as ensuring that a subject has the correct expression prior to recognition. Furthermore, the subject recognition rate is improved by removing those pixels which encode expression. ii Finally, preliminary work into feature detection on surface normals by extending Haar-like features is presented which is also shown to be useful for correcting the pose of the head as part of a fully operational device. The system operates with an accuracy of 98.65% at a false acceptance rate of only 0.01 on front facing heads with neutral expressions. The work has shown how new avenues of enquiry inspired by our observation of the human visual system can offer useful advantages towards achieving more robust autonomous computer-based facial recognition.

Computers in industry, 2018

Machine vision systems offer great potential for automating crop control, harvesting, fruit picki... more Machine vision systems offer great potential for automating crop control, harvesting, fruit picking, and a range of other agricultural tasks. However, most of the reported research on machine vision in agriculture involves a 2D approach, where the utility of the resulting data is often limited by effects such as parallax, perspective, occlusion and changes in background light - particularly when operating in the field. The 3D approach to plant and crop analysis described in this paper offers potential to obviate many of these difficulties by utilising the richer information that 3D data can generate. The methodologies presented, such as four-light photometric stereo, also provide advanced functionalities, such as an ability to robustly recover 3D surface texture from plants at very high resolution. This offers potential for enabling, for example, reliable detection of the meristem (the part of the plant where growth can take place), to within a few mm, for directed weeding (with all...

Computers in industry, 2018

Leaf venation extraction studies have been strongly discouraged by considerable challenges posed ... more Leaf venation extraction studies have been strongly discouraged by considerable challenges posed by venation architectures that are complex, diverse and subtle. Additionally, unpredictable local leaf curvatures, undesirable ambient illuminations, and abnormal conditions of leaves may coexist with other complications. While leaf venation extraction has high potential for assisting with plant phenotyping, speciation and modelling, its investigations to date have been confined to colour image acquisition and processing which are commonly confounded by the aforementioned biotic and abiotic variations. To bridge the gaps in this area, we have designed a 3D imaging system for leaf venation extraction, which can overcome dark or bright ambient illumination and can allow for 3D data reconstruction in high resolution. We further propose a novel leaf venation extraction algorithm that can obtain illumination-independent surface normal features by performing Photometric Stereo reconstruction a...

Computers in industry, 2018

Here we propose a low-cost automated system for the unobtrusive and continuous welfare monitoring... more Here we propose a low-cost automated system for the unobtrusive and continuous welfare monitoring of dairy cattle on the farm. We argue that effective and regular monitoring of multiple condition traits is not currently practicable and go on to propose 3D imaging technology able to acquire differing forms of related animal condition data (body condition, lameness and weight), concurrently using a single device. Results obtained under farm conditions in continuous operation are shown to be comparable or better than manual scoring of the herd. We also consider inherent limitations of using scoring and argue that sensitivity to relative change over successive observations offers greater benefit than the use of what may be considered abstract and arbitrary scoring systems.

Biosystems Engineering, 2017

Disclaimer UWE has obtained warranties from all depositors as to their title in the material depo... more Disclaimer UWE has obtained warranties from all depositors as to their title in the material deposited and as to their right to deposit such material. UWE makes no representation or warranties of commercial utility, title, or fitness for a particular purpose or any other warranty, express or implied in respect of any material deposited. UWE makes no representation that the use of the materials will not infringe any patent, copyright, trademark or other property or proprietary rights. UWE accepts no liability for any infringement of intellectual property rights in any material deposited but will remove such material from public view pending investigation in the event of an allegation of any such infringement.