Dlib Research Papers - Academia.edu (original) (raw)

There are a variety of potential uses for the classification of eye conditions, including tiredness detection, psychological condition evaluation, etc. Because of its significance, many studies utilizing typical neural network algorithms... more

There are a variety of potential uses for the classification of eye conditions, including tiredness detection, psychological condition evaluation, etc. Because of its significance, many studies utilizing typical neural network algorithms have already been published in the literature, with good results. Convolutional neural networks (CNNs) are employed in real-time applications to achieve two goals: high accuracy and speed. However, identifying drowsiness at an early stage significantly improves the chances of being saved from accidents. Drowsiness detection can be automated by using the potential of artificial intelligence (AI), which allows us to assess more cases in less time and with a lower cost. With the help of modern deep learning (DL) and digital image processing (DIP) techniques, in this paper, we suggest a CNN model for eye state categorization, and we tested it on three CNN models (VGG16, VGG19, and 4D). A novel CNN model named the 4D model was designed to detect drowsine...

Blink detection is an important technique in a variety of settings, including facial movement analysis and signal processing. However, automatic blink detection is very challenging because of the blink rate. This research work proposed a... more

Blink detection is an important technique in a variety of settings, including facial movement analysis and signal processing. However, automatic blink detection is very challenging because of the blink rate. This research work proposed a real-time method for detecting eye blinks in a video series. Automatic facial landmarks detectors are trained on a real-world dataset and demonstrate exceptional resilience to a wide range of environmental factors, including lighting conditions, face emotions, and head position. For each video frame, the proposed method calculates the facial landmark locations and extracts the vertical distance between the eyelids using the facial landmark positions. Our results show that the recognizable landmarks are sufficiently accurate to determine the degree of eye-opening and closing consistently. The proposed algorithm estimates the facial landmark positions, extracts a single scalar quantity by using Modified Eye Aspect Ratio (Modified EAR) and characterizi...

Driver fatigue and drowsiness are some of the prime reasons for road accidents around the globe. Anyone can become a victim of drowsiness while driving after tiring physical conditions, short periods of sleep, or during long journeys.... more

Driver fatigue and drowsiness are some of the prime reasons for road accidents around the globe. Anyone can become a victim of drowsiness while driving after tiring physical conditions, short periods of sleep, or during long journeys. Drowsiness or inactivity causes effects driving in these couple of major areas. It increases reaction time, loss of coordination which makes drivers respond late which may result in an occurrence of accidents, several injuries, may loss of people. Every year more than 1,000,000 police-reported crashes involve drowsy driving. The occurrence of road accidents has been increasing, with the rise in population. Due to this, various studies were done in designing systems that can examine inactivity and drowsiness or driver fatigue and alert him/her beforehand thus preventing them from falling asleep and causing an accident. Our proposed method is to develop a deep learning algorithm using Convolutional Neural Networks, Computer Vision to detect drowsiness of...

Due to progress in the field of deep learning in order to find and track objects through the use of computer vision in the service of large segments of the population, as it was adopted in the field of serving people b with special needs... more

Due to progress in the field of deep learning in order to find and track objects through the use of computer vision in the service of large segments of the population, as it was adopted in the field of serving people b with special needs for the sake of dialogue and implementation of many requests in this research, a series of commands for use by people with special needs with speech problems or paralysis, where the ability to use the eye blink is very useful for social communication, were developed. In this research, the orders needed by the target people with special needs were studied, and (11) commands were identified that can be increased according to the intended sample. A table of commands was built depending on the length of the eye blink period. By creating a modified CNN: Convolutional Neural Network structure and training it on 2 different database, deep learning was used to identify and determine if the eye is closed or open (mrlEye2018 and Closed Eye in the Wlid:CEW).It...

Background: The purpose of this review paper is to develop a driver drowsiness detection system along with an alert system using deep learning techniques. The goal will be to develop a system that can accurately determine whether the... more

Background: The purpose of this review paper is to develop a driver drowsiness detection system along with an alert system using deep learning techniques. The goal will be to develop a system that can accurately determine whether the driver is sleepy or not. This project uses CNN-based detection of drivers' drowsiness. Objectives: The objectives of this project are to identify the driver's drowsiness, reduce the number of accidents caused by the driver's drowsiness and provide safety to the driver early and cost-effectively. Methods: An integrated approach depends on the eye and mouth closure status (PERCLOS) along with the calculation of the new proposed vector FAR (Facial Aspect Ratio), similarly to EAR and MAR. This helps to find the status of the closed eyes or opened mouth, like yawning, and any frame that has hand gestures like nodding or covering the opened mouth with a hand, as is the innate nature of humans when trying to control sleepiness. Statistical Analysis: The CNN Yolo model algorithm is used to achieve precise results and ensure safety by preventing accidents resulting from driver drowsiness. The system includes a drowsiness detection mechanism that promptly alerts the driver to mitigate the risk of road accidents caused by drowsiness, thereby preventing potential incidents. Findings: Existing systems that rely on specific facial features, such as the ear, nose, and mouth, for drowsiness detection have limitations in accuracy. Applications: We are developing an improved system that considers all facial features for more reliable predictions. Additionally, we have integrated an alarm to alert drivers if they are becoming too drowsy. Improvements: An automatic and efficient drowsiness detection and driver mood predictionbased system is required to be implemented for real-time applications. This will help to reduce road accidents and increase people's safety.

The subject of fatigue monitoring is becoming more important in transportation and traffic management (including, for instance, the development of systems to detect and prevent driver drowsiness). People who work in offices are also... more

The subject of fatigue monitoring is becoming more important in transportation and traffic management (including, for instance, the development of systems to detect and prevent driver drowsiness). People who work in offices are also susceptible to exhaustion, but there is currently no widely deployed system that is able to monitor this condition. In most cases, the driver’s eyelids will become heavy due to exhaustion after lengthy hours of driving or in the absence of mental concentration. Typically, when the driver’s concentration begins to fade, audio alert would be provided to force the drivers awake. In recent times, drowsiness is risky since it can result in an accident. Thus, a solution has been proposed to identify driver drowsiness by comparing several algorithms to find improved accuracy and execution time. Besides, this system will alert the driver with an audible warning in the event of drowsiness is detected.

Drowsy driving causes many accidents. Driver alertness and automobile control are challenged. Thus, a driver drowsiness detection system is becoming a necessity. In fact, invasive approaches that analyze electroencephalography signals... more

Drowsy driving causes many accidents. Driver alertness and automobile control are challenged. Thus, a driver drowsiness detection system is becoming a necessity. In fact, invasive approaches that analyze electroencephalography signals with head electrodes are inconvenient for drivers. Other non-invasive fatigue detection studies focus on yawning or eye blinks. The analysis of several facial components has yielded promising results, but it is not yet enough to predict hypovigilance. In this paper, we propose a “non-invasive” approach based on a deep learning model to classify vigilance into five states. The first step is using MediaPipe Face Mesh to identify the target areas. This step calculates the driver’s gaze and eye state descriptors and the 3D head position. The detection of the iris area of interest allows us to compute a normalized image to identify the state of the eyes relative to the eyelids. A transfer learning step by the MobileNetV3 model is performed on the normalized...

Educational Data Mining (EDM) research has taking an important place as it helps in exposing useful knowledge from educational data sets to be employed and serve several purposes such as predicting students’ achievements. Predicting... more

Educational Data Mining (EDM) research has taking an important place as it helps in exposing useful knowledge from educational data sets to be employed and serve several purposes such as predicting students’ achievements. Predicting student’s achievements might be useful for building and adopting several changes in the educational environments as a re-action in the current educational systems. Most of the existing research have used machine learning to predict students’ achievements by using diverse attributes such as family income, students gender, students absence and level etc. In this paper, the effort is made to explore the effectiveness of using the deep learning algorithm more precisely CNN to predict students’ achievements which could hlp in predicting if student will be able to finish their degree or not. The experimental results reveal how the proposed model outperformed the existing approaches in terms of prediction accuracy.

The main idea behind this project is to develop a system that can detect the drowsiness of the driver and issue a timely warning. Driver Fatigue is the main reason for a large number of road accidents. The detection can be done in many... more

The main idea behind this project is to develop a system that can detect the drowsiness of the driver and issue a timely
warning. Driver Fatigue is the main reason for a large number of road accidents. The detection can be done in many different
ways and by using different parameters.

Drowsy driving is one of the major causes of road accidents and death. Hence, detection of driver's fatigue and its indication is an active research area. Most of the conventional methods are either vehicle based, or behavioral based or... more

Drowsy driving is one of the major causes of road accidents and death. Hence, detection of driver's fatigue and its indication is an active research area. Most of the conventional methods are either vehicle based, or behavioral based or physiological based.

This paper presents a practical human-computer interaction system for wheelchair motion through eye tracking and eye blink detection. In this system, the pupil in the eye image has been extracted after binarization, and the center of the... more

This paper presents a practical human-computer interaction system for wheelchair motion through eye tracking and eye blink detection. In this system, the pupil in the eye image has been extracted after binarization, and the center of the pupil was localized to capture the trajectory of eye movement and determine the direction of eye gaze. Meanwhile, convolutional neural networks for feature extraction and classification of open-eye and closed-eye images have been built, and machine learning was performed by extracting features from multiple individual images of open-eye and closed-eye states for input to the system. As an application of this human-computer interaction control system, experimental validation was carried out on a modified wheelchair and the proposed method proved to be effective and reliable based on the experimental results.

Every year, hundreds of people die in traffic accidents all over the world. According to national statistics, human factors play a determining role in 90-95% of car accidents in Iran. Globally, 25% of accidents are caused by driver... more

Every year, hundreds of people die in traffic accidents all over the world. According to national statistics, human factors play a determining role in 90-95% of car accidents in Iran. Globally, 25% of accidents are caused by driver fatigue and around 60% of road accidents result in death or serious injury. In a National Transportation Research Institute (NTSRB) study of 107 randomly selected car crashes, fatigue accounted for 58% of all crashes. Drowsy driving is a major factor in serious road accidents that claim thousands of lives every year. The application of intelligent systems in automobiles has developed considerably in recent years. These systems use wireless sensor networks to monitor and transmit car and driver status. Smart cars that use software technology to control engine speed, steering, transmission, braking, etc. Management and control, the quality of management has been greatly improved. Ad hoc networks are the first systems to develop automatic navigation in cars. A notable weakness of these systems is that their response to changes in the environment is not in real time. This is especially important in driving, where time is a key factor in a driver's decision. On the other hand, another way to check driver fatigue is to monitor the physical condition and facial expression of the driver, which cannot be processed and transmitted accurately enough by wireless sensor networks. Driver fatigue is a major factor in a large number of road accidents. Recent statistics estimate that 1,200 deaths and 76,000 injuries are attributable to fatigue-related accidents each year.

This paper describes a software application that records student engagement in an on-screen task. The application records in real time the on-screen activity and simultaneously estimates the emotional state and head pose of the learner.... more

This paper describes a software application that records student engagement in an on-screen task. The application records in real time the on-screen activity and simultaneously estimates the emotional state and head pose of the learner. The head pose is used to detect when the screen is being viewed and the emotional state provides feedback on the form of engagement. The application works without recording images of the learner. On completing the task, the percentage of time spent viewing the screen and statistics on emotional state (neutral, happy, sad) are produced. A graph depicting the learner's engagement and emotional state synchronised with the screen captured video is also produced. It is envisaged that the tool will find application in learning activity and learning object design.

The fatigue-related accident is increasing due to long work hours, medical reasons, and age that decrease response time in a moment of hazard. One of drowsiness and fatigue visual indicators is excessive yawning. In this paper, a... more

The fatigue-related accident is increasing due to long work hours, medical reasons, and age that decrease response time in a moment of hazard. One of drowsiness and fatigue visual indicators is excessive yawning. In this paper, a non-optical sensor presented as a car dashcam that is used to record driving scenarios and imitates real-life driving situations such as being distracted or talking to a passenger next to the driver. We built a deep CNN model as the classifier to classify each frame as a yawning or non- yawning driver. We can classify the drivers' fatigue into three levels, alert, early fatigue and fatigue based on the judgement of the number of yawns. Alert level means when the driver is not yawning, while, early fatigue is when the driver yawns once in a minute. Fatigued is when the driver yawns more than once in a minute. An overall decision is made by analyzing the source score and the condition of the driver's fatigue state. The robustness of the proposed metho...

Fatigue is an important component for screening for "Fit for Duty" at work place. The main objective of this research paper is to identify a novel deep learning technique that can be used to screen fatigue in workplace setting. In order... more

Fatigue is an important component for screening for "Fit for Duty" at work place. The main objective of this research paper is to identify a novel deep learning technique that can be used to screen fatigue in workplace setting. In order to achieve personal and professional goals, enhance the structure of the organization and to sustain one's living conditions in an appropriate manner, it is necessary to take into consideration the aspects of health and safety of the employees. This research paper outlines our work on the importance of detecting health, safety and fatigue in the workplace with state-of-art technology before even starting a job. This paper proposes a real-time comprehensive employee fatigue detection algorithm based on different facial landmarks to improve the detection accuracy, which detects the employee's fatigue status by using facial video sequences without equipping them with sensor devices. The facial area is analyzed including detection of left and right eye along with the mouth region. In this paper we are proposing a novel deep learning technique to classify high, mid and low levels of fatigue. We are performing this activity at a safe entry station (SES) which also measures other vital parameters such as Body Temperature, Eye Redness, Heart Rate and Respiration Rate. The focus of the current study is on fatigue detection and our AI pipeline achieved 91% accuracy on data points collected at various sites in identification of fatigue levels.

The use of mobile devices can easily divert a driver's attention away from the road. Dangerous driving, such as texting and driving, can cause havoc in traffic and jeopardize safety. The goal of this project is to develop an... more

The use of mobile devices can easily divert a driver's attention away from the road. Dangerous driving, such as texting and driving, can cause havoc in traffic and jeopardize safety. The goal of this project is to develop an accident-avoidance system that can detect the presence of a mobile phone in the driver's hand by installing an in-car camera facing the driver and running a YOLOv3-Tiny algorithm for mobile phone detection. In addition, the model will issue an audio alert to the driver and use face detection algorithms to determine the driver's identity. Twilio APIs are being used to send live messages to car owners about the actions taken using the car's location information.

This study discusses the problem of oil and gas faults that lead to spills or explosions that lead to a lot of losses in human life, oil field extraction, and costs. Petrol is an important field in our lives because it controls all... more

This study discusses the problem of oil and gas faults that lead to spills or explosions that lead to a lot of losses in human life, oil field extraction, and costs. Petrol is an important field in our lives because it controls all aspects of human life and their way of life, so our research focused on petrol and its problems in order to introduce a better way of life. The data used in this research was taken from the 3w database that was prepared by Petrobras, the Brazilian oil holding. The 9 classes classified in that work include the normal state that indicates the factors that will not lead to a problem. Deep learning classification techniques were used in this study. 99% accuracy was obtained in that model, and it refers to a successful prediction and classification of each class. Different results were observed when different hidden layers, optimizers, neurons, epochs, and activation functions were used. 99% was achieved when using Adam's optimizer and Tanh's activation function.

During COVID-19, wearing a mask was globally mandated in various workplaces, departments, and offices. New deep learning convolutional neural network (CNN) based classifications were proposed to increase the validation accuracy of face... more

During COVID-19, wearing a mask was globally mandated in various workplaces, departments, and offices. New deep learning convolutional neural network (CNN) based classifications were proposed to increase the validation accuracy of face mask detection. This work introduces a face mask model that is able to recognize whether a person is wearing mask or not. The proposed model has two stages to detect and recognize the face mask; at the first stage, the Haar cascade detector is used to detect the face, while at the second stage, the proposed CNN model is used as a classification model that is built from scratch. The experiment was applied on masked faces (MAFA) dataset with images of 160x160 pixels size and RGB color. The model achieved lower computational complexity and number of layers, while being more reliable compared with other algorithms applied to recognize face masks. The findings reveal that the model's validation accuracy reaches 97.55% to 98.43% at different learning rates and different values of features vector in the dense layer, which represents a neural network layer that is connected deeply of the CNN proposed model training. Finally, the suggested model enhances recognition performance parameters such as precision, recall, and area under the curve (AUC).

The Library Linkage Project (PrEBi) has as main objective to share the bibliographic acervus resident in ISTEC Institutions, in order to satisfy the demand of Researchers, Teachers and Students. To manage the bibliography requirements... more

The Library Linkage Project (PrEBi) has as main objective to share the bibliographic acervus resident in ISTEC Institutions, in order to satisfy the demand of Researchers, Teachers and Students. To manage the bibliography requirements (journal articles, book chapters, congress proceedings, theses and patents) made by users, it's been developed the Celsius Software. Nowadays the main job of Celsius administrators is to lookup requests in ISTEC and non-ISTEC Institutions' catalogs by hand which takes many hours to check all until the resource is found. In order to perform this task more efficiently, it has been created in PrEBi the Celsius Bloodhound Project. This project, currently in analysis and design stage, intends to automatize this tedious job by searching and fetching records from library servers. Bloodhound's design is flexible enough to allow connections from almost any application and to connect to many different remote catalogues. Bloodhound will be able to int...

Drowsy driving is a major cause for car accidents.It has caused severe impact on the lives of people. Accidents can be avoided by alerting or warning drivers beforehand. Embedded systems and sensors that are equipped on the dashboard of... more

Several factors often contribute to car accidents, most of them caused by human error, and the most notable are drowsiness, fatigue, distracted driving, and alcohol. Although self-driving cars are the best solution to save human lives and... more

Several factors often contribute to car accidents, most of them caused by human error, and the most notable are drowsiness, fatigue, distracted driving, and alcohol. Although self-driving cars are the best solution to save human lives and avoid car accidents, they are expensive. The roads in many countries are not prepared for the movement of this type of car. Scare new technologies included in modern cars, such as backup cameras and sensors, contributed to keeping drivers safer in this paper. A driver monitoring system is based on determining the driver’s face’s main points, which provide the required vital information for face analysis. The EfficientNet convolutional neural network (ConvNet) model is used for facial landmarks prediction, which is employed to detect face drowsiness and fatigue in real-time. The system is trained to detect multiple traits, including facial expressions, yawning and head poses. The results show that employing facial landmarks will assist in efficientl...