Classifying User Environment for Mobile Applications using Linear Autoencoding of Ambient Audio (original) (raw)
Related papers
Acoustic environment classification
ACM Transactions on Speech and Language Processing, 2006
The acoustic environment provides a rich source of information on the types of activity, communication modes, and people involved in many situations. It can be accurately classified using recordings from microphones commonly found in PDAs and other consumer devices. We describe a prototype HMM-based acoustic environment classifier incorporating an adaptive learning mechanism and a hierarchical classification model. Experimental results show that we can accurately classify a wide variety of everyday environments. We also show good results classifying single sounds, although classification accuracy is influenced by the granularity of the classification.
Electronics
The identification of Activities of Daily Living (ADL) is intrinsic with the user’s environment recognition. This detection can be executed through standard sensors present in every-day mobile devices. On the one hand, the main proposal is to recognize users’ environment and standing activities. On the other hand, these features are included in a framework for the ADL and environment identification. Therefore, this paper is divided into two parts—firstly, acoustic sensors are used for the collection of data towards the recognition of the environment and, secondly, the information of the environment recognized is fused with the information gathered by motion and magnetic sensors. The environment and ADL recognition are performed by pattern recognition techniques that aim for the development of a system, including data collection, processing, fusion and classification procedures. These classification techniques include distinctive types of Artificial Neural Networks (ANN), analyzing v...
A COMPARISON OF TECHNIQUES FOR AUDIO ENVIRONMENT CLASSIFICATION
Excessive background noise is one of the most common complaints from hearing aid users. Background noise classification systems can be used in hearing aids to adjust the response based on the noise environment. This paper examines and compares several classification techniques in the form of the k-nearest neighbours (K-NN)classifier, the non-windowed artificial neural network (ANN) and the hidden Markov model (HMM), to an artificial neural network using windowed input (WANN). Results obtained indicate that the WANN gives an accuracy of up to 97.9%, which is the highest accuracy of the tested classifiers. The memory and computational requirements of the windowed ANN are also small compared to the HMM and K-NN. Overall, the WANN is able to give excellent accuracy and reliability and is considered to be a good choice for background noise classification in hearing aids.
An unsupervised noise classification smartphone app for hearing improvement devices
2017 IEEE Signal Processing in Medicine and Biology Symposium (SPMB), 2017
This paper presents an app for running a previously developed unsupervised noise classifier in realtime on smartphone/tablet platforms. The steps taken to enable the development of this app are discussed. The app is utilized to carry out field testing of the unsupervised classification of actual encountered noise environments without any prior training and without specifying the number of noise classes or clusters. Two objective measures of cluster purity and normalized mutual information are considered to examine the performance of the app in the field with the user acting as the identifier of the ground truth classes. The results obtained indicate the effectiveness of this real-time smartphone app for carrying out the environmental noise classification in an unsupervised manner.
DeepEar: Robust Smartphone Audio Sensing in Unconstrained Acoustic Environments using Deep Learning
Microphones are remarkably powerful sensors of human behavior and context. However, audio sensing is highly susceptible to wild fluctuations in accuracy when used in diverse acoustic environments (such as, bedrooms, vehicles, or cafes), that users encounter on a daily basis. Towards addressing this challenge, we turn to the field of deep learning; an area of machine learning that has radically changed related audio modeling domains like speech recognition. In this paper, we present DeepEar -the first mobile audio sensing framework built from coupled Deep Neural Networks (DNNs) that simultaneously perform common audio sensing tasks. We train DeepEar with a large-scale dataset including unlabeled data from 168 place visits. The resulting learned model, involving 2.3M parameters, enables DeepEar to significantly increase inference robustness to background noise beyond conventional approaches present in mobile devices. Finally, we show DeepEar is feasible for smartphones by building a cloud-free DSP-based prototype that runs continuously, using only 6% of the smartphone's battery daily.
Mobile User Environment Detection using Deep Learning based Multi-Output Classification
2019 12th IFIP Wireless and Mobile Networking Conference (WMNC), 2019
Future mobile networks can hugely benefit from cognition of mobile user behavior. Indeed, knowing what/when/where/how the user consumes their mobile services can notably improve the self-adaptation and self-optimization capabilities of these networks and, in turn, ensure user satisfaction. The cognition of mobile user behavior will thus help 5G networks to face the variable consuming habits of users which in turn impact the network conditions, by predicting them in advance. In this paper, we focus on the "where" part, i.e., the detection of the environment where a given user consumes different mobile applications. A statistical study on the real activity of users reveals that there are multiple various environment types corresponding to the mobile phone usage. A Deep Learning based model is introduced to intelligently detect the user environment, using supervised and semi-supervised multi-output classification. Relevant multi-class schemes are proposed to efficiently regro...
An environmental audio-based context recognition system using smartphones
2014
Environmental sound/audio is a rich source of information that can be used to infer a person's context in daily life. Almost every activity produces some sound patterns, e.g., speaking, walking, washing, or typing on computer. Most locations have usually a specific sound pattern too, e.g., restaurants, offices or streets. This thesis addresses the design and development of an application for real-time detection and recognition of user activities using audio signals on mobile phones. The audio recognition application increases the capability, intelligence and feature of the mobile phones and, thus, First of all, I would like to thank Almighty God, who has blessed and guided me so that I am able to accomplish this thesis. In this very special occasion, I would like to express my deepest gratitude and appreciation to my Supervisor, Dr. Nirvana Meratnia, who gave her valuable time, guidance, advice, criticism and corrections to the thesis from the beginning till the end. She was always available for my questions and she was positive and gave generously her time and vast knowledge. I also want to thank all of the lecturers and professors of the Faculty who have thought and guided me during the years of my study at the University. In addition, I would like to thank the University of Twente Scholarship for providing me with financial help and funding, without which it would not have been possible to successfully finish my study.
Neural networks for recognizing human activities in home-like environments
Integrated Computer-Aided Engineering, 2018
Using Context-Awareness information in human-robot shared environments enhances the characterization of the interaction scenario, improving the experience of human collaboration with the robot. Aspects such as environmental location and human-robot dialog have traditionally been used to infer the current situation, which can be helpful in the decision making process of any robot. However, the acoustic signals available in every interaction scenario are commonly obviated, thus removing an important source of information. This paper presents the design, development and tests of a Context Awareness Component that labels users' activities using localization information, dialog flow, and time of day, and adds an environment recognition component supported by acoustic signals that improves the inference system. To this end, this research proposes a feedforward neural network solution based on a multilayer perceptron approach. The paper also discusses the configuration of the neural network for optimizing the recognition of human activity in "at home" environments, using the four inputs previously mentioned. Finally, the validation of the approach proposed is done by comparing the results when the sound recognition system is used and when it is not.
Robust Audio Sensing with Multi-Sound Classification
2019 IEEE International Conference on Pervasive Computing and Communications (PerCom, 2019
Audio data is a highly rich form of information, often containing patterns with unique acoustic signatures. In pervasive sensing environments, because of the empowered smart devices, we have witnessed an increasing research interest in sound sensing to detect ambient environment, recognise users' daily activities, and infer their health conditions. However, the main challenge is that the real-world environment often contains multiple sound sources, which can significantly compromise the robustness of the above environment, event, and activity detection applications. In this paper, we explore different approaches in multi-sound classification, and propose a stacked classifier based on the recent advance in deep learning. We evaluate our proposed approach in a comprehensive set of experiments on both sound effect and real-world datasets. The results have demonstrated that our approach can robustly identify each sound category among mixed acoustic signals, without the need of any a priori knowledge about the number and signature of sounds in the mixed signals.