Caroline Chanel - Academia.edu (original) (raw)

Papers by Caroline Chanel

2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

Brain-Computer Interfaces (BCI) provide a unique communication channel between the brain and comp... more Brain-Computer Interfaces (BCI) provide a unique communication channel between the brain and computer systems. After extensive research and implementation on ample fields of application, numerous challenges to assure reliable and quick data processing have resulted in the hybrid BCI (hBCI) paradigm, consisting on the combination of two BCI systems. However, not all challenges have been properly addressed (e.g. re-calibration, idle-state modelling, adaptive thresholds, etc) to allow hBCI implementation outside of the lab. In this paper, we review electroencephalography based hBCI studies and state potential limitations. We propose a sequential decision-making framework based on Partially Observable Markov Decision Process (POMDP) to design and to control hBCI systems. The POMDP framework is an excellent candidate to deal with the limitations raised above. To illustrate our opinion, an example of architecture using a POMDP-based hBCI control system is provided, and future directions are discussed. We believe this framework will encourage research efforts to provide relevant means to combine information from BCI systems and push BCI out of the laboratory.

Le Centre pour la Communication Scientifique Directe - HAL - Diderot, Jun 28, 2021

Le cerveau humain, pour allouer de manière optimale les ressources attentionnelles limitées dont ... more Le cerveau humain, pour allouer de manière optimale les ressources attentionnelles limitées dont il dispose, supprime ou renforce l'activation de circuits neuronaux : il implémente des heuristiques. Dans une approche novatrice, nous proposons d'utiliser l'apprentissage par renforcement inverse pour caractériser la dynamique d'activation de ces réseaux. Un protocole expérimental est proposé, et les données collectées devraient permettre, à terme, de vérifier cette démarche.

Proceedings of the 14th International Conference on Agents and Artificial Intelligence, 2022

Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task... more Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are endowed with invariant reward and transition functions with respect to some transformations of the current state and action. Being able to detect and exploit these structures could benefit not only the learning of the MDP but also the computation of its subsequent optimal control policy. In this work we propose a paradigm, based on Density Estimation methods, that aims to detect the presence of some already supposed transformations of the state-action space for which the MDP dynamics is invariant. We tested the proposed approach in a discrete toroidal grid environment and in two notorious environments of OpenAI's Gym Learning Suite. The results demonstrate that the model distributional shift is reduced when the dataset is augmented with the data obtained by using the detected symmetries, allowing for a more thorough and data-efficient learning of the transition functions.

Eye tracking has become a valuable tool to observe and interpret operators’ mental state. This ma... more Eye tracking has become a valuable tool to observe and interpret operators’ mental state. This may be due to the fact that it is less intrusive than other physiological tools, and also capable of assessing various mental states such as mental fatigue and engagement (e.g. using blink frequency and fixation duration). These states are critical in aeronautical contexts. Therefore, a real-time eye-tracking processing system was developed using eye-tracking glasses worn by the pilot while performing flight and interacting with unmanned aerial vehicles (UAVs). The proposed system is capable of finding areas of interest from different positions, orientations, and distances from a continuously changing field of view. Four computer screens were tracked and in one of them three different areas of interest were defined and tracked in real-time. The goal is to ease the pilot’s mental state assessment in ecological settings and ultimately to enhance the pilot-UAVs interaction in Manned-Unmanned ...

HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific re... more HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

2016 XIII Latin American Robotics Symposium and IV Brazilian Robotics Symposium (LARS/SBR), 2016

In the present document, the authors introduce the Cooperative Exploration problem as well as the... more In the present document, the authors introduce the Cooperative Exploration problem as well as the most relevant approaches in order to show the most common drawbacks and opportunities to improve the state of art solutions. Subsequently, a preliminary version of a multi-robot exploration proposal is described. The first results obtained in simulated scenarios support the underlying ideas are feasible and promising. They show that is possible to cope with real communication constraints (always present in practice), being more fault tolerant and still having good performance regarding the total exploration time. Next steps to fully implement a more reliable and robust system are discussed.

PAIS 2022

This paper addresses a safe path planning problem for UAV urban navigation, under uncertain GNSS ... more This paper addresses a safe path planning problem for UAV urban navigation, under uncertain GNSS availability. The problem can be modeled as a POMDP and solved with sampling-based algorithms. However, such a complex domain suffers from high computational cost and achieves poor results under real-time constraints. Recent research seeks to integrate offline learning in order to efficiently guide online planning. Inspired by the state-of-the-art CAMP (Context-specific Markov decision Process) formalization, this paper proposes an offline process which learns the path constraint to impose during online POMDP solving in order to reduce the policy search space. More precisely, the offline learnt constraint selector returns the best path constraint according to the GNSS availability probability in the environment. Conclusions of experiments, carried out for three environments, show that using the proposed approach allows to improve the quality of a solution reached by an online planner, wi...

HHAI2022: Augmenting Human Intellect

In this study, a formal framework aiming to drive the interaction between a human operator and a ... more In this study, a formal framework aiming to drive the interaction between a human operator and a team of unmanned aerial vehicles (UAVs) is experimentally tested. The goal is to enhance human performance by controlling the interaction between agents based on an online monitoring of the operator’s mental workload (MW) and performance. The proposed solution uses MW estimation via a classifier applied to cardiac features. The classifier output is introduced as a human MW state observation in a Partially Observable Markov Decision Process (POMDP) which models the human-system interaction dynamics, and aims to control the interaction to optimize the human agent’s performance. Based on the current belief state about the operator’s MW and performance, along with the mission phase, the POMDP policy solution controls which task should be suggested -or not- to the operator, assuming the UAVs are capable of supporting the human agent. The framework was evaluated using an experiment in which 13...

OATAO is an open access repository that collects the work of Toulouse researchers and makes it fr... more OATAO is an open access repository that collects the work of Toulouse researchers and makes it freely available over the web where possible. This is an author-deposited version published in: http://oatao.univ-toulouse.fr/ Eprints ID: 16598 Abstract. In order to enhance the behavior of autonomous service robots, we are exploring multiple paradigms for their planning and execution strategy (the way of in-terleaving the planning, selection and execution of actions). In this paper we focus on continuous proactive planning with multiple hypotheses in order to continuously generate multiple solution-plans from which an action can be selected when appropriate. To illustrate the concepts, we develop how it could be used for autonomous navigation in dynamic environments, and analyze the tests we realized with several instantiations. We also discuss several aspects and concerns about the proposed strategy, and how integrating more semantic information could enhance the capabilities of service...

Path planning is a research domain very active and applied among others on autonomous vehicle suc... more Path planning is a research domain very active and applied among others on autonomous vehicle such as UAV. In recent years, a lot of progress has been made on path planning under uncertainties issued by a vehicle navigation system, for example in localization or environment mapping. However, such uncertainties are often treated by the path planner in a deterministic way. That is, the navigation system's performance is deterministically given in function of the environment. This paper tackles a more complex problem of UAV safe path planning in an urbain environment, in which UAV is at risks of GPS signal occusion and obstacle collision. The key idea is to make a UAV path planning along with its navigation and guidance mode planning, where each of such mode uses different set of sensors and whose availability and performance are environment-dependent. A partial knowledge on the environment is supposed to be available, in a form of probability maps of obstacles and sensor availabil...

Unmanned Aerial Vehicles (UAVs) can nowadays, in certain conditions, be employed for different ap... more Unmanned Aerial Vehicles (UAVs) can nowadays, in certain conditions, be employed for different applications ranging from service robotics to surveillance applications in network monitoring or in search and rescue missions. For this aim, and for widening the UAV application field, it is mandatory for an UAV to have some capabilities for autonomous safe navigation in cluttered environments. This navigation capability includes environment mapping, localization and guidance functionalities relative to the environment. Especially, one can find intensive research work proposing different UAV relative localization and guidance solutions based on vision: visual odometry, visual SLAM (Simultaneous Localization and Mapping), visual servoing, etc. Such solutions can be embedded in the UAV onboard flight system as an alternative navigation function to the nominal ones (in most cases, the GPS/INS localization with the waypoint navigation). However, the decision of switching the navigation and gu...

ArXiv, 2021

Offline model learning for planning is a branch of machine learning that trains agents to perform... more Offline model learning for planning is a branch of machine learning that trains agents to perform actions in an unknown environment using a fixed batch of previously collected experiences. The limited size of the data set hinders the estimate of the Value function of the relative Markov Decision Process (MDP), bounding the performance of the obtained policy in the real world. In this context, recent works showed that planning with a discount factor lower than the one used during the evaluation phase yields more performing policies. However, the optimal discount factor is finally chosen by cross-validation. Our aim is to show that looking for a sub-optimal solution of a Bayesian MDP might lead to better performances with respect to the current baselines that work in the offline setting. Hence, we propose Exploitation vs Caution (EvC), an algorithm which automatically selects the policy that solves a Risk-sensitive Bayesian MDP in a set of policies obtained by solving several MDPs cha...

Aircraft equipment health monitoring system plays a promising role for airlines operation cost re... more Aircraft equipment health monitoring system plays a promising role for airlines operation cost reduction, as it can be exploited to perform predictive maintenance. In this vein, a hierarchical sequential decision making model is proposed to plan predictive maintenance. It combines linear optimization for routing assignment and MDP planning to handle maintenance actions based on the stochastic evolution of health indicators. This entangled model should reduce planning time while ensuring a cost-efficient policy.

This paper tackles high-level decision-making techniques for robotic missions, which involve both... more This paper tackles high-level decision-making techniques for robotic missions, which involve both active sensing and symbolic goal reaching, under uncertain probabilistic environments and strong time constraints. Our case study is a POMDP model of an online multi-target detection and recognition mission by an autonomous UAV. The POMDP model of the multi-target detection and recognition problem is generated online from a list of areas of interest, which are automatically extracted at the beginning of the flight from a coarse-grained high altitude observation of the scene. The POMDP observation model relies on a statistical abstraction of an image processing algorithm's output used to detect targets. As the POMDP problem cannot be known and thus optimized before the beginning of the flight, our main contribution is an "optimize-while-execute" algorithmic framework: it drives a POMDP sub-planner to optimize and execute the POMDP policy in parallel under action duration co...

This paper tackles a problem of UAV safe path planning in an urban environment in which UAV is at... more This paper tackles a problem of UAV safe path planning in an urban environment in which UAV is at risks of GPS signal occlusion and obstacle collision. The key idea is to perform the UAV path planning along with its navigation and guidance mode planning, where each of these modes uses different sensors whose availability and performance are environment-dependent. A partial knowledge on the environment is supposed to be available in the form of probability maps of obstacles and sensor availabilities. This paper proposes a planner model based on Mixed Observability Markov Decision Process (MOMDP). It allows the planner to propagate such probability map information to the future path for choosing the best action. This paper provides a MOMDP model for the planner with an approximation of the belief states by Mixture of Gaussian functions.

This paper tackles a problem of UAV safe path planning in an urban environment where the onboard ... more This paper tackles a problem of UAV safe path planning in an urban environment where the onboard sensors can be unavailable such as GPS occlusion. The key idea is to perform UAV path planning along with its navigation an guidance mode planning where each of these modes uses different set of sensors and whose availability and performance are environment-dependent. It is supposed to have a-priori knowledge in a form of gaussians mixture maps of obstacles and sensors availabilities. These maps allow the use of an Extended Kalman Filter (EKF) to have an accurate state estimate. This paper proposes a planner model based on Mixed Observability Markov Decision Process (MOMDP) and EKF. It allows the planner to propagate such probability map information to the future path for choosing the best action minimizing the expected cost.

Due to the high complexity of probabilistic planning algorithms, roboticists often opt for determ... more Due to the high complexity of probabilistic planning algorithms, roboticists often opt for deterministic replanning paradigms, which can quickly adapt the current plan to the environment's changes. However, probabilistic planning suffers in practice from the common misconception that it is needed to generate complete or closed policies, which would not require to be adapted on-line. In this work, we propose an intermediate approach, which generates incomplete partial policies taking into account mid-term probabilistic uncertainties, continually improving them on a gliding horizon or regenerating them when they fail. Our algorithm is a configurable anytime meta-planner that drives any sub-(PO)MDP standard planner, dealing with all pending and time-bounded planning requests sent by the execution framework from many reachable possible future execution states, in anticipation of the probabilistic evolution of the system. We assess our approach on generic robotic problems and on comb...

The recent progress in robotics and artificial intelligence raises the question of the efficient ... more The recent progress in robotics and artificial intelligence raises the question of the efficient artificial agents interaction with humans. For instance, artificial intelligence has achieved technical advances in perception and decision making in several domains ranging from games to a variety of operational situations, (e.g. face recognition [51] and firefighting missions [23]). Such advanced automated systems still depend on human operators as far as complex tactical, legal or ethical decisions are concerned. Usually the human is considered as an ideal agent, that is able to take control in case of automated (artificial) agent’s limit range of action or even failure (e.g embedded sensor failures or low confidence in identification tasks). However, this approach needs to be revised as revealed by several critical industrial events (e.g. aviation and nuclear powerplant) that were due to conflicts between humans and complex automated system [13]. In this context, this paper reviews s...

En planification classique, une precondition sur une action est une formule booleenne, qui verifi... more En planification classique, une precondition sur une action est une formule booleenne, qui verifie si une action est realisable pour un etat donne. Cet element crucial pour des applications realistes, ou par exemple des actions considerees dangereuses doivent etre eliminees, n'a pas ete formellement modelise pour les POMDPs a notre connaissance. Une raison est que les preconditions sont definies sur des etats, i.e. le domaine d'application de l'action, alors que les decisions prises dans un POMDP sont definies sur l'etat de croyance courant de l'agent. Definir simplement des preconditions sur des etats de croyance n'est pas suffisant, puisque chaque etat de croyance peut-etre defini sur plusieurs etats, et il n'y a pas de garantie d'eviter que l'agent applique une action infaisable. Augmenter l'espace d'observations avec des actions realisables n'est pas non plus satisfaisant, d'abord parce que l'information sur les actions app...

Missions involving humans with automated systems become increasingly common and are subject to ri... more Missions involving humans with automated systems become increasingly common and are subject to risk of failing due to human factors. In fact, missions workload may generate stress or mental fatigue increasing the accident risk. The idea of our project is to refine human-robot supervision by using data from physiological sensors(eye tracking and heart rate monitoring devices) giving information about the operator's state. The proof of concept mission consists of a ground robot, autonomous or controlled by a human operator, which has to fight fires that catch randomly. We proposed to use the planning framework called Partially Observable Markov Decision Process (POMDP) along with machine learning techniques to improve human-machine interactions by optimizing the decision of the mode (autonomous or controlled robot) and of the display of alarms in the form of visual stimuli.A dataset of demonstrations produced by remote volunteers through an online video game simulating the mission...

2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

Brain-Computer Interfaces (BCI) provide a unique communication channel between the brain and comp... more Brain-Computer Interfaces (BCI) provide a unique communication channel between the brain and computer systems. After extensive research and implementation on ample fields of application, numerous challenges to assure reliable and quick data processing have resulted in the hybrid BCI (hBCI) paradigm, consisting on the combination of two BCI systems. However, not all challenges have been properly addressed (e.g. re-calibration, idle-state modelling, adaptive thresholds, etc) to allow hBCI implementation outside of the lab. In this paper, we review electroencephalography based hBCI studies and state potential limitations. We propose a sequential decision-making framework based on Partially Observable Markov Decision Process (POMDP) to design and to control hBCI systems. The POMDP framework is an excellent candidate to deal with the limitations raised above. To illustrate our opinion, an example of architecture using a POMDP-based hBCI control system is provided, and future directions are discussed. We believe this framework will encourage research efforts to provide relevant means to combine information from BCI systems and push BCI out of the laboratory.

Le Centre pour la Communication Scientifique Directe - HAL - Diderot, Jun 28, 2021

Le cerveau humain, pour allouer de manière optimale les ressources attentionnelles limitées dont ... more Le cerveau humain, pour allouer de manière optimale les ressources attentionnelles limitées dont il dispose, supprime ou renforce l'activation de circuits neuronaux : il implémente des heuristiques. Dans une approche novatrice, nous proposons d'utiliser l'apprentissage par renforcement inverse pour caractériser la dynamique d'activation de ces réseaux. Un protocole expérimental est proposé, et les données collectées devraient permettre, à terme, de vérifier cette démarche.

Proceedings of the 14th International Conference on Agents and Artificial Intelligence, 2022

Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task... more Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are endowed with invariant reward and transition functions with respect to some transformations of the current state and action. Being able to detect and exploit these structures could benefit not only the learning of the MDP but also the computation of its subsequent optimal control policy. In this work we propose a paradigm, based on Density Estimation methods, that aims to detect the presence of some already supposed transformations of the state-action space for which the MDP dynamics is invariant. We tested the proposed approach in a discrete toroidal grid environment and in two notorious environments of OpenAI's Gym Learning Suite. The results demonstrate that the model distributional shift is reduced when the dataset is augmented with the data obtained by using the detected symmetries, allowing for a more thorough and data-efficient learning of the transition functions.

Eye tracking has become a valuable tool to observe and interpret operators’ mental state. This ma... more Eye tracking has become a valuable tool to observe and interpret operators’ mental state. This may be due to the fact that it is less intrusive than other physiological tools, and also capable of assessing various mental states such as mental fatigue and engagement (e.g. using blink frequency and fixation duration). These states are critical in aeronautical contexts. Therefore, a real-time eye-tracking processing system was developed using eye-tracking glasses worn by the pilot while performing flight and interacting with unmanned aerial vehicles (UAVs). The proposed system is capable of finding areas of interest from different positions, orientations, and distances from a continuously changing field of view. Four computer screens were tracked and in one of them three different areas of interest were defined and tracked in real-time. The goal is to ease the pilot’s mental state assessment in ecological settings and ultimately to enhance the pilot-UAVs interaction in Manned-Unmanned ...

HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific re... more HAL is a multidisciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

2016 XIII Latin American Robotics Symposium and IV Brazilian Robotics Symposium (LARS/SBR), 2016

In the present document, the authors introduce the Cooperative Exploration problem as well as the... more In the present document, the authors introduce the Cooperative Exploration problem as well as the most relevant approaches in order to show the most common drawbacks and opportunities to improve the state of art solutions. Subsequently, a preliminary version of a multi-robot exploration proposal is described. The first results obtained in simulated scenarios support the underlying ideas are feasible and promising. They show that is possible to cope with real communication constraints (always present in practice), being more fault tolerant and still having good performance regarding the total exploration time. Next steps to fully implement a more reliable and robust system are discussed.

PAIS 2022

This paper addresses a safe path planning problem for UAV urban navigation, under uncertain GNSS ... more This paper addresses a safe path planning problem for UAV urban navigation, under uncertain GNSS availability. The problem can be modeled as a POMDP and solved with sampling-based algorithms. However, such a complex domain suffers from high computational cost and achieves poor results under real-time constraints. Recent research seeks to integrate offline learning in order to efficiently guide online planning. Inspired by the state-of-the-art CAMP (Context-specific Markov decision Process) formalization, this paper proposes an offline process which learns the path constraint to impose during online POMDP solving in order to reduce the policy search space. More precisely, the offline learnt constraint selector returns the best path constraint according to the GNSS availability probability in the environment. Conclusions of experiments, carried out for three environments, show that using the proposed approach allows to improve the quality of a solution reached by an online planner, wi...

HHAI2022: Augmenting Human Intellect

In this study, a formal framework aiming to drive the interaction between a human operator and a ... more In this study, a formal framework aiming to drive the interaction between a human operator and a team of unmanned aerial vehicles (UAVs) is experimentally tested. The goal is to enhance human performance by controlling the interaction between agents based on an online monitoring of the operator’s mental workload (MW) and performance. The proposed solution uses MW estimation via a classifier applied to cardiac features. The classifier output is introduced as a human MW state observation in a Partially Observable Markov Decision Process (POMDP) which models the human-system interaction dynamics, and aims to control the interaction to optimize the human agent’s performance. Based on the current belief state about the operator’s MW and performance, along with the mission phase, the POMDP policy solution controls which task should be suggested -or not- to the operator, assuming the UAVs are capable of supporting the human agent. The framework was evaluated using an experiment in which 13...

OATAO is an open access repository that collects the work of Toulouse researchers and makes it fr... more OATAO is an open access repository that collects the work of Toulouse researchers and makes it freely available over the web where possible. This is an author-deposited version published in: http://oatao.univ-toulouse.fr/ Eprints ID: 16598 Abstract. In order to enhance the behavior of autonomous service robots, we are exploring multiple paradigms for their planning and execution strategy (the way of in-terleaving the planning, selection and execution of actions). In this paper we focus on continuous proactive planning with multiple hypotheses in order to continuously generate multiple solution-plans from which an action can be selected when appropriate. To illustrate the concepts, we develop how it could be used for autonomous navigation in dynamic environments, and analyze the tests we realized with several instantiations. We also discuss several aspects and concerns about the proposed strategy, and how integrating more semantic information could enhance the capabilities of service...

Path planning is a research domain very active and applied among others on autonomous vehicle suc... more Path planning is a research domain very active and applied among others on autonomous vehicle such as UAV. In recent years, a lot of progress has been made on path planning under uncertainties issued by a vehicle navigation system, for example in localization or environment mapping. However, such uncertainties are often treated by the path planner in a deterministic way. That is, the navigation system's performance is deterministically given in function of the environment. This paper tackles a more complex problem of UAV safe path planning in an urbain environment, in which UAV is at risks of GPS signal occusion and obstacle collision. The key idea is to make a UAV path planning along with its navigation and guidance mode planning, where each of such mode uses different set of sensors and whose availability and performance are environment-dependent. A partial knowledge on the environment is supposed to be available, in a form of probability maps of obstacles and sensor availabil...

Unmanned Aerial Vehicles (UAVs) can nowadays, in certain conditions, be employed for different ap... more Unmanned Aerial Vehicles (UAVs) can nowadays, in certain conditions, be employed for different applications ranging from service robotics to surveillance applications in network monitoring or in search and rescue missions. For this aim, and for widening the UAV application field, it is mandatory for an UAV to have some capabilities for autonomous safe navigation in cluttered environments. This navigation capability includes environment mapping, localization and guidance functionalities relative to the environment. Especially, one can find intensive research work proposing different UAV relative localization and guidance solutions based on vision: visual odometry, visual SLAM (Simultaneous Localization and Mapping), visual servoing, etc. Such solutions can be embedded in the UAV onboard flight system as an alternative navigation function to the nominal ones (in most cases, the GPS/INS localization with the waypoint navigation). However, the decision of switching the navigation and gu...

ArXiv, 2021

Offline model learning for planning is a branch of machine learning that trains agents to perform... more Offline model learning for planning is a branch of machine learning that trains agents to perform actions in an unknown environment using a fixed batch of previously collected experiences. The limited size of the data set hinders the estimate of the Value function of the relative Markov Decision Process (MDP), bounding the performance of the obtained policy in the real world. In this context, recent works showed that planning with a discount factor lower than the one used during the evaluation phase yields more performing policies. However, the optimal discount factor is finally chosen by cross-validation. Our aim is to show that looking for a sub-optimal solution of a Bayesian MDP might lead to better performances with respect to the current baselines that work in the offline setting. Hence, we propose Exploitation vs Caution (EvC), an algorithm which automatically selects the policy that solves a Risk-sensitive Bayesian MDP in a set of policies obtained by solving several MDPs cha...

Aircraft equipment health monitoring system plays a promising role for airlines operation cost re... more Aircraft equipment health monitoring system plays a promising role for airlines operation cost reduction, as it can be exploited to perform predictive maintenance. In this vein, a hierarchical sequential decision making model is proposed to plan predictive maintenance. It combines linear optimization for routing assignment and MDP planning to handle maintenance actions based on the stochastic evolution of health indicators. This entangled model should reduce planning time while ensuring a cost-efficient policy.

This paper tackles high-level decision-making techniques for robotic missions, which involve both... more This paper tackles high-level decision-making techniques for robotic missions, which involve both active sensing and symbolic goal reaching, under uncertain probabilistic environments and strong time constraints. Our case study is a POMDP model of an online multi-target detection and recognition mission by an autonomous UAV. The POMDP model of the multi-target detection and recognition problem is generated online from a list of areas of interest, which are automatically extracted at the beginning of the flight from a coarse-grained high altitude observation of the scene. The POMDP observation model relies on a statistical abstraction of an image processing algorithm's output used to detect targets. As the POMDP problem cannot be known and thus optimized before the beginning of the flight, our main contribution is an "optimize-while-execute" algorithmic framework: it drives a POMDP sub-planner to optimize and execute the POMDP policy in parallel under action duration co...

This paper tackles a problem of UAV safe path planning in an urban environment in which UAV is at... more This paper tackles a problem of UAV safe path planning in an urban environment in which UAV is at risks of GPS signal occlusion and obstacle collision. The key idea is to perform the UAV path planning along with its navigation and guidance mode planning, where each of these modes uses different sensors whose availability and performance are environment-dependent. A partial knowledge on the environment is supposed to be available in the form of probability maps of obstacles and sensor availabilities. This paper proposes a planner model based on Mixed Observability Markov Decision Process (MOMDP). It allows the planner to propagate such probability map information to the future path for choosing the best action. This paper provides a MOMDP model for the planner with an approximation of the belief states by Mixture of Gaussian functions.

This paper tackles a problem of UAV safe path planning in an urban environment where the onboard ... more This paper tackles a problem of UAV safe path planning in an urban environment where the onboard sensors can be unavailable such as GPS occlusion. The key idea is to perform UAV path planning along with its navigation an guidance mode planning where each of these modes uses different set of sensors and whose availability and performance are environment-dependent. It is supposed to have a-priori knowledge in a form of gaussians mixture maps of obstacles and sensors availabilities. These maps allow the use of an Extended Kalman Filter (EKF) to have an accurate state estimate. This paper proposes a planner model based on Mixed Observability Markov Decision Process (MOMDP) and EKF. It allows the planner to propagate such probability map information to the future path for choosing the best action minimizing the expected cost.

Due to the high complexity of probabilistic planning algorithms, roboticists often opt for determ... more Due to the high complexity of probabilistic planning algorithms, roboticists often opt for deterministic replanning paradigms, which can quickly adapt the current plan to the environment's changes. However, probabilistic planning suffers in practice from the common misconception that it is needed to generate complete or closed policies, which would not require to be adapted on-line. In this work, we propose an intermediate approach, which generates incomplete partial policies taking into account mid-term probabilistic uncertainties, continually improving them on a gliding horizon or regenerating them when they fail. Our algorithm is a configurable anytime meta-planner that drives any sub-(PO)MDP standard planner, dealing with all pending and time-bounded planning requests sent by the execution framework from many reachable possible future execution states, in anticipation of the probabilistic evolution of the system. We assess our approach on generic robotic problems and on comb...

The recent progress in robotics and artificial intelligence raises the question of the efficient ... more The recent progress in robotics and artificial intelligence raises the question of the efficient artificial agents interaction with humans. For instance, artificial intelligence has achieved technical advances in perception and decision making in several domains ranging from games to a variety of operational situations, (e.g. face recognition [51] and firefighting missions [23]). Such advanced automated systems still depend on human operators as far as complex tactical, legal or ethical decisions are concerned. Usually the human is considered as an ideal agent, that is able to take control in case of automated (artificial) agent’s limit range of action or even failure (e.g embedded sensor failures or low confidence in identification tasks). However, this approach needs to be revised as revealed by several critical industrial events (e.g. aviation and nuclear powerplant) that were due to conflicts between humans and complex automated system [13]. In this context, this paper reviews s...

En planification classique, une precondition sur une action est une formule booleenne, qui verifi... more En planification classique, une precondition sur une action est une formule booleenne, qui verifie si une action est realisable pour un etat donne. Cet element crucial pour des applications realistes, ou par exemple des actions considerees dangereuses doivent etre eliminees, n'a pas ete formellement modelise pour les POMDPs a notre connaissance. Une raison est que les preconditions sont definies sur des etats, i.e. le domaine d'application de l'action, alors que les decisions prises dans un POMDP sont definies sur l'etat de croyance courant de l'agent. Definir simplement des preconditions sur des etats de croyance n'est pas suffisant, puisque chaque etat de croyance peut-etre defini sur plusieurs etats, et il n'y a pas de garantie d'eviter que l'agent applique une action infaisable. Augmenter l'espace d'observations avec des actions realisables n'est pas non plus satisfaisant, d'abord parce que l'information sur les actions app...

Missions involving humans with automated systems become increasingly common and are subject to ri... more Missions involving humans with automated systems become increasingly common and are subject to risk of failing due to human factors. In fact, missions workload may generate stress or mental fatigue increasing the accident risk. The idea of our project is to refine human-robot supervision by using data from physiological sensors(eye tracking and heart rate monitoring devices) giving information about the operator's state. The proof of concept mission consists of a ground robot, autonomous or controlled by a human operator, which has to fight fires that catch randomly. We proposed to use the planning framework called Partially Observable Markov Decision Process (POMDP) along with machine learning techniques to improve human-machine interactions by optimizing the decision of the mode (autonomous or controlled robot) and of the display of alarms in the form of visual stimuli.A dataset of demonstrations produced by remote volunteers through an online video game simulating the mission...