Abdelhakim Benechehab - Academia.edu (original) (raw)

Abdelhakim Benechehab

Uploads

Papers by Abdelhakim Benechehab

Research paper thumbnail of A Multi-step Loss Function for Robust Learning of the Dynamics in Model-based Reinforcement Learning

arXiv (Cornell University), Feb 5, 2024

Research paper thumbnail of Deep autoregressive density nets vs neural ensembles for model-based offline reinforcement learning

arXiv (Cornell University), Feb 5, 2024

Research paper thumbnail of Multi-timestep models for Model-based Reinforcement Learning

arXiv (Cornell University), Oct 8, 2023

In model-based reinforcement learning (MBRL), most algorithms rely on simulating trajectories fro... more In model-based reinforcement learning (MBRL), most algorithms rely on simulating trajectories from one-step dynamics models learned on data. A critical challenge of this approach is the compounding of one-step prediction errors as length of the trajectory grows. In this paper we tackle this issue by using a multitimestep objective to train one-step models. Our objective is a weighted sum of a loss function (e.g., negative log-likelihood) at various future horizons. We explore and test a range of weights profiles. We find that exponentially decaying weights lead to models that significantly improve the long-horizon R2 score. This improvement is particularly noticeable when the models were evaluated on noisy data. Finally, using a soft actor-critic (SAC) agent in pure batch reinforcement learning (RL) and iterated batch RL scenarios, we found that our multi-timestep models outperform or match standard one-step models. This was especially evident in a noisy variant of the considered environment, highlighting the potential of our approach in real-world applications.

Research paper thumbnail of A Multi-step Loss Function for Robust Learning of the Dynamics in Model-based Reinforcement Learning

arXiv (Cornell University), Feb 5, 2024

Research paper thumbnail of Deep autoregressive density nets vs neural ensembles for model-based offline reinforcement learning

arXiv (Cornell University), Feb 5, 2024

Research paper thumbnail of Multi-timestep models for Model-based Reinforcement Learning

arXiv (Cornell University), Oct 8, 2023

In model-based reinforcement learning (MBRL), most algorithms rely on simulating trajectories fro... more In model-based reinforcement learning (MBRL), most algorithms rely on simulating trajectories from one-step dynamics models learned on data. A critical challenge of this approach is the compounding of one-step prediction errors as length of the trajectory grows. In this paper we tackle this issue by using a multitimestep objective to train one-step models. Our objective is a weighted sum of a loss function (e.g., negative log-likelihood) at various future horizons. We explore and test a range of weights profiles. We find that exponentially decaying weights lead to models that significantly improve the long-horizon R2 score. This improvement is particularly noticeable when the models were evaluated on noisy data. Finally, using a soft actor-critic (SAC) agent in pure batch reinforcement learning (RL) and iterated batch RL scenarios, we found that our multi-timestep models outperform or match standard one-step models. This was especially evident in a noisy variant of the considered environment, highlighting the potential of our approach in real-world applications.

Log In