Devendra Singh Chaplot (original) (raw)
I am a member of the Founding Team at Thinking Machines Lab where I work on Research + Product. Previously, I was part of the Founding Team at Mistral AI, where I worked on training Mistral 7B, Mixtral 8x7B & Mistral Large, and led the Multimodal research team that trained Pixtral 12B and Pixtral Large. Earlier I was a Reasearch Scientist at Facebook AI Research where I worked at the intersection of Computer Vision and Robotics. I received my Ph.D. at the Machine Learning Department, School of Computer Science, Carnegie Mellon University. During my Ph.D., I worked with Prof. Ruslan Salakhutdinov and Prof. Abhinav Gupta on Building Intelligent Autonomous Navigation Agents. Prior to joining CMU, I graduated from IIT Bombay, India with a B.Tech in Computer Science & Engineering and a minor in Applied Statistics in August 2014. I also worked for about a year at Samsung Electronics HQ in South Korea.
CV | Google Scholar | Github |
Email: dc[at]thinkingmachines.ai
Ph.D. Thesis
Building Intelligent Autonomous Navigation Agents
PDF | Talk | Slides
Updates
Nov '24
Announced Pixtral Large, frontier class multimodal model with open-weights.
Sep '24
Announced Pixtral 12B, Mistral AI's first multimodal model, Apache 2.0.
Jul '24
Announced Mistral Large 2, the new generation of our flagship model.
Dec '23
Released Mistral 8x7B, high-quality sparse mixture of experts model, Apache 2.0 .
Oct '23
Mistral 7B paper is on Arxiv.
Sep '23
Released Mistral 7B, the best 7B model to date, Apache 2.0 .
Mar '23
New ICCV Paper on Navigating to Objects specified by Images.
Feb '23
New Science Robotics paper on Navigating to Objects in the Real World.
Jan '23
Paper on Multi-skill Mobile Manipulation (M3) accepted to ICLR 2023 as spotlight!
Dec '22
Our Multi-skill Mobile Manipulation (M3) model wins the Habitat Rearrangement Challenge at NeurIPS 2022!
Oct '22
Released the HM3D-Semantics Dataset, the largest public dataset of real-world 3D spaces with dense semantic annotations!
Oct '22
New paper on Retrospectives on the Embodied AI Workshop .
Oct '22
New paper on Instance-Specific Image Goal Navigation task.
Jun '22
Co-organizing the Embodied AI Workshop and Habitat ObjectNav Challenge at CVPR 2022
Feb '22
PONI accepted to CVPR 2022 as oral!
Jan '22
FILM accepted to ICLR 2022
Aug '21
For students applying for graduate fellowships, I uploaded my research statement from the 2020 Facebook Fellowship application.
Jul '21
New ICML 2021 paper on Differentiable Spatial Planning using Transformers.
Apr '21
Excited to join Facebook AI Research as a Research Scientist!
Apr '21
Released code and pre-trained models for NeurIPS-20 paper on Object Goal Navigation.
Mar '21
I successfully defended my Ph.D. Thesis on "Building Intelligent Autonomous Navigation Agents".
Sep '20
Paper on Object Goal Navigation accepted at NeurIPS-20.
Jun '20
We won the CVPR 2020 Habitat ObjectNav Challenge! Thanks Google for $10,000 GCP Credits.
Summer '20
Excited to intern with Jitendra Malik and Deepak Pathak at Facebook AI Research!
Apr '20
Released code and pre-trained models for ICLR-20 Active Neural SLAM paper.
Feb '20
Paper on Neural Topological SLAM for Visual Navigation accepted at CVPR-20.
Jan '20
Received the Facebook Graduate Fellowship!
Dec '19
Received the Nvidia Graduate Fellowship! (Declined)
Dec '19
New paper on Learning to Explore using Active Neural SLAM accepted at ICLR-20.
Oct '19
We received the Facebook Research Award for PyRobot: Democratizing Robotics!
Jun '19
We won the CVPR-19 Habitat Navigation Challenge! Thanks Google for $20,000 GCP Credits.
Summer '19
Interned with Abhinav Gupta and Saurabh Gupta at Facebook AI Research, Pittsburgh.
Feb '19
New paper on Embodied Multimodal Multitask Learning posted on arxiv.
Oct '18
Excited to be a Workflow chair for ICML 2019.
Jun '18
Gave a talk at AIED-18 for our paper on Learning Cognitive Models using Neural Networks.
Summer '18
Interned with Dhruv Batra and Devi Parikh at Facebook AI Research, Menlo Park.
Jun '18
New AIED-18 paper on Learning Cognitive Models using Neural Networks posted on arxiv.
Jun '18
Paper on end-to-end global pose estimation using Neural Graph Optimization received Best Paper Award at the CVPR-18 Deep Learning for Visual SLAM workshop!
Jun '18
New ICML-18 paper on Gated Path Planning Networks posted on arxiv.
Apr '18
Released code, environment and pre-trained models for ICLR-18 Localization paper.
Mar '18
Gave a talk on DeepRL in 3D environments at the Nvidia GTC 2018.
Feb '18
New paper on our work at Apple AI Research on end-to-end global pose estimation using Neural Graph Optimization posted on arxiv.
Feb '18
Gave two talks at AAAI-18, on WSD and Language Grounding.
Jan '18
Paper on Active Neural Localization accepted at ICLR-18.
Jan '18
Released code, environment and pre-trained model for AAAI-18 Language Grounding paper.
Jan '18
AAAI-18 WSD paper available on arXiv.
Dec '17
Released code and pre-trained models for our Doom AI agent, Arnold. Play against our Doom AI agent which won the Visual Doom AI Competition 2017 Full Deathmatch.
Dec '17
Gave a talk on Active Neural Localization at the NIPS-17 DeepRL Symposium.
Nov '17
Two papers accepted for oral presentation at AAAI-18 on WSD and Language Grounding.
Sep '17
Our Doom AI agent, Arnold won the Visual Doom AI Competition 2017 Full Deathmatch.
Jun '17
MIT TechReview and Inverse write about our recent paper on language grounding!
Jun '17
New paper on Language Grounding posted on arXiv.
Select Research Projects
Navigating to Objects in the Real World
SEAL: Self-supervised Embodied Active Learning
Goal-Oriented Semantic Exploration
Neural Topological SLAM
Active Neural SLAM
Learning to follow language instructions
Learning to play deathmatches in Doom
Playing FPS Games with Deep Reinforcement Learning
Guillaume Lample*, Devendra Singh Chaplot*. (2017)
31st AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, USA.
Arnold: An Autonomous Agent to play FPS Games
(Best Demo Award)
Devendra Singh Chaplot*, Guillaume Lample*. (2017)
31st AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, USA. (demo)
PDF | Code | Pre-trained models | Demo videos (300K+ views)
Media:TechCrunch,Popular Science,Engadget,Daily Mail,Salon,Kotaku, ScienceAlert,Pittsburgh Post-Gazette,Inverse,CMU News
First place Visual Doom AI Competition 2017 Full Deathmatch
Second place Visual Doom AI Competition 2016 Full Deathmatch
See Google Scholar for a complete list of publications.
Talks
Building Intelligent Autonomous Navigation Agents
Ph.D. Thesis Defense
Video,Slides
Semantic Curiosity for Active Visual Learning
ECCV 2020 (spotlight)
Video,Slides
Object Goal Navigation using Goal-Oriented Semantic Exploration
CVPR 2020 Embodied AI Workshop
Winning entry for Habitat ObjectNav Challenge
Video,Slides
Neural Topological SLAM for Visual Navigation
CVPR 2020 Main Conference
CVPR 2020 Workshop on 3D Scene Understanding for Vision, Graphics, and Robotics
Video,Slides
Learning to Explore using Active Neural SLAM
ICLR 2020
CVPR 2019 Habitat Embodied Agents Workshop, Winning entry
Video,Slides
Tutorial on Deep Reinforcement Learning
2019 Summer Workshop on Machine Learning, Tepper School of Business, CMU, Pittsburgh
Workshop,Google Colab Notebook
Playing FPS Games with Deep Reinforcement Learning
Doom and Unreal Game Engines
Embodied Agents and Environments Workshop 2018, Facebook AI Research, Menlo Park
Slides
Gated-Attention Architectures for Task-Oriented Language Grounding
AAAI 2018, oral
Slides
Knowledge-based Word Sense Disambiguation using Topic Models
AAAI 2018, oral
Slides
Active Neural Localization
NIPS 2017, Deep Reinforcement Learning Symposium
Video,Slides