CS 330 Deep Multi-Task and Meta Learning (original) (raw)

Fall 2020, Class: Mon, Wed 1:00-2:20pm

Description:

While deep learning has achieved remarkable success in supervised and reinforcement learning problems, such as image classification, speech recognition, and game playing, these models are, to a large degree, specialized for the single task they are trained for. This course will cover the setting where there are multiple tasks to be solved, and study how the structure arising from multiple tasks can be leveraged to learn more efficiently or effectively. This includes:

This is a graduate-level course. By the end of the course, students will be able to understand and implement the state-of-the-art multi-task learning and meta-learning algorithms and be ready to conduct research on these topics.

Format:

The course will include live lectures over zoom, three homework assignments, a fourth optional homework assignment, and a course project. The lectures will discuss the fundamentals of topics required for understanding and designing multi-task and meta-learning algorithms in both supervised learning and reinforcement learning domains. The assignments will focus on coding problems that emphasize these fundamentals. Finally, students will present a short spotlight of their project proposal and, at the end of the quarter, their completed projects.

Prerequisites:

CS 229 or an equivalent introductory machine learning course is required. CS 221 or an equivalent introductory artificial intelligence course is recommended but not required.

Lecture Videos:

If you are looking for publicly-available lecture videos from the Fall 2019 offering, they are here. Other materials from the Fall 2019 offering are here. Lecture videos from this Fall 2020 offering will be processed and made publicly available after the course. For students enrolled in the course, recorded lecture videos will be posted to canvas after each lecture.

Staff

Chelsea Finn

Prof. Chelsea Finn

Instructor
OH: Mon 2:30-3:30 pm
Webpage

Karol Hausman

Dr. Karol Hausman

Co-Lecturer
Webpage

Rafael Rafailov

Rafael Rafailov

Teaching Assistant
OH: Sun 1-2:30 pm

Dilip Arumugam

Dilip Arumugam

Teaching Assistant
OH: Fri 10-11:30 am

Mason Swofford

Mason Swofford

Teaching Assistant
OH: Thur 12-1:30 pm

Albert Tung

Albert Tung

Teaching Assistant
OH: Tue 4-5:30 pm

Karen Yang

Karen Yang

Teaching Assistant
OH: Wed 4:30-6 pm

Nikita Demir

Nikita Demir

Teaching Assistant
OH: Mon 6.30-8 pm

Suraj Nair

Suraj Nair

Teaching Assistant
OH: Thur 7-8:30 pm

Timeline

Date Lecture Deadlines Optional reading
Week 1 Mon, Sep 14 Lecture Course introduction
Week 1 Wed, Sep 16 Lecture Supervised multi-task learning, transfer learning P1: Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. Kendall et al. (2018) P2: Universal Language Model Fine-tuning for Text Classification. Howard et al. (2018)
Week 1 Thu, Sep 17 TA Session TensorFlow tutorial
Week 2 Mon, Sep 21 Lecture Meta-learning problem statement, black-box meta-learning Homework 1 out [PDF][Colab Notebook] P1:One-shot Learning with Memory-Augmented Neural Networks. Santoro et al. (2016)
Week 2 Wed, Sep 23 Lecture Optimization-based meta-learning P1: Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. Finn et al. (2017) P2: Meta-Learning with Differentiable Convex Optimization. Lee et al. (2019)
Week 3 Mon, Sep 28 Guest Lecture Automatic differentiation (Matthew Johnson, Google Brain) [Class Colab][Additional Colab]
Week 3 Wed, Sep 30 Lecture Few-shot learning via metric learning Due Homework 1 P1: Matching Networks for One Shot Learning. Vinyals et al. (2017) P2: Prototypical Networks for Few-shot Learning. Snell et al. (2017)
Week 4 Mon, Oct 5 Lecture Advanced meta-learning topics Homework 2 out [PDF][Colab Notebook] P1: Meta-Learning without Memorization. Yin et al. (2020)
Week 4 Wed, Oct 7 Leacture Bayesian meta-learning P1: Conditional Neural Processes. Garnelo et al. (2018) P2: Meta-Learning Probabilistic Inference For Prediction. Gordon et al. (2019)
Week 5 Mon, Oct 12 Lecture Renforcement learning primer, multi-task RL, goal-conditioned RL (Karol Hausman) P1: Hindsight Experience Replay. Andrychowicz et al. (2018)
Week 5 Wed, Oct 14 Presentations Project Proposal Spotlight Presentations Due Project proposal
Week 5 Fri, Oct 16 Due Homework 2 Homework 3 out [PDF][Colab Notebook]
Week 6 Mon, Oct 19 Lecture Model-based RL for multi-task learning P1: Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control. Ebert et al. (2018) P2: Deep Dynamics Models for Learning Dexterous Manipulation. Nagabandi et al. (2019)
Week 6 Wed, Oct 21 Lecture Meta-RL: Adaptable models and policies
Week 7 Mon, Oct 26 Lecture Meta-RL: Learning to explore Due Homework 3 Optional Homework 4 out [PDF][Colab Notebook] P1: VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning. Zingraf et al. (2020)
Week 7 Wed, Oct 28 Lecture A graphical model perspective on multi-task and meta-RL (Karol Hausman) P1: Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review. Levine et al. (2018) P2: Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables. Rakelly et al. (2019)
Week 7 Thu, Oct 29 TA Session Pytorch tutorial
Week 8 Mon, Nov 2 Lecture Hierarchical RL and skill discovery (Karol Hausman) Due Project milestone P1: Data-Efficient Hierarchical Reinforcement Learning. Nachum et al. (2018) P2: Diversity is All You Need: Learning Skills without a Reward Function. Eysenbach et al. (2018) P3: Dynamics-Aware Unsupervised Discovery of Skills. Sharma et al. (2019)
Week 8 Wed, Nov 4 Lecture Lifelong learning: problem statements, forward & backward transfer (Karol Hausman)
Week 9 Mon, Nov 9 Guest Lecture Meta-learning & cognitive science (Jane Wang, DeepMind) Due Optional Homework 4 Lecture is at 9 am PST
Week 9 Wed, Nov 11 Lecture Frontiers and open problems
Week 10 Mon, Nov 16 Presentations Final project presentations Due Final presentations slides
Week 10 Wed, Nov 18 Presentations Final project presentations
Week 10 Fri, Nov 20 Due Final project report

Grading and Course Policies

Homeworks (15% each): There are three homework assignments, each worth 15% of the grade. Assignments will require training neural networks in TensorFlow in a Colab notebook. There is also a fourth homework assignment that will either replace one prior homework grade or part of the project grade (whichever is better for grade). All assignments are due to Gradescope at 11:59 pm Pacific Time on the respective due date.

Project (55%): There's a research-level project of your choice. You may form groups of 1-3 students to complete the project, and you are encouraged to start early! Further guidelines on the project will be posted shortly.

Late Days: You have 6 total late days across homeworks and project-related assignment submissions. You may use a maximum of 2 late days for any single assignment.

Honor Code: You are free to form study groups and discuss homeworks. However, you must write up homeworks and code from scratch independently. When debugging code together, you are only allowed to look at the input-output behavior of each other's programs and not the code itself.

Note on Financial Aid

All students should retain receipts for books and other course-related expenses, as these may be qualified educational expenses for tax purposes. If you are an undergraduate receiving financial aid, you may be eligible for additional financial aid for required books and course materials if these expenses exceed the aid amount in your award letter. For more information, review your award letter or visit the Student Budget website.

All Projects