Human Motion Modeling and Analysis (original) (raw)

Motion

Instructors: Yaser Sheikh (CMU), Leonid Sigal (Disney Research), Iain Matthews (Disney Research)

MW 3:00-4:20 GHC 5222

Office Hours
Yaser Sheikh: EDSH 221 Tuesday 2pm-3pm
Leon Sigal: CIC (Lower Level) Thursday 1pm-2pm

Human motion analysis is used in applications as varied as special effects in movies, animation, sport training, physical rehabilitation for the disabled, and human-robot/human-computer interaction. This course will survey state-of-the-art techniques, in the industry and academia, to capture, model, and analyze human motion. The course will be a mix between lectures and seminar-style paper reading of recent research into human motion modeling and analysis. The course evaluation will be project-based, in which students will capture their own body and face motion, and build projects around the data they collect individually and as a group.

In particular, we will cover:
Capture Techniques: We will describe and use various systems including motion capture, video-based capture, depth sensors, scanners, and eye-gaze trackers.
Modeling and Representation: We will cover classic and contemporary representations of face and body pose and motion, including statistical and physics-based techniques.
Applications: As human motion analysis becomes increasingly mature, new applications are emerging. We will study recent progress in animation, synthesis, classification, and rehabilitation.

Date

Topic

Material

Lecture 1

September 10

Introduction to Human Motion

[ PDF ]

Part I: Capture

Lecture 2

September 12

Virtualizing Reality: Introduction to markerless motion capture
Reading Assignment: T. Kanade and P. J. Narayanan, Virtualized Reality: Perspectives on 4D Digitization of Dynamic Events, IEEE Computer Graphics and Applications, 2007.

[ PDF ]

Lecture 3

September 17

Marker-based Capture: Introduction to marker-based motion capture
Reading Assignment: G. Welch and E. Foxlin, Motion tracking: No silver bullet, but a respectable arsenal, IEEE Computer Graphics and Applications 2002.

[ PDF ]

Lecture 4

September 19

Motion Capture Lab Demo: Live demonstration of motion capture at the CMU motion capture lab.

Lecture 5

September 24

Game Capture: Introduction to game capture tech.
Reading Assignment: Jamie Shotton et al., Real-Time Human Pose Recognition in Parts from a Single Depth Image, CVPR 2011.

[ PDF ]

Lecture 6

September 26

Facial Capture: Introduction to face capture tech.
Reading Assignment: T. Beeler et al., High-quality passive facial performance capture using anchor frames, ACM Transactions on Graphics, 2011.

[ PDF ]

Part II: Pose Modeling

Lecture 7

October 1

Human Body Representation: Articulated Systems, Anatomical Classification
Reading Assignment: J. O'Brien, R. Bodenheimer, G. Brostow, and J. Hodgins Automatic Joint Parameter Estimation from Magnetic Motion Capture Data, 2000.

[ [PDF](Lecture-7-Articulated Body Representation.pdf) ]

Lecture 8

October 3

Facial Representations: Facial Action Units, Blendshapes, Pose Space Deformations
Reading Assignment: J. P. Lewis, Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation, SIGGRAPH, 2000.

[ [PDF](Lecture-8-Face Representation.pdf) ]

Lecture 9

October 8

Mesh Representation: Triangulation, Skinning
Reading Assignment: B. Allen et al., The space of all body shapes: reconstruction and parameterization from range scans. SIGGRAPH 2003.

[ [PDF](Lecture-9-Skinning and Body Representations.pdf) ]

Lecture 10

October 10

Project Pitch: Students pitch their final projects for comments and discussion. Each student will have 3 minutes to present

Lecture 11

October 15

Capture Project Review: Student teams will present their capture results. Each team will have 15 minutes to present
Group 1: Motion Capture
Group 2: Face Scans
Group 3: Multiview Performance
Group 4: Face Motion Capture

Part III: Motion Modeling & Applications

Lecture 12

October 17

Statistical Models I: Latent Variable Models, PCA, GPLVMs, Isomap
Reading Assignment: Safanova et al., Synthesizing Physically Realistic Human Motion in Low-Dimensional, Behavior-Speciﬁc Spaces .

[ PDF ]

Lecture 13

October 22

Application: Motion Retargeting
Paper 1: Semantic Deformation Transfer
Paper 2: Style-based Inverse Kinematics

Lecture 14

October 24

Statistical Models II: Linear Dynamical Models
Reading Assignment: D. Fleet, Motion models for people tracking. Guide to Visual Analysis of Humans: Looking at People, 2011.

[ [PDF](Lecture-12-Dynamical Models.pdf) ]