CMU Panoptic Dataset (original) (raw)
8:30-8:45
Opening
8:45-9:05
Invited Talk: Many-Camera Systems: How They Started
9:05-9:30
Design Principles for Multiview System, Hardware Design, Structure, Synchronization, and Networking
- How to select cameras?
- How many cameras do we need?
- Where to place cameras?
- How to synchronize cameras?
- How do the cameras communicate?
9:30-10:00
Invited Talk: New Methods for Marler-less Motion and Performance Capture and the Multi-Camera Studio Behind
Absract: In this talk, I will review some of our work on marker-less motion and performance capture with multi-view video. While originally, we looked into methods working in more or less controlled indoor settings, we recently began to look into approaches for motion and performance capture that work in less constrained and more general outdoor scenes, with a much lower number of cameras, even a single one. As an example, I will show our recent VNect approach (SIGGRAPH 2017), the first method for real-time 3D human motion capture with a single webcam. All our works depend on a versatile multi-view video studio for capturing data. I will show how this studio looks like and what guided our studio design decisions.
10:00-10:30
Data/storage, Capture System, Calibration, Multiple Kinect System
- How to design storage system?
- How to design and run capture software?
- How to calibrate multiview camera system?
- How to build multiple kinect system (sync and calibration)?
10:30-11:00
Coffee Break
11:00-11:30
Invited Talk: Multi-view Capture for High Resolution Digital Humans
Absract: Creating realistic digital humans is becoming increasingly more important in the visual effects industry. Capturing the likeness of an actor using multi-view reconstruction is now a standard procedure. In order to help avoid the infamous uncanny valley for the digital character, capturing the shape and motion at the highest possible quality and resolution is a necessity. In this talk we will briefly outline a number of capture setups and corresponding reconstruction algorithms for capturing actors in high resolution. We will touch upon the particulars of different setups, including cameras, lighting, synchronization, calibration, portability, and comfort for the actor. We will primarily focus on facial capture, but also dive a little into reconstructing hair and eyes. As part of this talk we will describe our Medusa Facial Performance Capture System, currently in use by Industrial Light & Magic and already demonstrated on over a dozen feature films.
11:30-12:00
3D Reconstruction Methods, Multiview Bootstrapping, and Panoptic Studio Dataset
- 3D Volume Reconstruction
- 3D Trajectory Reconstruction
- 3D Keypoint Reconstruction
- Multiview Bootstrapping
- Why do we need multiview system in deep learning era?
- The Panoptic Studio Dataset
12:00-12:30
DIY A Multiview System and Demo
In conjunction with presentations, we will demonstrate a multicamera system (~10 cameras) from part assembly to realtime 3D reconstruction.
- How to build a high-end multicamera system with consumer grade equipment (no specialized hardware) and open source software?
- How to achieve real time computation through a cost effective single board computer (e.g., Raspberry Pi, Odroid, nVidia TX2)?
- How much does it cost?
- How can I contribute to the multicamera system community in terms of hardware design and software?