The Frankencamera: An Experimental Platform for Computational Photography (original) (raw)

Adams, A., Talvala, E., Park, S. H., Jacobs, D. E., Ajdin, B., Gelfand, N., Dolson, J., Vaquero, D., Baek, J., Tico, M., Lensch, H. P., Matusik, W., Pulli, K., Horowitz, M., and Levoy, M. 2010. The Frankencamera: an experimental platform for computational photography. In ACM SIGGRAPH 2010 Papers (Los Angeles, California, July 26 - 30, 2010). H. Hoppe, Ed. SIGGRAPH '10. ACM, New York, NY, 1-12. DOI= http://doi.acm.org/10.1145/1833349.1778766

New!

We've publically released the FCam API for N900. This is the API for the Frankencamera architecture described in the paper, and it's how we implemented all of the applications in the paper. It's open source, and comes with some neat example apps. Go here to get started.

Abstract

Although there has been much interest in computational photography within the research and photography communities, progress has been hampered by the lack of a portable, programmable camera with sufficient image quality and computing power. To address this problem, we have designed and implemented an open architecture and API for such cameras: the Frankencamera. It consists of a base hardware specification, a software stack based on Linux, and an API for C++. Our architecture permits control and synchronization of the sensor and image processing pipeline at the microsecond time scale, as well as the ability to incorporate and synchronize external hardware like lenses and flashes. This paper specifies our architecture and API, and it describes two reference implementations we have built. Using these implementations we demonstrate six computational photography applications: HDR viewfinding and capture, low-light viewfinding and capture, automated acquisition of extended dynamic range panoramas, foveal imaging, IMU-based hand shake detection, and rephotography. Our goal is to standardize the architecture and distribute Frankencameras to researchers and students, as a step towards creating a community of photographer-programmers who develop algorithms, applications, and hardware for computational cameras.

At ACM: Full reference

Preprint: <fcam.pdf>

SIGGRAPH Presentation Slides (pdf): <fcam-sigtalk.pdf>

Video: <fcam.mov>

Supplemental Material: API Documentation and Example Programs

Reprinted in: CACM, November 2012, with an introductory technical perspective by Richard Szeliski

FCam Software: The FCam API is now available for the Nokia N900. Get it here.

Related paper: If you're wondering how we think the Frankencamera (and future platforms like it) might affect the camera industry, read our article in IEEE CG&A,Experimental platforms for computational photography.

Articles in the press: The web page of ourCamera 2.0 projectincludes a list of articles that have been written about this project. However, not all of them are accurate. We recommend starting with the two Stanford Reports.