Computational Photography and Video @ MIT (original) (raw)

The digital photography revolution has greatly facilitated the way in which we take and share pictures. However, it has mostly relied on a rigid imaging model inherited from traditional photography. Computational photography and video go one step further and exploit digital technology to enable arbitrary computation between the light array and the final image or video. Such computation can overcome limitations of the imaging hardware and enable new applications. It can also enable new imaging setups and postprocessing tools that empower users to enhance and interact with their images and videos.

Matting and layer extraction

Exploring Defocus Matting: Nonparametric Acceleration, Super-Resolution, and Off-Center Matting
Neel Joshi, Wojciech Matusik Shai Avidan, Hanspeter Pfister, William T. Freeman
IEEE CG&A March/April 2007 (Vol. 27, No. 2)

Spectral Matting.
Anat Levin, Alex Rav-Acha, Dani Lischinski.
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Minneapolis, June 2007

Separating Reflections from Images by Use of Independent Components Analysis,
H. Farid and E.H. Adelson,
Journal of the Optical Society of America A, 16(9):2136-2145, (1999)

Motion depiction

Motion Magnification
Ce Liu, Antonio Torralba, William T. Freeman, Frédo Durand and Edward H. Adelson
SIGGRAPH 2005

Shapetime photography
W. T Freeman and H. Zhang
IEEE Computer Vision and Pattern Recognition (CVPR), Madison, WI, June, 2003

Motion Without Movement
Freeman, W. T., Adelson, E. H., and Heeger, D. J.
ACM Computer Graphics (SIGGRAPH), 25(4):27-30 (1991).

Video analysis and representation

A Topological Approach to Hierarchical Segmentation using Mean Shift
Sylvain Paris and Frédo Durand
Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR'07)

Analysis of contour motions
C. Liu, W. T. Freeman and E. H. Adelson
Advances in Neural Information Processing Systems (NIPS 2006)

Learning Motion Analysis
W. T. Freeman, J. A. Haddon, and E. C. Pasztor
To appear in "Statistical Theories of the Brain", edited by R. Rao, B. Olshausen, and M. Lewicki, MIT Press, 2001. MERL-TR2000-32.

Layered Representations for Vision and Video
Adelson, E. H.,
Proceedings of IEEE Workshop on Representation of Visual Scenes, in conjunction with ICCV '95, pp.3-9, Cambridge, MA; June (1995).

Applying Mid-Level Vision Techniques for Video Data Compression and Manipulation
Wang, J. Y. A., Adelson, E. H., and Desai, U.,
Proceedings of SPIE on Digital Video Compression on Personal Computers: Algorithms and Technologies, 2187:116-127 San Jose; February (1994).

Representing Moving Images with Layers Wang, J. Y. A., and Adelson, E. H.
IEEE Transactions on Image Processing, 3(5):625-638, (1994).

Probability Distributions of Optical Flow
Simoncelli, E. P., Adelson, E. H., and Heeger, D. J.
IEEE Conference on Computer Vision and Pattern Recognition, Mauii, Hawaii; June (1991).

Deblurring, shake removal

Removing camera shake from a single image R. Fergus, B. Singh, A. Hertzmann, S. Roweis, and W. T. Freeman
SIGGRAPH 2006

Texture synthesis and transfer

Image quilting for texture synthesis and transfer
A. Efros and W. T Freeman
SIGGRAPH 2001

Super-resolution

Single-frame Text Super-resolution: A Bayesian Approach
G. Dalley, W. T. Freeman, and J. Marks
International Conference on Image Processing (ICIP), Oct. 2004

Exploiting the sparse derivative prior for super-resolution and image demosaicing
M. F. Tappen, B. C. Russell, and W. T. Freeman
3rd Intl. Workshop on Statistical and Computational Theories of Vision (associated with Intl. Conf. on Computer Vision), Nice, France, October, 2003

Example-based super-resolution
William T. Freeman, Thouis R. Jones, and Egon C. Pasztor
IEEE Computer Graphics and Applications, March/April, 2002.

Learning low-level vision.
William T. Freeman, Egon C. Pasztor
IEEE International Conference on Computer Vision, Corfu, Greece, 1999.

Learning to estimate scenes from images.
William T. Freeman, Egon C. Pasztor
Neural Information Processing Systems, volume 11, 1999

Denoising

Noise estimation from a single image
C. Liu, W. T. Freeman, R. Szeliski, and S. B. Kang
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) New York, NY, June, 2006

Noise Removal via Bayesian Wavelet Coring
Simoncelli, E. P., and Adelson, E. H.
Third IEEE International Conference on Image Processing. Lausanne, Switzerland; September (1996).

Subband Coring for Image Noise Reduction
Adelson, E. H.
Internal Report: RCA Sarnoff Labortories, Princeton, NJ (1986).

Image processing foundations

The steerable pyramid: a flexible architecture for multi-scale derivative computation E. P. Simoncelli and W. T. Freeman
2nd Annual IEEE International Conference on Image Processing, Washington, DC.

Shiftable Multiscale Transforms
Simoncelli, E. P., Freeman, W. T., Adelson, E. H., and Heeger, D. J.
IEEE Transactions on Information Theory, 38:587-607 (1992).

Steerable Filters and Local Analysis of Image Structure
W. T. Freeman
Ph.D. Thesis, Massachusetts Institute of Technology, 1992

The Design and Use of Steerable Filters
Freeman, W. H., and Adelson, E. H.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 13:891-906 (1991).

Subband Transforms
Simoncelli, E., and Adelson, E. H.
In J. Woods (ed.), Subband Image Coding (pp. 143-192), Norwell, MA: Kluwer Academic Publishers (1991).

Orthogonal Pyramid Transforms for Image Coding
Adelson, E. H., Simoncelli, E., and Hingorani, R.
Visual Communications and Image Processing II, Proc. SPIE, Vol. 845, pp. 50-58 Cambridge, MA; October 27-29 (1987).

Pyramid-Based Computer Graphics
Ogden, J. M., Adelson, E. H., Bergen, J. R., and Burt, P. J.
RCA Engineer, 30(5):4-15 (1985).

Pyramid Methods in Image Processing
Adelson, E. H., Anderson, C. H., Bergen, J. R., Burt, P. J., and Ogden, J. M.
RCA Engineer, 29(6):33-41 (1984).

The Laplacian Pyramid as a Compact Image Code
Burt, P., and Adelson, E. H.
IEEE Transactions on Communication, COM-31:532-540 (1983).

A Multiresolution Spline with Application to Image Mosaics
Burt, P., and Adelson, E. H.
ACM Transactions on Graphics, 2(4):217-236 (1983). (portal color version)

Saliency and emphasis

Contextual guidance of eye movements and attention in real-world scenes: The role of global features on object search
Antonio Torralba, Aude Oliva, Monica Castelhano, John Henderson
Psychological Review, Vol. 113, No. 4. (October 2006), pp. 766-786.

De-Emphasis of Distracting Image Regions Using Texture Power Maps
Sara L. Su, Frédo Durand, and Maneesh Agrawala
Proc. of Texture 2005, Beijing, China, October 2005.

Human Learning of Contextual Priors for Object Search: Where does the time go?
B. Hidalgo-Sotelo, A. Oliva, and A. Torralba
Proceedings of the 3rd Workshop on Attention and Performance in Computer Vision at the Int. CVPR, 2005.

Contextual Influences on Saliency
A. Torralba
Neurobiology of Attention, Eds. L. Itti, G. Rees and J. Tsotsos. Pages 586-593. Academic Press / Elsevier. 2005

Saliency, objects and scenes: global scene factors in attention and object detection
A. Torralba, A. Oliva, M. Castelhano and J. M. Henderson
Vision Sciences Society Annual Meeting, Sarasota. 2004.

Modeling global scene factors in attention
A. Torralba
Journal of Optical Society of America A. Special Issue on Bayesian and Statistical Approaches to Vision. Vol. 20(7): 1407-1418, 2003.

Top-down control of visual attention in object detection
A. Oliva, A. Torralba, M. S. Castelhano and J. M. Henderson
Proceedings of the IEEE International Conference on Image Processing. Vol. I, pages 253-256; September 14-17, in Barcelona, Spain, 2003.

Color and white balance

Bayesian model of human color constancy
D. H. Brainard, P. Longere, P. B. Delahunt, W. T. Freeman, J. M. Kraft, and B. Xiao
Journal of Vision, 6, 1267-1281

Exploiting spatial and spectral image regularities for color constancy
B. Singh, W. T. Freeman, and D. H. Brainard
3rd Intl. Workshop on Statistical and Computational Theories of Vision (associated with Intl. Conf. on Computer Vision), Nice, France, October, 2003

Bayesian Color Constancy
D. H. Brainard and W. T. Freeman
Journal of the Optical Society of America, A, 14(7), pp. 1393-1411, July, 1997

Bayesian decision theory, the maximum local mass estimate, and color constancy
W. T. Freeman and D. H. Brainard
Fifth International Conference on Computer Vision, IEEE Computer Society, Cambridge, MA, U.S.A, June, 1995, pp. 210 - 217

Non-Photorealistic styles

An Interactive Artificial Ant Approach to Non-Photorealistic Rendering
Yann Semet, Una-May O'Reilly, Frédo Durand
GECCO'04: Genetic and Evolutionary COmputation Conference

An Invitation to Discuss Computer Depiction
Frédo Durand
ACM/Eurographics Symp. NPAR'02.

Decoupling Strokes and High-Level Attributes for Interactive Traditional Drawing
Frédo Durand, Victor Ostromoukhov, Mathieu Miller, François Duranleau, and Julie Dorsey
in the Proceedings of the 12th Eurographics Workshop on Rendering, June 2001.