Peter Belhumeur - Academia.edu (original) (raw)

Uploads

Papers by Peter Belhumeur

Research paper thumbnail of Representations for Recognition Under Variable Illumination

Lecture Notes in Computer Science, 1999

ABSTRACT

Research paper thumbnail of Shadows, Shading, and Projective Ambiguity

Lecture Notes in Computer Science, 1999

In a scene observed from a fixed viewpoint, the set of shadow curves in an image changes as a poi... more In a scene observed from a fixed viewpoint, the set of shadow curves in an image changes as a point light source (nearby or at infinity) assumes different locations. We show that for any finite set of point light sources illuminating an object viewed under either orthographic or per- spective projection, there is an equivalence class of object shapes having the same set of shadows. Members of this equivalence class differ by a four parameter family of projective transformations, and the shadows of a transformed object are identical when the same transformation is applied to the light source locations. Under orthographic projection, this family is the generalized bas-relief (GBR) transformation, and we show that the GBR transformation is the only family of transformations of an object’s shape for which the complete set of imaged shadows is identical. Furthermore, for objects with Lambertian surfaces illuminated by dis- tant light sources, the equivalence class of object shapes which preserves shadows also preserves surface shading. Finally, we show that given mul- tiple images under differing and unknown light source directions, it is possible to reconstruct an object’s shape up to these transformations from the shadows alone.

Research paper thumbnail of <title>Imaging in multiplexed illumination</title>

Nonimaging Optics and Efficient Illumination Systems, 2004

Lighting plays an important role in many applications of computer vision, machine vision and comp... more Lighting plays an important role in many applications of computer vision, machine vision and computer graphics. Often, an object needs to be photographed multiple times, in each of which lighting comes from a different direction. Lighting which uses a single source per image is prone to dynamic range problems, especially in dark areas and in specular highlights. In addition, it becomes a practical problem to use an increasingly larger number of discrete sources (say, hundreds). To counter these problems we develop a novel illumination strategy. In each image, multiple light sources irradiate the scene simultaneously. The set of light sources is different in each image, but not mutually exclusive. Then, the contribution of each individual source is extracted in computational post-processing. The number of acquired images using this approach is the same as the number used in single-source images. However, thanks to the multiplexing of light in the raw images, more light is used from a variety of directions, diminishing problems of dynamic range. We derive the optimal illumination multiplexing scheme, which increases the SNR of the images by (&surd;n}/2, where {n} is the number of sources. This lighting strategy is complemented by a novel illumination setup. The setup is easily built and scaled to a huge number of sources, and is controllable by the computer. These advantages are obtained since the apparatus is based on indirect lighting originating from an LCD projector.

Research paper thumbnail of Specularity Removal in Images and Videos: A PDE Approach

Lecture Notes in Computer Science, 2006

Research paper thumbnail of Leafsnap: A Computer Vision System for Automatic Plant Species Identification

Lecture Notes in Computer Science, 2012

ABSTRACT We describe the first mobile app for identifying plant species using automatic visual re... more ABSTRACT We describe the first mobile app for identifying plant species using automatic visual recognition. The system --- called Leafsnap --- identifies tree species from photographs of their leaves. Key to this system are computer vision components for discarding non-leaf images, segmenting the leaf from an untextured background, extracting features representing the curvature of the leaf&#39;s contour over multiple scales, and identifying the species from a dataset of the 184 trees in the Northeastern United States. Our system obtains state-of-the-art performance on the real-world images from the new Leafsnap Dataset --- the largest of its kind. Throughout the paper, we document many of the practical steps needed to produce a computer vision system such as ours, which currently has nearly a million users.

Research paper thumbnail of Searching the World’s Herbaria: A System for Visual Identification of Plant Species

Lecture Notes in Computer Science, 2008

Research paper thumbnail of Sensor and network topologies of formations with direction, bearing and angle information between agents

42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475), 2003

Research paper thumbnail of FaceTracer: A Search Engine for Large Collections of Images with Faces

Lecture Notes in Computer Science, 2008

Abstract. We have created the first image search engine based entirely on faces. Using simple tex... more Abstract. We have created the first image search engine based entirely on faces. Using simple text queries such as “smiling men with blond hair and mustaches,” users can search through over 3.1 million faces which have been automatically labeled on the basis of several ...

Research paper thumbnail of Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction

Research paper thumbnail of POOF: Part-Based One-vs.-One Features for Fine-Grained Categorization, Face Verification, and Attribute Estimation

2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013

ABSTRACT From a set of images in a particular domain, labeled with part locations and class, we p... more ABSTRACT From a set of images in a particular domain, labeled with part locations and class, we present a method to automatically learn a large and diverse set of highly discriminative intermediate features that we call Part-based One-vs.-One Features (POOFs). Each of these features specializes in discrimination between two particular classes based on the appearance at a particular part. We demonstrate the particular usefulness of these features for fine-grained visual categorization with new state-of-the-art results on bird species identification using the Caltech UCSD Birds (CUB) dataset and parity with the best existing results in face verification on the Labeled Faces in the Wild (LFW) dataset. Finally, we demonstrate the particular advantage of POOFs when training data is scarce.

Research paper thumbnail of Dichromatic Separation: Specularity Removal and Editing (sketches 0668)

Research paper thumbnail of Toward£

Research paper thumbnail of General Index to Taxon Volume 55 (2006)

Research paper thumbnail of 1996 European Conference on Computer Vision

Research paper thumbnail of Area Chairs

Research paper thumbnail of COMMUNICATIONS IN INFORMATION AND SYSTEMS

Page 1. COMMUNICATIONS IN INFORMATION AND SYSTEMS 2006 Volume 6, Number 1 CONTENTS 1 Enhancement ... more Page 1. COMMUNICATIONS IN INFORMATION AND SYSTEMS 2006 Volume 6, Number 1 CONTENTS 1 Enhancement of Stochastic Resonance Using Optimization Theory Xingxing Wu, Zhong-ping Jiang, Daniel W. Repperger, and Yi Guo 19 Network Error Correction, Part I: Basic Concepts and Upper Bounds Raymond W. Yeung and Ning Cai 37 Network Error Correction, Part II: Lower Bounds Ning Cai and Raymond W. Yeung 55 Hilbert Space Methods for Control Theoretic Splines: a Unified Treatment Y. Zhou, M. Egerstedt, and C. Martin

Research paper thumbnail of Using Computer Vision in an Electronic Field Guide for Plants

We describe an ongoing project to digitize information about plant specimens and make it availabl... more We describe an ongoing project to digitize information about plant specimens and make it available to botanists in the field. This first requires digital images and models, and then effective retrieval and mobile computing mechanisms for accessing this information. We have almost ...

Research paper thumbnail of Organizing Committee Chairs

Research paper thumbnail of Proceedings Workshop on Photometric Analysis for Computer Vision (PACV'2007)

Research paper thumbnail of Leafsnap: Mobile applications for plant identification for ecologists and citizen scientists using image recognition technology

ABSTRACT Background/Question/Methods Both ecologists and citizen scientists need fast, reliable, ... more ABSTRACT Background/Question/Methods Both ecologists and citizen scientists need fast, reliable, and easy-to-use field guides for plant identification. Traditional paper-based guides or more recently introduced electronic field guides essentially use the same methodology, which is based on following dichotomous keys or selecting specific information on morphological features of the focal plant. With these types of identification aids a certain level of botanical skill is often required and in many cases a specific feature needed to make a correct identification (i.e., flower, fruit) is unavailable. We have developed a new automated identification tool that makes use of computerized image recognition technology. The tool, called Leafsnap, employs an algorithm that establishes the contours of the leaf of an unidentified tree and uses uniquely developed visual recognition software to find a match from a previously constructed digital library of leaf images. Results/Conclusions This free mobile app, currently available for the iPhone and iPad, identifies trees from photographs of their leaves and contains high-resolution images of their flowers, fruits, seeds, and bark. After taking a photo of a leaf of the plant to be identified, the iPhone will compare the photograph to a central library of 8000 images that have been collected and stored in the home database server. A ranked list of possible species is returned to the user at which time Leafsnap displays high-resolution images of a branch, leaf, flower, fruit, and bark of the identified species. The app also supplies descriptive information as well as native distribution. Leafsnap returns search results in 5 to 20 seconds, depending on the speed of the network connection. Ultimately it is up to the user to make the correct determination by using the high-resolution images in the database to compare critical diagnostic features with the species under question. Once an identification is made, the image is automatically sent to Leafsnap&#39;s home database along with mapping information taken from the smart phone&#39;s GPS. This information will eventually be used to track the geographic ranges of trees as they change over time. Currently the app is specific to about 200 tree species of the northeastern United States. It is planned that within two to three years Leafsnap will cover all the trees of North America, nearly 800 species in total.

Research paper thumbnail of Representations for Recognition Under Variable Illumination

Lecture Notes in Computer Science, 1999

ABSTRACT

Research paper thumbnail of Shadows, Shading, and Projective Ambiguity

Lecture Notes in Computer Science, 1999

In a scene observed from a fixed viewpoint, the set of shadow curves in an image changes as a poi... more In a scene observed from a fixed viewpoint, the set of shadow curves in an image changes as a point light source (nearby or at infinity) assumes different locations. We show that for any finite set of point light sources illuminating an object viewed under either orthographic or per- spective projection, there is an equivalence class of object shapes having the same set of shadows. Members of this equivalence class differ by a four parameter family of projective transformations, and the shadows of a transformed object are identical when the same transformation is applied to the light source locations. Under orthographic projection, this family is the generalized bas-relief (GBR) transformation, and we show that the GBR transformation is the only family of transformations of an object’s shape for which the complete set of imaged shadows is identical. Furthermore, for objects with Lambertian surfaces illuminated by dis- tant light sources, the equivalence class of object shapes which preserves shadows also preserves surface shading. Finally, we show that given mul- tiple images under differing and unknown light source directions, it is possible to reconstruct an object’s shape up to these transformations from the shadows alone.

Research paper thumbnail of <title>Imaging in multiplexed illumination</title>

Nonimaging Optics and Efficient Illumination Systems, 2004

Lighting plays an important role in many applications of computer vision, machine vision and comp... more Lighting plays an important role in many applications of computer vision, machine vision and computer graphics. Often, an object needs to be photographed multiple times, in each of which lighting comes from a different direction. Lighting which uses a single source per image is prone to dynamic range problems, especially in dark areas and in specular highlights. In addition, it becomes a practical problem to use an increasingly larger number of discrete sources (say, hundreds). To counter these problems we develop a novel illumination strategy. In each image, multiple light sources irradiate the scene simultaneously. The set of light sources is different in each image, but not mutually exclusive. Then, the contribution of each individual source is extracted in computational post-processing. The number of acquired images using this approach is the same as the number used in single-source images. However, thanks to the multiplexing of light in the raw images, more light is used from a variety of directions, diminishing problems of dynamic range. We derive the optimal illumination multiplexing scheme, which increases the SNR of the images by (&surd;n}/2, where {n} is the number of sources. This lighting strategy is complemented by a novel illumination setup. The setup is easily built and scaled to a huge number of sources, and is controllable by the computer. These advantages are obtained since the apparatus is based on indirect lighting originating from an LCD projector.

Research paper thumbnail of Specularity Removal in Images and Videos: A PDE Approach

Lecture Notes in Computer Science, 2006

Research paper thumbnail of Leafsnap: A Computer Vision System for Automatic Plant Species Identification

Lecture Notes in Computer Science, 2012

ABSTRACT We describe the first mobile app for identifying plant species using automatic visual re... more ABSTRACT We describe the first mobile app for identifying plant species using automatic visual recognition. The system --- called Leafsnap --- identifies tree species from photographs of their leaves. Key to this system are computer vision components for discarding non-leaf images, segmenting the leaf from an untextured background, extracting features representing the curvature of the leaf&#39;s contour over multiple scales, and identifying the species from a dataset of the 184 trees in the Northeastern United States. Our system obtains state-of-the-art performance on the real-world images from the new Leafsnap Dataset --- the largest of its kind. Throughout the paper, we document many of the practical steps needed to produce a computer vision system such as ours, which currently has nearly a million users.

Research paper thumbnail of Searching the World’s Herbaria: A System for Visual Identification of Plant Species

Lecture Notes in Computer Science, 2008

Research paper thumbnail of Sensor and network topologies of formations with direction, bearing and angle information between agents

42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475), 2003

Research paper thumbnail of FaceTracer: A Search Engine for Large Collections of Images with Faces

Lecture Notes in Computer Science, 2008

Abstract. We have created the first image search engine based entirely on faces. Using simple tex... more Abstract. We have created the first image search engine based entirely on faces. Using simple text queries such as “smiling men with blond hair and mustaches,” users can search through over 3.1 million faces which have been automatically labeled on the basis of several ...

Research paper thumbnail of Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction

Research paper thumbnail of POOF: Part-Based One-vs.-One Features for Fine-Grained Categorization, Face Verification, and Attribute Estimation

2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013

ABSTRACT From a set of images in a particular domain, labeled with part locations and class, we p... more ABSTRACT From a set of images in a particular domain, labeled with part locations and class, we present a method to automatically learn a large and diverse set of highly discriminative intermediate features that we call Part-based One-vs.-One Features (POOFs). Each of these features specializes in discrimination between two particular classes based on the appearance at a particular part. We demonstrate the particular usefulness of these features for fine-grained visual categorization with new state-of-the-art results on bird species identification using the Caltech UCSD Birds (CUB) dataset and parity with the best existing results in face verification on the Labeled Faces in the Wild (LFW) dataset. Finally, we demonstrate the particular advantage of POOFs when training data is scarce.

Research paper thumbnail of Dichromatic Separation: Specularity Removal and Editing (sketches 0668)

Research paper thumbnail of Toward£

Research paper thumbnail of General Index to Taxon Volume 55 (2006)

Research paper thumbnail of 1996 European Conference on Computer Vision

Research paper thumbnail of Area Chairs

Research paper thumbnail of COMMUNICATIONS IN INFORMATION AND SYSTEMS

Page 1. COMMUNICATIONS IN INFORMATION AND SYSTEMS 2006 Volume 6, Number 1 CONTENTS 1 Enhancement ... more Page 1. COMMUNICATIONS IN INFORMATION AND SYSTEMS 2006 Volume 6, Number 1 CONTENTS 1 Enhancement of Stochastic Resonance Using Optimization Theory Xingxing Wu, Zhong-ping Jiang, Daniel W. Repperger, and Yi Guo 19 Network Error Correction, Part I: Basic Concepts and Upper Bounds Raymond W. Yeung and Ning Cai 37 Network Error Correction, Part II: Lower Bounds Ning Cai and Raymond W. Yeung 55 Hilbert Space Methods for Control Theoretic Splines: a Unified Treatment Y. Zhou, M. Egerstedt, and C. Martin

Research paper thumbnail of Using Computer Vision in an Electronic Field Guide for Plants

We describe an ongoing project to digitize information about plant specimens and make it availabl... more We describe an ongoing project to digitize information about plant specimens and make it available to botanists in the field. This first requires digital images and models, and then effective retrieval and mobile computing mechanisms for accessing this information. We have almost ...

Research paper thumbnail of Organizing Committee Chairs

Research paper thumbnail of Proceedings Workshop on Photometric Analysis for Computer Vision (PACV'2007)

Research paper thumbnail of Leafsnap: Mobile applications for plant identification for ecologists and citizen scientists using image recognition technology

ABSTRACT Background/Question/Methods Both ecologists and citizen scientists need fast, reliable, ... more ABSTRACT Background/Question/Methods Both ecologists and citizen scientists need fast, reliable, and easy-to-use field guides for plant identification. Traditional paper-based guides or more recently introduced electronic field guides essentially use the same methodology, which is based on following dichotomous keys or selecting specific information on morphological features of the focal plant. With these types of identification aids a certain level of botanical skill is often required and in many cases a specific feature needed to make a correct identification (i.e., flower, fruit) is unavailable. We have developed a new automated identification tool that makes use of computerized image recognition technology. The tool, called Leafsnap, employs an algorithm that establishes the contours of the leaf of an unidentified tree and uses uniquely developed visual recognition software to find a match from a previously constructed digital library of leaf images. Results/Conclusions This free mobile app, currently available for the iPhone and iPad, identifies trees from photographs of their leaves and contains high-resolution images of their flowers, fruits, seeds, and bark. After taking a photo of a leaf of the plant to be identified, the iPhone will compare the photograph to a central library of 8000 images that have been collected and stored in the home database server. A ranked list of possible species is returned to the user at which time Leafsnap displays high-resolution images of a branch, leaf, flower, fruit, and bark of the identified species. The app also supplies descriptive information as well as native distribution. Leafsnap returns search results in 5 to 20 seconds, depending on the speed of the network connection. Ultimately it is up to the user to make the correct determination by using the high-resolution images in the database to compare critical diagnostic features with the species under question. Once an identification is made, the image is automatically sent to Leafsnap&#39;s home database along with mapping information taken from the smart phone&#39;s GPS. This information will eventually be used to track the geographic ranges of trees as they change over time. Currently the app is specific to about 200 tree species of the northeastern United States. It is planned that within two to three years Leafsnap will cover all the trees of North America, nearly 800 species in total.