Jérémy RIVIERE - Academia.edu (original) (raw)
Papers by Jérémy RIVIERE
Computer Graphics Forum, Aug 25, 2015
Flash-based reflectometry (b) LCD-based reflectometry (c) Appearance transfer Figure 1: Examples ... more Flash-based reflectometry (b) LCD-based reflectometry (c) Appearance transfer Figure 1: Examples of surface reflectance recovered using mobile reflectometry: (a) A spatially varying rough specular material acquired using our handheld mobile flash-based reflectometry. (b) Highly specular surface reflectance recovered using mobile LCD-based reflectometry, with enhanced mesostructure from close-up observations under natural lighting. (c) Surface reflectance of a large spatial varying material sample recovered using appearance transfer under natural lighting from surface reflectance obtained using the LCD-based approach for a small reference patch.
ACM Transactions on Graphics, 2021
For several decades, researchers have been advancing techniques for creating and rendering 3D dig... more For several decades, researchers have been advancing techniques for creating and rendering 3D digital faces, where a lot of the effort has gone into geometry and appearance capture, modeling and rendering techniques. This body of research work has largely focused on facial skin, with much less attention devoted to peripheral components like hair, eyes and the interior of the mouth. As a result, even with the best technology for facial capture and rendering, in most high-end productions a lot of artist time is still spent modeling the missing components and fine-tuning the rendering parameters to combine everything into photo-real digital renders. In this work we propose to combine incomplete, high-quality renderings showing only facial skin with recent methods for neural rendering of faces, in order to automatically and seamlessly create photo-realistic full-head portrait renders from captured data without the need for artist intervention. Our method begins with traditional face ren...
TRS Transmitted Radiance Sinusoid DOLP Degree of Linear Polarization ρd Difuse albedo fr (σ ;η; ω... more TRS Transmitted Radiance Sinusoid DOLP Degree of Linear Polarization ρd Difuse albedo fr (σ ;η; ω⃗i , ω⃗o ) Specular BRDF n⃗ Surface normal Li (ω⃗i ) Incoming radiance θB Brewster angle E∥ Unit vector parallel to the plane of incidence E⊥ Unit vector perpendicular to the plane of incidence R∥ Relectance of p-polarized light R⊥ Relectance of s-polarized light δ Phase retardation between the orthogonally projected images of the electric vector Imax Maximum intensity of the Transmitted Radiance Sinusoid (TRS) Imin Minimum intensity of the Transmitted Radiance Sinusoid (TRS) φ Angular phase of the TRS φo Angle of rotation of the polariser with respect to camera coordinates xi,o,r ef Subscript indicates input (i), output (o) or relection (ref), where x can be any symbol listed thereafter P Degree of Linear Polarization (DOLP) ψ Angle of polarization expressed in local coordinates χ Ellipticity angle expressed in local coordinates ω⃗ Unit direction in spherical coordinates (ω⃗ = (θ , φ ))...
The rapid development of Augmented Reality (AR) and Virtual Reality (VR) applications over the pa... more The rapid development of Augmented Reality (AR) and Virtual Reality (VR) applications over the past years has created the need to quickly and accurately scan the real world to populate immersive, realistic virtual environments for the end user to enjoy. While geometry processing has already gone a long way towards that goal, with self-contained solutions commercially available for on-site acquisition of large scale 3D models, capturing the appearance of the materials that compose those models remains an open problem in general uncontrolled environments. The appearance of a material is indeed a complex function of its geometry, intrinsic physical properties and furthermore depends on the illumination conditions in which it is observed, thus traditionally limiting the scope of reflectometry to highly controlled lighting conditions in a laboratory setup. With the rapid development of digital photography, especially on mobile devices, a new trend in the appearance modelling community has emerged, that investigates novel acquisition methods and algorithms to relax the hard constraints imposed by laboratory-like setups, for easy use by digital artists. While arguably not as accurate, we demonstrate the ability of such self-contained methods to enable quick and easy solutions for on-site reflectometry, able to produce compelling, photo-realistic imagery. In particular, this dissertation investigates novel methods for on-site acquisition of surface reflectance based on off-the-shelf, commodity hardware. We successfully demonstrate how a mobile device can be utilised to capture high quality reflectance maps of spatially-varying planar surfaces in general indoor lighting conditions. We further present a novel methodology for the acquisition of highly detailed reflectance maps of permanent on-site, outdoor surfaces by exploiting polarisation from reflection under natural illumination. We demonstrate the versatility of the presented approaches by scanning various surfaces from the real world and show good qualitative and quantitative agreement with existing methods for appearance acquisition employing controlled or semi-controlled illumination setups.Open Acces
ACM Transactions on Graphics, 2017
We present a novel approach for on-site acquisition of surface reflectance for planar, spatially ... more We present a novel approach for on-site acquisition of surface reflectance for planar, spatially varying, isotropic samples in uncontrolled outdoor environments. Our method exploits the naturally occurring linear polarization of incident and reflected illumination for this purpose. By rotating a linear polarizing filter in front of a camera at three different orientations, we measure the polarization reflected off the sample and combine this information with multi-view analysis and inverse rendering in order to recover per-pixel, high resolution reflectance and surface normal maps. Specifically, we employ polarization imaging from two near orthogonal views close to the Brewster angle of incidence in order to maximize polarization cues for surface reflectance estimation. To the best of our knowledge, our method is the first to successfully extract a complete set of reflectance parameters with passive capture in completely uncontrolled outdoor settings. To this end, we analyze our app...
ACM Transactions on Graphics, 2018
We present a method to acquire dynamic properties of facial skin appearance, including dynamic di... more We present a method to acquire dynamic properties of facial skin appearance, including dynamic diffuse albedo encoding blood flow, dynamic specular intensity, and per-frame high resolution normal maps for a facial performance sequence. The method reconstructs these maps from a purely passive multi-camera setup, without the need for polarization or requiring temporally multiplexed illumination. Hence, it is very well suited for integration with existing passive systems for facial performance capture. To solve this seemingly underconstrained problem, we demonstrate that albedo dynamics during a facial performance can be modeled as a combination of: (1) a static, high-resolution base albedo map, modeling full skin pigmentation; and (2) a dynamic, one-dimensional component in the CIE L*a*b* color space, which explains changes in hemoglobin concentration due to blood flow. We leverage this albedo subspace and additional constraints on appearance and surface geometry to also estimate spec...
ACM SIGGRAPH 2014 Posters on - SIGGRAPH '14, 2014
Flash-based reflectometry (b) LCD-based reflectometry (c) Appearance transfer Figure 1: Examples ... more Flash-based reflectometry (b) LCD-based reflectometry (c) Appearance transfer Figure 1: Examples of surface reflectance recovered using mobile reflectometry: (a) A spatially varying rough specular material acquired using our handheld mobile flash-based reflectometry. (b) Highly specular surface reflectance recovered using mobile LCD-based reflectometry, with enhanced mesostructure from close-up observations under natural lighting. (c) Surface reflectance of a large spatial varying material sample recovered using appearance transfer under natural lighting from surface reflectance obtained using the LCD-based approach for a small reference patch.
ACM Transactions on Graphics, 2020
We propose a new light-weight face capture system capable of reconstructing both high-quality geo... more We propose a new light-weight face capture system capable of reconstructing both high-quality geometry and detailed appearance maps from a single exposure. Unlike currently employed appearance acquisition systems, the proposed technology does not require active illumination and hence can readily be integrated with passive photogrammetry solutions. These solutions are in widespread use for 3D scanning humans as they can be assembled from off-the-shelf hardware components, but lack the capability of estimating appearance. This paper proposes a solution to overcome this limitation, by adding appearance capture to photogrammetry systems. The only additional hardware requirement to these solutions is that a subset of the cameras are cross-polarized with respect to the illumination, and the remaining cameras are parallel-polarized. The proposed algorithm leverages the images with the two different polarization states to reconstruct the geometry and to recover appearance properties. We do ...
Computer Graphics Forum, Aug 25, 2015
Flash-based reflectometry (b) LCD-based reflectometry (c) Appearance transfer Figure 1: Examples ... more Flash-based reflectometry (b) LCD-based reflectometry (c) Appearance transfer Figure 1: Examples of surface reflectance recovered using mobile reflectometry: (a) A spatially varying rough specular material acquired using our handheld mobile flash-based reflectometry. (b) Highly specular surface reflectance recovered using mobile LCD-based reflectometry, with enhanced mesostructure from close-up observations under natural lighting. (c) Surface reflectance of a large spatial varying material sample recovered using appearance transfer under natural lighting from surface reflectance obtained using the LCD-based approach for a small reference patch.
ACM Transactions on Graphics, 2021
For several decades, researchers have been advancing techniques for creating and rendering 3D dig... more For several decades, researchers have been advancing techniques for creating and rendering 3D digital faces, where a lot of the effort has gone into geometry and appearance capture, modeling and rendering techniques. This body of research work has largely focused on facial skin, with much less attention devoted to peripheral components like hair, eyes and the interior of the mouth. As a result, even with the best technology for facial capture and rendering, in most high-end productions a lot of artist time is still spent modeling the missing components and fine-tuning the rendering parameters to combine everything into photo-real digital renders. In this work we propose to combine incomplete, high-quality renderings showing only facial skin with recent methods for neural rendering of faces, in order to automatically and seamlessly create photo-realistic full-head portrait renders from captured data without the need for artist intervention. Our method begins with traditional face ren...
TRS Transmitted Radiance Sinusoid DOLP Degree of Linear Polarization ρd Difuse albedo fr (σ ;η; ω... more TRS Transmitted Radiance Sinusoid DOLP Degree of Linear Polarization ρd Difuse albedo fr (σ ;η; ω⃗i , ω⃗o ) Specular BRDF n⃗ Surface normal Li (ω⃗i ) Incoming radiance θB Brewster angle E∥ Unit vector parallel to the plane of incidence E⊥ Unit vector perpendicular to the plane of incidence R∥ Relectance of p-polarized light R⊥ Relectance of s-polarized light δ Phase retardation between the orthogonally projected images of the electric vector Imax Maximum intensity of the Transmitted Radiance Sinusoid (TRS) Imin Minimum intensity of the Transmitted Radiance Sinusoid (TRS) φ Angular phase of the TRS φo Angle of rotation of the polariser with respect to camera coordinates xi,o,r ef Subscript indicates input (i), output (o) or relection (ref), where x can be any symbol listed thereafter P Degree of Linear Polarization (DOLP) ψ Angle of polarization expressed in local coordinates χ Ellipticity angle expressed in local coordinates ω⃗ Unit direction in spherical coordinates (ω⃗ = (θ , φ ))...
The rapid development of Augmented Reality (AR) and Virtual Reality (VR) applications over the pa... more The rapid development of Augmented Reality (AR) and Virtual Reality (VR) applications over the past years has created the need to quickly and accurately scan the real world to populate immersive, realistic virtual environments for the end user to enjoy. While geometry processing has already gone a long way towards that goal, with self-contained solutions commercially available for on-site acquisition of large scale 3D models, capturing the appearance of the materials that compose those models remains an open problem in general uncontrolled environments. The appearance of a material is indeed a complex function of its geometry, intrinsic physical properties and furthermore depends on the illumination conditions in which it is observed, thus traditionally limiting the scope of reflectometry to highly controlled lighting conditions in a laboratory setup. With the rapid development of digital photography, especially on mobile devices, a new trend in the appearance modelling community has emerged, that investigates novel acquisition methods and algorithms to relax the hard constraints imposed by laboratory-like setups, for easy use by digital artists. While arguably not as accurate, we demonstrate the ability of such self-contained methods to enable quick and easy solutions for on-site reflectometry, able to produce compelling, photo-realistic imagery. In particular, this dissertation investigates novel methods for on-site acquisition of surface reflectance based on off-the-shelf, commodity hardware. We successfully demonstrate how a mobile device can be utilised to capture high quality reflectance maps of spatially-varying planar surfaces in general indoor lighting conditions. We further present a novel methodology for the acquisition of highly detailed reflectance maps of permanent on-site, outdoor surfaces by exploiting polarisation from reflection under natural illumination. We demonstrate the versatility of the presented approaches by scanning various surfaces from the real world and show good qualitative and quantitative agreement with existing methods for appearance acquisition employing controlled or semi-controlled illumination setups.Open Acces
ACM Transactions on Graphics, 2017
We present a novel approach for on-site acquisition of surface reflectance for planar, spatially ... more We present a novel approach for on-site acquisition of surface reflectance for planar, spatially varying, isotropic samples in uncontrolled outdoor environments. Our method exploits the naturally occurring linear polarization of incident and reflected illumination for this purpose. By rotating a linear polarizing filter in front of a camera at three different orientations, we measure the polarization reflected off the sample and combine this information with multi-view analysis and inverse rendering in order to recover per-pixel, high resolution reflectance and surface normal maps. Specifically, we employ polarization imaging from two near orthogonal views close to the Brewster angle of incidence in order to maximize polarization cues for surface reflectance estimation. To the best of our knowledge, our method is the first to successfully extract a complete set of reflectance parameters with passive capture in completely uncontrolled outdoor settings. To this end, we analyze our app...
ACM Transactions on Graphics, 2018
We present a method to acquire dynamic properties of facial skin appearance, including dynamic di... more We present a method to acquire dynamic properties of facial skin appearance, including dynamic diffuse albedo encoding blood flow, dynamic specular intensity, and per-frame high resolution normal maps for a facial performance sequence. The method reconstructs these maps from a purely passive multi-camera setup, without the need for polarization or requiring temporally multiplexed illumination. Hence, it is very well suited for integration with existing passive systems for facial performance capture. To solve this seemingly underconstrained problem, we demonstrate that albedo dynamics during a facial performance can be modeled as a combination of: (1) a static, high-resolution base albedo map, modeling full skin pigmentation; and (2) a dynamic, one-dimensional component in the CIE L*a*b* color space, which explains changes in hemoglobin concentration due to blood flow. We leverage this albedo subspace and additional constraints on appearance and surface geometry to also estimate spec...
ACM SIGGRAPH 2014 Posters on - SIGGRAPH '14, 2014
Flash-based reflectometry (b) LCD-based reflectometry (c) Appearance transfer Figure 1: Examples ... more Flash-based reflectometry (b) LCD-based reflectometry (c) Appearance transfer Figure 1: Examples of surface reflectance recovered using mobile reflectometry: (a) A spatially varying rough specular material acquired using our handheld mobile flash-based reflectometry. (b) Highly specular surface reflectance recovered using mobile LCD-based reflectometry, with enhanced mesostructure from close-up observations under natural lighting. (c) Surface reflectance of a large spatial varying material sample recovered using appearance transfer under natural lighting from surface reflectance obtained using the LCD-based approach for a small reference patch.
ACM Transactions on Graphics, 2020
We propose a new light-weight face capture system capable of reconstructing both high-quality geo... more We propose a new light-weight face capture system capable of reconstructing both high-quality geometry and detailed appearance maps from a single exposure. Unlike currently employed appearance acquisition systems, the proposed technology does not require active illumination and hence can readily be integrated with passive photogrammetry solutions. These solutions are in widespread use for 3D scanning humans as they can be assembled from off-the-shelf hardware components, but lack the capability of estimating appearance. This paper proposes a solution to overcome this limitation, by adding appearance capture to photogrammetry systems. The only additional hardware requirement to these solutions is that a subset of the cameras are cross-polarized with respect to the illumination, and the remaining cameras are parallel-polarized. The proposed algorithm leverages the images with the two different polarization states to reconstruct the geometry and to recover appearance properties. We do ...