yunzhe li | Columbia University (original) (raw)

Papers by yunzhe li

Research paper thumbnail of Plasmonic Directional Photodetectors for Edge Enhancement

Conference on Lasers and Electro-Optics

Angle-sensitive plasmonic photodetectors that can perform optical-domain spatial filtering operat... more Angle-sensitive plasmonic photodetectors that can perform optical-domain spatial filtering operations are developed. The edge enhancement capabilities of these devices are demonstrated via computational imaging simulations based on their measured angular response.

Research paper thumbnail of A one-for-all deep learning approach for imaging through diffusers (Conference Presentation)

Adaptive Optics and Wavefront Control for Biological Systems V

Research paper thumbnail of Towards reliable deep learning based phase microscopy (Conference Presentation)

Research paper thumbnail of Scalable and reliable deep learning for computational microscopy in complex media

Research paper thumbnail of Reliable deep-learning-based phase imaging with uncertainty quantification: erratum

Emerging deep-learning (DL)-based techniques have significant potential to revolutionize biomedic... more Emerging deep-learning (DL)-based techniques have significant potential to revolutionize biomedical imaging. However, one outstanding challenge is the lack of reliability assessment in the DL predictions, whose errors are commonly revealed only in hindsight. Here, we propose a new Bayesian convolutional neural network (BNN)-based framework that overcomes this issue by quantifying the uncertainty of DL predictions. Foremost, we show that BNN-predicted uncertainty maps provide surrogate estimates of the true error from the network model and measurement itself. The uncertainty maps characterize imperfections often unknown in real-world applications, such as noise, model error, incomplete training data, and out-of-distribution testing data. Quantifying this uncertainty provides a per-pixel estimate of the confidence level of the DL prediction as well as the quality of the model and data set. We demonstrate this framework in the application of large space–bandwidth product phase imaging ...

Research paper thumbnail of Coherent imaging through scatter using an interpretable deep neural network

Frontiers in Optics, 2020

We propose a deep neural network model that is agnostic to scatter displacements, and system defo... more We propose a deep neural network model that is agnostic to scatter displacements, and system defocus up to 10X depth of field. We develop an analysis framework for interpreting the mechanism of our DNN model.

Research paper thumbnail of Convolutional neural network for Fourier ptychography video reconstruction: learning temporal dynamics from spatial ensembles

ArXiv, 2018

Convolutional neural networks (CNNs) have gained tremendous success in solving complex inverse pr... more Convolutional neural networks (CNNs) have gained tremendous success in solving complex inverse problems for both problems involving independent datasets from input-output pairs of static objects, as well as sequential datasets from dynamic objects. In order to learn the underlying temporal statistics, a video sequence is typically used at the cost of network complexity and computation. The aim of this work is to develop a novel CNN framework to reconstruct video sequence of dynamic live cells captured using a computational microscopy technique, Fourier ptychographic microscopy (FPM). The unique feature of the FPM is its capability to reconstruct images with both wide field-of-view (FOV) and high resolution, i.e. a large space-bandwidth-product (SBP), by taking a series of low resolution intensity images. For live cell imaging, a single FPM frame contains thousands of cell samples with different morphological features. Our idea is to fully exploit the statistical information provided...

Research paper thumbnail of Imaging through diffusers with extended depth-of-field using a deep neural network

Research paper thumbnail of Deep learning approach to Fourier ptychographic microscopy

arXiv: Computer Vision and Pattern Recognition, 2018

We would like to thank NVIDIA Corporation for supporting us with the GeForce Titan Xp through the... more We would like to thank NVIDIA Corporation for supporting us with the GeForce Titan Xp through the GPU Grant Program. (NVIDIA Corporation; GeForce Titan Xp through the GPU Grant Program)

Research paper thumbnail of Deep learning approach to scalable imaging through scattering media

Imaging and Applied Optics 2019 (COSI, IS, MATH, pcAOP), 2019

We propose a deep learning technique to exploit “deep speckle correlations”. Our work paves the w... more We propose a deep learning technique to exploit “deep speckle correlations”. Our work paves the way to a highly scalable deep learning approach for imaging through scattering media.

Research paper thumbnail of 3D resolution-enhanced intensity diffraction tomographic microscopy

We propose label-free and motion-free resolution-enhanced intensity diffraction tomography (reIDT... more We propose label-free and motion-free resolution-enhanced intensity diffraction tomography (reIDT) recovering the 3D complex refractive index distribution of an object. By combining an annular illumination strategy with a high numerical aperture (NA) condenser, we achieve near diffraction-limited lateral resolution of 346 nm and axial resolution of 1.2 μm over 130 × 130 × 8 μm3 volume. Our annular pattern matches the system’s maximum NA to reduce the data requirement to 48 intensity frames. The reIDT system is directly built on a standard commercial microscope with a simple LED array source and condenser lens adds-on, and promises broad applications for natural biological imaging with minimal hardware modifications. To test the capabilities of our technique, we present the 3D complex refractive index reconstructions on an absorptive USAF resolution target. Our work provides an important step in intensity-based diffraction tomography towards high resolution imaging applications.

Research paper thumbnail of A deep learning approach to high space-bandwidth product phase microscopy with coded illumination (Conference Presentation)

We investigate quantitative phase imaging techniques based on oblique illumination including diff... more We investigate quantitative phase imaging techniques based on oblique illumination including differential phase contrast microscopy (DPC) and Fourier Ptychography Microscopy (FPM). DPC uses partially coherent, asymmetric illumination to achieve 2X resolution improvement but has small field of view (FOV). FPM achieves both wide FOV and high resolution but requires a large number of measurements. Achieving high space-bandwidth product (SBP) imaging in real-time remains challenging. Our goal is to develop a data-driven approach to enable highly multiplexed illumination to substantially improve the acquisition speed for high-SBP quantitative phase imaging. To do so, we abandon the traditional sampling strategy and phase retrieval algorithms. Instead we design a convolutional neural network (CNN) that uses only 4 brightfield and 3 darkfield images under asymmetrically coded illuminations as input and predicts high-SBP phase images. Particularly, instead of restoring a deterministic image...

Research paper thumbnail of Deep-Learning-Based Computational Biomedical Microscopy with Uncertainty Quantification

I will present several deep learning based computational microscopy techniques including phase mi... more I will present several deep learning based computational microscopy techniques including phase microscopy and imaging oximetry. Emphasis will be put on an uncertainty quantification framework for assessing the reliability of these techniques.

Research paper thumbnail of A deep-learning approach for high-speed Fourier ptychographic microscopy

Imaging and Applied Optics 2018 (3D, AO, AIO, COSI, DH, IS, LACSEA, LS&C, MATH, pcAOP)

Nehmetallah. 2018. "A deep-learning approach for high-speed Fourier ptychographic microscopy." Im... more Nehmetallah. 2018. "A deep-learning approach for high-speed Fourier ptychographic microscopy." Imaging Systems and Applications. Imaging Systems and Applications.

Research paper thumbnail of Interpretable deep learning for imaging through scattering medium

Emerging Topics in Artificial Intelligence (ETAI) 2021

Imaging through scattering medium has wide applications across many areas. Here, we present a new... more Imaging through scattering medium has wide applications across many areas. Here, we present a new deep learning framework for improving the robustness against physical perturbations of the scattering medium. The trained DNN can make high-quality predictions beyond the training range which is across 10X depth-of-field (DOF). We develop a new analysis framework based on dimensionality reduction for revealing the information contained in the speckle dataset, interpreting the mechanism of our DNN, and visualizing the generalizability of the DNN model. This allows us to further elucidate on the information encoded in both the raw speckle measurements and the working principle of our speckle-imaging deep learning model.

Research paper thumbnail of Multiplexed cytometry by deep virtual staining from multi-contrast microscopy

Emerging Topics in Artificial Intelligence (ETAI) 2021

Research paper thumbnail of Resolution-enhanced intensity diffraction tomography in high numerical aperture label-free microscopy

Photonics Research

We propose label-free and motion-free resolution-enhanced intensity diffraction tomography (reIDT... more We propose label-free and motion-free resolution-enhanced intensity diffraction tomography (reIDT) recovering the 3D complex refractive index distribution of an object. By combining an annular illumination strategy with a high numerical aperture (NA) condenser, we achieve near-diffraction-limited lateral resolution of 346 nm and axial resolution of 1.2 μm over 130 μm×130 μm×8 μm volume. Our annular pattern matches the system’s maximum NA to reduce the data requirement to 48 intensity frames. The reIDT system is directly built on a standard commercial microscope with a simple LED array source and condenser lens adds-on, and promises broad applications for natural biological imaging with minimal hardware modifications. To test the capabilities of our technique, we present the 3D complex refractive index reconstructions on an absorptive USAF resolution target and Henrietta Lacks (HeLa) and HT29 human cancer cells. Our work provides an important step in intensity-based diffraction tomography toward high-resolution imaging applications.

Research paper thumbnail of Low-cost diffuser-based computational funduscope

Optics and Biophotonics in Low-Resource Settings VII

Diffuser-based sensing has shown potentials in inexpensive and compact optical systems. Here we d... more Diffuser-based sensing has shown potentials in inexpensive and compact optical systems. Here we demonstrate a low-cost diffuser-based computational funduscope that can recover pathological features of the model eye fundus. Our system implements an infinite-conjugate design by relaying the ocular lens onto the diffuser which provides shift-invariance across a wide field-of-view (FOV). Our experiments show that fundus images can be reconstructed over 33 degree FOV and our device is robust to 4D refractive error using a single point-spread-function.

Research paper thumbnail of Plasmonic Computational Compound-Eye Camera

Optics and Photonics News

Recently, we have reported a novel compound-eye camera architecture—one that leverages the great ... more Recently, we have reported a novel compound-eye camera architecture—one that leverages the great design flexibility of metasurface nanophotonics and the advanced data-processing capabilities of computational imaging, to provide an ultrawide FOV greater than 150° in a planar lensless format.

Research paper thumbnail of Diffuser-based computational imaging funduscope

Optics Express

Poor access to eye care is a major global challenge that could be ameliorated by low-cost, portab... more Poor access to eye care is a major global challenge that could be ameliorated by low-cost, portable, and easy-to-use diagnostic technologies. Diffuser-based imaging has the potential to enable inexpensive, compact optical systems that can reconstruct a focused image of an object over a range of defocus errors. Here, we present a diffuser-based computational funduscope that reconstructs important clinical features of a model eye. Compared to existing diffuser-imager architectures, our system features an infinite-conjugate design by relaying the ocular lens onto the diffuser. This offers shift-invariance across a wide field-of-view (FOV) and an invariant magnification across an extended depth range. Experimentally, we demonstrate fundus image reconstruction over a 33° FOV and robustness to ±4D refractive error using a constant point-spread-function. Combined with diffuser-based wavefront sensing, this technology could enable combined ocular aberrometry and funduscopic screening through a single diffuser sensor.

Research paper thumbnail of Plasmonic Directional Photodetectors for Edge Enhancement

Conference on Lasers and Electro-Optics

Angle-sensitive plasmonic photodetectors that can perform optical-domain spatial filtering operat... more Angle-sensitive plasmonic photodetectors that can perform optical-domain spatial filtering operations are developed. The edge enhancement capabilities of these devices are demonstrated via computational imaging simulations based on their measured angular response.

Research paper thumbnail of A one-for-all deep learning approach for imaging through diffusers (Conference Presentation)

Adaptive Optics and Wavefront Control for Biological Systems V

Research paper thumbnail of Towards reliable deep learning based phase microscopy (Conference Presentation)

Research paper thumbnail of Scalable and reliable deep learning for computational microscopy in complex media

Research paper thumbnail of Reliable deep-learning-based phase imaging with uncertainty quantification: erratum

Emerging deep-learning (DL)-based techniques have significant potential to revolutionize biomedic... more Emerging deep-learning (DL)-based techniques have significant potential to revolutionize biomedical imaging. However, one outstanding challenge is the lack of reliability assessment in the DL predictions, whose errors are commonly revealed only in hindsight. Here, we propose a new Bayesian convolutional neural network (BNN)-based framework that overcomes this issue by quantifying the uncertainty of DL predictions. Foremost, we show that BNN-predicted uncertainty maps provide surrogate estimates of the true error from the network model and measurement itself. The uncertainty maps characterize imperfections often unknown in real-world applications, such as noise, model error, incomplete training data, and out-of-distribution testing data. Quantifying this uncertainty provides a per-pixel estimate of the confidence level of the DL prediction as well as the quality of the model and data set. We demonstrate this framework in the application of large space–bandwidth product phase imaging ...

Research paper thumbnail of Coherent imaging through scatter using an interpretable deep neural network

Frontiers in Optics, 2020

We propose a deep neural network model that is agnostic to scatter displacements, and system defo... more We propose a deep neural network model that is agnostic to scatter displacements, and system defocus up to 10X depth of field. We develop an analysis framework for interpreting the mechanism of our DNN model.

Research paper thumbnail of Convolutional neural network for Fourier ptychography video reconstruction: learning temporal dynamics from spatial ensembles

ArXiv, 2018

Convolutional neural networks (CNNs) have gained tremendous success in solving complex inverse pr... more Convolutional neural networks (CNNs) have gained tremendous success in solving complex inverse problems for both problems involving independent datasets from input-output pairs of static objects, as well as sequential datasets from dynamic objects. In order to learn the underlying temporal statistics, a video sequence is typically used at the cost of network complexity and computation. The aim of this work is to develop a novel CNN framework to reconstruct video sequence of dynamic live cells captured using a computational microscopy technique, Fourier ptychographic microscopy (FPM). The unique feature of the FPM is its capability to reconstruct images with both wide field-of-view (FOV) and high resolution, i.e. a large space-bandwidth-product (SBP), by taking a series of low resolution intensity images. For live cell imaging, a single FPM frame contains thousands of cell samples with different morphological features. Our idea is to fully exploit the statistical information provided...

Research paper thumbnail of Imaging through diffusers with extended depth-of-field using a deep neural network

Research paper thumbnail of Deep learning approach to Fourier ptychographic microscopy

arXiv: Computer Vision and Pattern Recognition, 2018

We would like to thank NVIDIA Corporation for supporting us with the GeForce Titan Xp through the... more We would like to thank NVIDIA Corporation for supporting us with the GeForce Titan Xp through the GPU Grant Program. (NVIDIA Corporation; GeForce Titan Xp through the GPU Grant Program)

Research paper thumbnail of Deep learning approach to scalable imaging through scattering media

Imaging and Applied Optics 2019 (COSI, IS, MATH, pcAOP), 2019

We propose a deep learning technique to exploit “deep speckle correlations”. Our work paves the w... more We propose a deep learning technique to exploit “deep speckle correlations”. Our work paves the way to a highly scalable deep learning approach for imaging through scattering media.

Research paper thumbnail of 3D resolution-enhanced intensity diffraction tomographic microscopy

We propose label-free and motion-free resolution-enhanced intensity diffraction tomography (reIDT... more We propose label-free and motion-free resolution-enhanced intensity diffraction tomography (reIDT) recovering the 3D complex refractive index distribution of an object. By combining an annular illumination strategy with a high numerical aperture (NA) condenser, we achieve near diffraction-limited lateral resolution of 346 nm and axial resolution of 1.2 μm over 130 × 130 × 8 μm3 volume. Our annular pattern matches the system’s maximum NA to reduce the data requirement to 48 intensity frames. The reIDT system is directly built on a standard commercial microscope with a simple LED array source and condenser lens adds-on, and promises broad applications for natural biological imaging with minimal hardware modifications. To test the capabilities of our technique, we present the 3D complex refractive index reconstructions on an absorptive USAF resolution target. Our work provides an important step in intensity-based diffraction tomography towards high resolution imaging applications.

Research paper thumbnail of A deep learning approach to high space-bandwidth product phase microscopy with coded illumination (Conference Presentation)

We investigate quantitative phase imaging techniques based on oblique illumination including diff... more We investigate quantitative phase imaging techniques based on oblique illumination including differential phase contrast microscopy (DPC) and Fourier Ptychography Microscopy (FPM). DPC uses partially coherent, asymmetric illumination to achieve 2X resolution improvement but has small field of view (FOV). FPM achieves both wide FOV and high resolution but requires a large number of measurements. Achieving high space-bandwidth product (SBP) imaging in real-time remains challenging. Our goal is to develop a data-driven approach to enable highly multiplexed illumination to substantially improve the acquisition speed for high-SBP quantitative phase imaging. To do so, we abandon the traditional sampling strategy and phase retrieval algorithms. Instead we design a convolutional neural network (CNN) that uses only 4 brightfield and 3 darkfield images under asymmetrically coded illuminations as input and predicts high-SBP phase images. Particularly, instead of restoring a deterministic image...

Research paper thumbnail of Deep-Learning-Based Computational Biomedical Microscopy with Uncertainty Quantification

I will present several deep learning based computational microscopy techniques including phase mi... more I will present several deep learning based computational microscopy techniques including phase microscopy and imaging oximetry. Emphasis will be put on an uncertainty quantification framework for assessing the reliability of these techniques.

Research paper thumbnail of A deep-learning approach for high-speed Fourier ptychographic microscopy

Imaging and Applied Optics 2018 (3D, AO, AIO, COSI, DH, IS, LACSEA, LS&C, MATH, pcAOP)

Nehmetallah. 2018. "A deep-learning approach for high-speed Fourier ptychographic microscopy." Im... more Nehmetallah. 2018. "A deep-learning approach for high-speed Fourier ptychographic microscopy." Imaging Systems and Applications. Imaging Systems and Applications.

Research paper thumbnail of Interpretable deep learning for imaging through scattering medium

Emerging Topics in Artificial Intelligence (ETAI) 2021

Imaging through scattering medium has wide applications across many areas. Here, we present a new... more Imaging through scattering medium has wide applications across many areas. Here, we present a new deep learning framework for improving the robustness against physical perturbations of the scattering medium. The trained DNN can make high-quality predictions beyond the training range which is across 10X depth-of-field (DOF). We develop a new analysis framework based on dimensionality reduction for revealing the information contained in the speckle dataset, interpreting the mechanism of our DNN, and visualizing the generalizability of the DNN model. This allows us to further elucidate on the information encoded in both the raw speckle measurements and the working principle of our speckle-imaging deep learning model.

Research paper thumbnail of Multiplexed cytometry by deep virtual staining from multi-contrast microscopy

Emerging Topics in Artificial Intelligence (ETAI) 2021

Research paper thumbnail of Resolution-enhanced intensity diffraction tomography in high numerical aperture label-free microscopy

Photonics Research

We propose label-free and motion-free resolution-enhanced intensity diffraction tomography (reIDT... more We propose label-free and motion-free resolution-enhanced intensity diffraction tomography (reIDT) recovering the 3D complex refractive index distribution of an object. By combining an annular illumination strategy with a high numerical aperture (NA) condenser, we achieve near-diffraction-limited lateral resolution of 346 nm and axial resolution of 1.2 μm over 130 μm×130 μm×8 μm volume. Our annular pattern matches the system’s maximum NA to reduce the data requirement to 48 intensity frames. The reIDT system is directly built on a standard commercial microscope with a simple LED array source and condenser lens adds-on, and promises broad applications for natural biological imaging with minimal hardware modifications. To test the capabilities of our technique, we present the 3D complex refractive index reconstructions on an absorptive USAF resolution target and Henrietta Lacks (HeLa) and HT29 human cancer cells. Our work provides an important step in intensity-based diffraction tomography toward high-resolution imaging applications.

Research paper thumbnail of Low-cost diffuser-based computational funduscope

Optics and Biophotonics in Low-Resource Settings VII

Diffuser-based sensing has shown potentials in inexpensive and compact optical systems. Here we d... more Diffuser-based sensing has shown potentials in inexpensive and compact optical systems. Here we demonstrate a low-cost diffuser-based computational funduscope that can recover pathological features of the model eye fundus. Our system implements an infinite-conjugate design by relaying the ocular lens onto the diffuser which provides shift-invariance across a wide field-of-view (FOV). Our experiments show that fundus images can be reconstructed over 33 degree FOV and our device is robust to 4D refractive error using a single point-spread-function.

Research paper thumbnail of Plasmonic Computational Compound-Eye Camera

Optics and Photonics News

Recently, we have reported a novel compound-eye camera architecture—one that leverages the great ... more Recently, we have reported a novel compound-eye camera architecture—one that leverages the great design flexibility of metasurface nanophotonics and the advanced data-processing capabilities of computational imaging, to provide an ultrawide FOV greater than 150° in a planar lensless format.

Research paper thumbnail of Diffuser-based computational imaging funduscope

Optics Express

Poor access to eye care is a major global challenge that could be ameliorated by low-cost, portab... more Poor access to eye care is a major global challenge that could be ameliorated by low-cost, portable, and easy-to-use diagnostic technologies. Diffuser-based imaging has the potential to enable inexpensive, compact optical systems that can reconstruct a focused image of an object over a range of defocus errors. Here, we present a diffuser-based computational funduscope that reconstructs important clinical features of a model eye. Compared to existing diffuser-imager architectures, our system features an infinite-conjugate design by relaying the ocular lens onto the diffuser. This offers shift-invariance across a wide field-of-view (FOV) and an invariant magnification across an extended depth range. Experimentally, we demonstrate fundus image reconstruction over a 33° FOV and robustness to ±4D refractive error using a constant point-spread-function. Combined with diffuser-based wavefront sensing, this technology could enable combined ocular aberrometry and funduscopic screening through a single diffuser sensor.