Konik Kothari | University of Illinois at Urbana-Champaign (original) (raw)

Papers by Konik Kothari

AGU Fall Meeting Abstracts, Dec 1, 2018

arXiv (Cornell University), Jun 10, 2020

We propose a general deep learning architecture for wave-based imaging problems. A key difficulty... more We propose a general deep learning architecture for wave-based imaging problems. A key difficulty in imaging problems with varying background wave speed is that the medium "bends" the waves differently depending on their position and direction. This space-bending geometry makes the equivariance to translations of convolutional networks an undesired inductive bias. We build an interpretable architecture based on wave physics, as captured by the Fourier integral operators (FIOs). FIOs appear in the description of a wide range of wave-based imaging modalities, from seismology and radar to Doppler and ultrasound. Their geometry is characterized by a canonical relation which governs the propagation of singularities. We learn this geometry via optimal transport in the wave packet representation. The proposed FIONet performs significantly better than the usual baselines on a number of inverse problems, especially in out-of-distribution tests. Preprint. Under review.

arXiv (Cornell University), May 29, 2018

We propose a new learning-based approach to solve ill-posed inverse problems in imaging. We addre... more We propose a new learning-based approach to solve ill-posed inverse problems in imaging. We address the case where ground truth training samples are rare and the problem is severely ill-posed-both because of the underlying physics and because we can only get few measurements. This setting is common in geophysical imaging and remote sensing. We show that in this case the common approach to directly learn the mapping from the measured data to the reconstruction becomes unstable. Instead, we propose to first learn an ensemble of simpler mappings from the data to projections of the unknown image into random piecewise-constant subspaces. We then combine the projections to form a final reconstruction by solving a deconvolution-like problem. We show experimentally that the proposed method is more robust to measurement noise and corruptions not seen during training than a directly learned inverse.

International Conference on Learning Representations, 2019

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

We propose a framework to jointly determine the deformation parameters and reconstruct the unknow... more We propose a framework to jointly determine the deformation parameters and reconstruct the unknown volume in electron cryotomography (CryoET). CryoET aims to reconstruct three-dimensional biological samples from two-dimensional projections. A major challenge is that we can only acquire projections for a limited range of tilts, and that each projection undergoes an unknown deformation during acquisition. Not accounting for these deformations results in poor reconstruction. The existing CryoET software packages attempt to align the projections, often in a workflow which uses manual feedback. Our proposed method sidesteps this inconvenience by automatically computing a set of undeformed projections while simultaneously reconstructing the unknown volume. We achieve this by learning a continuous representation of the undeformed measurements and deformation parameters. We show that our approach enables the recovery of high-frequency details that are destroyed without accounting for deformations.

IEEE Transactions on Computational Imaging

Most deep learning models for computational imaging regress a single reconstructed image. In prac... more Most deep learning models for computational imaging regress a single reconstructed image. In practice, however, ill-posedness, nonlinearity, model mismatch, and noise often conspire to make such point estimates misleading or insufficient. The Bayesian approach models images and (noisy) measurements as jointly distributed random vectors and aims to approximate the posterior distribution of unknowns. Recent variational inference methods based on conditional normalizing flows are a promising alternative to traditional MCMC methods, but they come with drawbacks: excessive memory and compute demands for moderate to high resolution images and underwhelming performance on hard nonlinear problems. In this work, we propose C-Trumpetsconditional injective flows specifically designed for imaging problems, which greatly diminish these challenges. Injectivity reduces memory footprint and training time while low-dimensional latent space together with architectural innovations like fixed-volumechange layers and skip-connection revnet layers, C-Trumpets outperform regular conditional flow models on a variety of imaging and image restoration tasks, including limited-view CT and nonlinear inverse scattering, with a lower compute and memory budget. C-Trumpets enable fast approximation of point estimates like MMSE or MAP as well as physically-meaningful uncertainty quantification.

arXiv (Cornell University), Nov 19, 2022

Background: Chromosome analysis is essential for diagnosing genetic disorders and cancer. For pre... more Background: Chromosome analysis is essential for diagnosing genetic disorders and cancer. For precision oncology, identification of somatic clonal aberrations by karyotyping remains the first-line testing and drives therapeutic decisions for leukemia and lymphoma. Clinically, karyotyping plays a unique role in diagnosing global genomic aberrations on a single-cell basis. However, it is time-consuming because of the largely manual process requiring special expertise. Efforts to automate karyotype analysis to date have fallen short in aberration detection. Methods: Using a training set of~10k patient specimens and~50k karyograms from over 5 years (2016-2020) of clinical data, we created a labeled set of images representing individual chromosomes. These individual chromosomes were used to train and assess deep learning models for classifying the 24 human chromosomes and identifying chromosomal aberrations. The top-accuracy models for both chromosome identification and aberration detection task utilized the recently introduced Topological Vision Transformers (TopViTs) with 2-level-block-Toeplitz masking, to incorporate structural inductive bias. To further assess the generalizability of the aberration detection models, we evaluated independently collected datasets from patient specimens tested in 2021-2022. Results: On the baseline task of chromosome identification, our transformer-based models outperformed CNN (Inception) models with >99.3% accuracy. When applied to disease aberration detection, these high-performing architectures exhibited accuracies >99% for most aberrations). Notably, we were able to show high-quality performance even in "few shot" learning scenarios, with limited examples of true aberrations. Incorporating the definition of clonality substantially improved both precision and recall (sensitivity). Interpretation: Karyotype AI can approach expert-level performance for chromosome aberration detection critical for precision oncology. This is the first study demonstrating the ability to accurately detect chromosome aberration by AI and the downstream effectiveness of TopViTs. These results open up exciting opportunities for not only expediting patient results but providing a scalable technology for early screening of low-abundance subclonal lesions.

arXiv (Cornell University), Nov 18, 2022

We propose a differentiable imaging framework to address uncertainty in measurement coordinates s... more We propose a differentiable imaging framework to address uncertainty in measurement coordinates such as sensor locations and projection angles. We formulate the problem as measurement interpolation at unknown nodes supervised through the forward operator. To solve it we apply implicit neural networks, also known as neural fields, which are naturally differentiable with respect to the input coordinates. We also develop differentiable spline interpolators which perform as well as neural networks, require less time to optimize and have well-understood properties. Differentiability is key as it allows us to jointly fit a measurement representation, optimize over the uncertain measurement coordinates, and perform image reconstruction which in turn ensures consistent calibration. We apply our approach to 2D and 3D computed tomography, and show that it produces improved reconstructions compared to baselines that do not account for the lack of calibration. The flexibility of the proposed framework makes it easy to extend to almost arbitrary imaging problems.

arXiv (Cornell University), Apr 15, 2022

Most deep learning models for computational imaging regress a single reconstructed image. In prac... more Most deep learning models for computational imaging regress a single reconstructed image. In practice, however, ill-posedness, nonlinearity, model mismatch, and noise often conspire to make such point estimates misleading or insufficient. The Bayesian approach models images and (noisy) measurements as jointly distributed random vectors and aims to approximate the posterior distribution of unknowns. Recent variational inference methods based on conditional normalizing flows are a promising alternative to traditional MCMC methods, but they come with drawbacks: excessive memory and compute demands for moderate to high resolution images and underwhelming performance on hard nonlinear problems. In this work, we propose C-Trumpets-conditional injective flows specifically designed for imaging problems, which greatly diminish these challenges. Injectivity reduces memory footprint and training time while lowdimensional latent space together with architectural innovations like fixed-volume-change layers and skip-connection revnet layers, C-Trumpets outperform regular conditional flow models on a variety of imaging and image restoration tasks, including limitedview CT and nonlinear inverse scattering, with a lower compute and memory budget. C-Trumpets enable fast approximation of point estimates like MMSE or MAP as well as physicallymeaningful uncertainty quantification.

Seismological Research Letters, Oct 30, 2019

Ill-posed seismic inverse problems are often solved using Tikhonov-type regularization, that is, ... more Ill-posed seismic inverse problems are often solved using Tikhonov-type regularization, that is, incorporation of damping and smoothing to obtain stable results. This typically results in overly smooth models, poor amplitude resolution, and a difficult choice between plausible models. Recognizing that the average of parameters can be better constrained than individual parameters, we propose a seismic tomography method that stabilizes the inverse problem by projecting the original high-dimension model space onto random low-dimension subspaces and then infers the high-dimensional solution from combinations of such subspaces. The subspaces are formed by functions constant in Poisson Voronoi cells, which can be viewed as the mean of parameters near a certain location. The low-dimensional problems are better constrained, and image reconstruction of the subspaces does not require explicit regularization. Moreover, the low-dimension subspaces can be recovered by subsets of the whole dataset, which increases efficiency and offers opportunities to mitigate uneven sampling of the model space. The final (highdimension) model is then obtained from the low-dimension images in different subspaces either by solving another normal equation or simply by averaging the low-dimension images. Importantly, model uncertainty can be obtained directly from images in different subspaces. Synthetic tests show that our method outperforms conventional methods both in terms of geometry and amplitude recovery. The application to southern California plate boundary region also validates the robustness of our method by imaging geologically consistent features as well as strong along-strike variations of San Jacinto fault that are not clearly seen using conventional methods.

Full paper: https://www.auai.org/uai2021/pdf/uai2021.491.pdf Abstract: We propose injective gener... more Full paper: https://www.auai.org/uai2021/pdf/uai2021.491.pdf Abstract: We propose injective generative models called Trumpets that generalize invertible normalizing flows. The proposed generators progressively increase dimension from a low-dimensional latent space. We demonstrate that Trumpets can be trained orders of magnitudes faster than standard flows while yielding samples of comparable or better quality. They retain many of the advantages of the standard flows such as training based on maximum likelihood and a fast, exact inverse of the generator. Since Trumpets are injective and have fast inverses, they can be effectively used for downstream Bayesian inference. To wit, we use Trumpet priors for maximum a posteriori estimation in the context of image reconstruction from compressive measurements, outperforming competitive baselines in terms of reconstruction quality and speed. We then propose an efficient method for posterior characterization and uncertainty quantification with...

ArXiv, 2020

We study injective ReLU neural networks. Injectivity plays an important role in generative models... more We study injective ReLU neural networks. Injectivity plays an important role in generative models where it facilitates inference; in inverse problems with generative priors it is a precursor to well posedness. We establish sharp conditions for injectivity of ReLU layers and networks, both fully connected and convolutional. We make no architectural assumptions beyond the ReLU activations so our results apply to a very general class of neural networks. We show through a layer-wise analysis that an expansivity factor of two is necessary for injectivity; we also show sufficiency by constructing weight matrices which guarantee injectivity. Further, we show that global injectivity with iid Gaussian matrices, a commonly used tractable model, requires considerably larger expansivity which might seem counterintuitive. We then derive the inverse Lipschitz constants and study the approximation-theoretic properties of injective neural networks. Using arguments from differential topology we prov...

Journal of Applied Mechanics, 2018

The skeleton of many natural and artificial soft materials can be abstracted as networks of fiber... more The skeleton of many natural and artificial soft materials can be abstracted as networks of fibers/polymers interacting in a nonlinear fashion. Here, we present a numerical model for networks of nonlinear, elastic polymer chains with rate-dependent crosslinkers similar to what is found in gels. The model combines the worm-like chain (WLC) at the polymer level with the transition state theory for crosslinker bond dynamics. We study the damage evolution and the force—displacement response of these networks under uniaxial stretching for different loading rates, network topology, and crosslinking density. Our results suggest a complex nonmonotonic response as the loading rate or the crosslinking density increases. We discuss this in terms of the microscopic deformation mechanisms and suggest a novel framework for increasing toughness and ductility of polymer networks using a bio-inspired sacrificial bonds and hidden length (SBHL) mechanism. This work highlights the role of local network...

Bulletin of the American Physical Society, 2018

The skeleton of many natural and artificial soft materials can be abstracted as networks of fiber... more The skeleton of many natural and artificial soft materials can be abstracted as networks of fibers/ polymers interacting in a non-linear fashion. Here, we present a numerical model for networks of nonlinear, elastic polymer chains with ratedependent crosslinkers similar to what is found in gels. The model combines the worm-like chain at the polymer level with the transition state theory for crosslinker bond dynamics. We study the damage evolution and the force displacement response of these networks under uniaxial stretching for different loading rates, network topology, and crosslinking density. Our results suggest a complex non-monotonic response as the loading rate or the crosslinking density increases. We discuss this in terms of the microscopic deformation mechanisms and suggest a novel framework for increasing toughness and ductility of polymer networks using a bio-inspired Sacrificial Bonds and Hidden Length (SBHL) mechanism. This work highlights the role of local network cha...

ArXiv, 2018

We develop a new learning-based approach to ill-posed inverse problems. Instead of directly learn... more We develop a new learning-based approach to ill-posed inverse problems. Instead of directly learning the complex mapping from the measured data to the reconstruction, we learn an ensemble of simpler mappings from data to projections of the unknown model into random low-dimensional subspaces. We form the reconstruction by combining the estimated subspace projections. Structured subspaces of piecewise-constant images on random Delaunay triangulations allow us to address inverse problems with extremely sparse data and still get good reconstructions of the unknown geometry. This choice also makes our method robust against arbitrary data corruptions not seen during training. Further, it marginalizes the role of the training dataset which is essential for applications in geophysics where ground-truth datasets are exceptionally scarce.

We propose a new learning-based approach to solve ill-posed inverse problems in imaging. We addre... more We propose a new learning-based approach to solve ill-posed inverse problems in imaging. We address the case where ground truth training samples are rare and the problem is severely ill-posed - both because of the underlying physics and because we can only get few measurements. This setting is common in geophysical imaging and remote sensing. We show that in this case the common approach to directly learn the mapping from the measured data to the reconstruction becomes unstable. Instead, we propose to first learn an ensemble of simpler mappings from the data to projections of the unknown image into random piecewise-constant subspaces. We then combine the projections to form a final reconstruction by solving a deconvolution-like problem. We show experimentally that the proposed method is more robust to measurement noise and corruptions not seen during training than a directly learned inverse.

We propose injective generative models called TRUMPETs that generalize invertible normalizing flo... more We propose injective generative models called TRUMPETs that generalize invertible normalizing flows. The proposed generators progressively increase dimension from a low-dimensional latent space. We demonstrate that TRUMPETs can be trained orders of magnitudes faster than standard flows while yielding samples of comparable or better quality. They retain many of the advantages of the standard flows such as training based on maximum likelihood and a fast, exact inverse of the generator. Since TRUMPETs are injective and have fast inverses, they can be effectively used for downstream Bayesian inference. To wit, we use TRUMPET priors for maximum a posteriori estimation in the context of image reconstruction from compressive measurements, outperforming competitive baselines in terms of reconstruction quality and speed. We then propose an efficient method for posterior characterization and uncertainty quantification with TRUMPETs by taking advantage of the low-dimensional latent space. Our ...

AGU Fall Meeting Abstracts, Dec 1, 2018

arXiv (Cornell University), Jun 10, 2020

arXiv (Cornell University), May 29, 2018

International Conference on Learning Representations, 2019

ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

IEEE Transactions on Computational Imaging

arXiv (Cornell University), Nov 19, 2022

arXiv (Cornell University), Nov 18, 2022

arXiv (Cornell University), Apr 15, 2022

Most deep learning models for computational imaging regress a single reconstructed image. In prac... more Most deep learning models for computational imaging regress a single reconstructed image. In practice, however, ill-posedness, nonlinearity, model mismatch, and noise often conspire to make such point estimates misleading or insufficient. The Bayesian approach models images and (noisy) measurements as jointly distributed random vectors and aims to approximate the posterior distribution of unknowns. Recent variational inference methods based on conditional normalizing flows are a promising alternative to traditional MCMC methods, but they come with drawbacks: excessive memory and compute demands for moderate to high resolution images and underwhelming performance on hard nonlinear problems. In this work, we propose C-Trumpets-conditional injective flows specifically designed for imaging problems, which greatly diminish these challenges. Injectivity reduces memory footprint and training time while lowdimensional latent space together with architectural innovations like fixed-volume-change layers and skip-connection revnet layers, C-Trumpets outperform regular conditional flow models on a variety of imaging and image restoration tasks, including limitedview CT and nonlinear inverse scattering, with a lower compute and memory budget. C-Trumpets enable fast approximation of point estimates like MMSE or MAP as well as physicallymeaningful uncertainty quantification.

Seismological Research Letters, Oct 30, 2019

ArXiv, 2020

Journal of Applied Mechanics, 2018

Bulletin of the American Physical Society, 2018

The skeleton of many natural and artificial soft materials can be abstracted as networks of fiber... more The skeleton of many natural and artificial soft materials can be abstracted as networks of fibers/ polymers interacting in a non-linear fashion. Here, we present a numerical model for networks of nonlinear, elastic polymer chains with ratedependent crosslinkers similar to what is found in gels. The model combines the worm-like chain at the polymer level with the transition state theory for crosslinker bond dynamics. We study the damage evolution and the force displacement response of these networks under uniaxial stretching for different loading rates, network topology, and crosslinking density. Our results suggest a complex non-monotonic response as the loading rate or the crosslinking density increases. We discuss this in terms of the microscopic deformation mechanisms and suggest a novel framework for increasing toughness and ductility of polymer networks using a bio-inspired Sacrificial Bonds and Hidden Length (SBHL) mechanism. This work highlights the role of local network cha...

ArXiv, 2018

We propose a new learning-based approach to solve ill-posed inverse problems in imaging. We addre... more We propose a new learning-based approach to solve ill-posed inverse problems in imaging. We address the case where ground truth training samples are rare and the problem is severely ill-posed - both because of the underlying physics and because we can only get few measurements. This setting is common in geophysical imaging and remote sensing. We show that in this case the common approach to directly learn the mapping from the measured data to the reconstruction becomes unstable. Instead, we propose to first learn an ensemble of simpler mappings from the data to projections of the unknown image into random piecewise-constant subspaces. We then combine the projections to form a final reconstruction by solving a deconvolution-like problem. We show experimentally that the proposed method is more robust to measurement noise and corruptions not seen during training than a directly learned inverse.