PTNet: A High-Resolution Infant MRI Synthesizer Based on Transformer (original) (raw)
Related papers
Computers in Biology and Medicine, 2020
Background and objective: One of the main issues in the analysis of clinical neonatal brain MRI is the low anisotropic resolution of the data. In most MRI analysis pipelines, data are first re-sampled using interpolation or single image super-resolution techniques and then segmented using (semi-)automated approaches. In other words, image reconstruction and segmentation are then performed separately. In this article, we propose a methodology and a software solution for carrying out simultaneously high-resolution reconstruction and segmentation of brain MRI data. Methods: Our strategy mainly relies on generative adversarial networks. The network architecture is described in detail. We provide information about its implementation, focusing on the most crucial technical points (whereas complementary details are given in a dedicated GitHub repository). We illustrate the behaviour of the proposed method for cortex analysis from neonatal MR images. Results: The results of the method, evaluated quantitatively (Dice, peak signal-to-noise ratio, structural similarity, number of connected components) and qualitatively on a research dataset (dHCP) and a clinical one (Epirmex), emphasize the relevance of the approach, and its ability to take advantage of data-augmentation strategies. Conclusions: Results emphasize the potential of our proposed method / software with respect to practical medical applications. The method is provided as a freely available software tool, which allows one to carry out his/her own experiments, and involve the method for the super-resolution reconstruction and segmentation of arbitrary cerebral structures from any MR image dataset.
HAL (Le Centre pour la Communication Scientifique Directe), 2019
One of the main issues in the analysis of clinical neonatal brain MRI is the low anisotropic resolution of the data. In most MRI analysis pipelines, data are first re-sampled using interpolation or single image super-resolution techniques and then segmented using (semi-)automated approaches. In other words, image reconstruction and segmentation are then performed separately. In this article, we propose a methodology and a software solution for carrying out simultaneously high-resolution reconstruction and segmentation of brain MRI data. Our strategy mainly relies on generative adversarial networks. The network architecture is described in details, such as the associated software tool. We illustrate its behaviour for cortex analysis from neonatal MR images, both in a quantitative way on a research MRI dataset, and more qualitatively on real clinical data. Results emphasize the potential of our proposed method / software with respect to practical medical applications.
2020
Synthesizing medical images, such as PET, is a challenging task due to the fact that the intensity range is much wider and denser than those in photographs and digital renderings and are often heavily biased toward zero. Above all, intensity values in PET have absolute significance, and are used to compute parameters that are reproducible across the population. Yet, usually much manual adjustment has to be made in pre-/post- processing when synthesizing PET images, because its intensity ranges can vary a lot, e.g., between -100 to 1000 in floating point values. To overcome these challenges, we adopt the Bidirectional Encoder Representations from Transformers (BERT) algorithm that has had great success in natural language processing (NLP), where wide-range floating point intensity values are represented as integers ranging between 0 to 10000 that resemble a dictionary of natural language vocabularies. BERT is then trained to predict a proportion of masked values images, where its &qu...
The role of generative adversarial networks in brain MRI: a scoping review
Insights into Imaging
The performance of artificial intelligence (AI) for brain MRI can improve if enough data are made available. Generative adversarial networks (GANs) showed a lot of potential to generate synthetic MRI data that can capture the distribution of real MRI. Besides, GANs are also popular for segmentation, noise removal, and super-resolution of brain MRI images. This scoping review aims to explore how GANs methods are being used on brain MRI data, as reported in the literature. The review describes the different applications of GANs for brain MRI, presents the most commonly used GANs architectures, and summarizes the publicly available brain MRI datasets for advancing the research and development of GANs-based approaches. This review followed the guidelines of PRISMA-ScR to perform the study search and selection. The search was conducted on five popular scientific databases. The screening and selection of studies were performed by two independent reviewers, followed by validation by a thir...
Longitudinal Prediction of Infant MR Images With Multi-Contrast Perceptual Adversarial Learning
Frontiers in Neuroscience, 2021
The infant brain undergoes a remarkable period of neural development that is crucial for the development of cognitive and behavioral capacities (Hasegawa et al., 2018). Longitudinal magnetic resonance imaging (MRI) is able to characterize the developmental trajectories and is critical in neuroimaging studies of early brain development. However, missing data at different time points is an unavoidable occurrence in longitudinal studies owing to participant attrition and scan failure. Compared to dropping incomplete data, data imputation is considered a better solution to address such missing data in order to preserve all available samples. In this paper, we adapt generative adversarial networks (GAN) to a new application: longitudinal image prediction of structural MRI in the first year of life. In contrast to existing medical image-to-image translation applications of GANs, where inputs and outputs share a very close anatomical structure, our task is more challenging as brain size, s...
2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), 2019
The analysis of clinical neonatal brain MRI remains challenging due to low anisotropic resolution of the data. In most pipelines, images are first re-sampled using interpolation or single image super-resolution techniques and then segmented using (semi-)automated approaches. Image reconstruction and segmentation are then performed separately. In this paper, we propose an end-to-end generative adversarial network for simultaneous high-resolution reconstruction and segmentation of brain MRI data. This joint approach is first assessed on the simulated low-resolution images of the high-resolution neonatal dHCP dataset. Then, the learned model is used to enhance and segment real clinical low-resolution images. Results demonstrate the potential of our proposed method with respect to practical medical applications.
Applications of Deep Learning to Neurodevelopment in Pediatric Imaging: Achievements and Challenges
Applied Sciences
Deep learning has achieved remarkable progress, particularly in neuroimaging analysis. Deep learning applications have also been extended from adult to pediatric medical images, and thus, this paper aims to present a systematic review of this recent research. We first introduce the commonly used deep learning methods and architectures in neuroimaging, such as convolutional neural networks, auto-encoders, and generative adversarial networks. A non-exhaustive list of commonly used publicly available pediatric neuroimaging datasets and repositories are included, followed by a categorical review of recent works in pediatric MRI-based deep learning studies in the past five years. These works are categorized into recognizing neurodevelopmental disorders, identifying brain and tissue structures, estimating brain age/maturity, predicting neurodevelopment outcomes, and optimizing MRI brain imaging and analysis. Finally, we also discuss the recent achievements and challenges on these applicat...
Deep Generative Adversarial Networks for Compressed Sensing Automates MRI
arXiv (Cornell University), 2017
Magnetic resonance image (MRI) reconstruction is a severely ill-posed linear inverse task demanding time and resource intensive computations that can substantially trade off accuracy for speed in real-time imaging. In addition, state-of-the-art compressed sensing (CS) analytics are not cognizant of the image diagnostic quality. To cope with these challenges we put forth a novel CS framework that permeates benefits from generative adversarial networks (GAN) to train a (low-dimensional) manifold of diagnostic-quality MR images from historical patients. Leveraging a mixture of least-squares (LS) GANs and pixel-wise 1 cost, a deep residual network with skip connections is trained as the generator that learns to remove the aliasing artifacts by projecting onto the manifold. LSGAN learns the texture details, while 1 controls the high-frequency noise. A multilayer convolutional neural network is then jointly trained based on diagnostic quality images to discriminate the projection quality. The test phase performs feed-forward propagation over the generator network that demands a very low computational overhead. Extensive evaluations are performed on a large contrast-enhanced MR dataset of pediatric patients. In particular, images rated based on expert radiologists corroborate that GANCS retrieves high contrast images with detailed texture relative to conventional CS, and pixel-wise schemes. In addition, it offers reconstruction under a few milliseconds, two orders of magnitude faster than state-of-the-art CS-MRI schemes. * The authors are with the Stanford University, Departments of Electrical Engineering 1 , Radiology 2 , Radiation Oncology 3 , and Computer Science 4 .
Magnetic Resonance Image Reconstruction using Inception-based Convolutional Neural Network
2023
Magnetic resonance imaging (MRI) is one of the best imaging techniques that produce highquality images of objects. The long scan time is one of the biggest challenges in MRI acquisitions. To address this challenge, many researchers have aimed at finding methods to speed up the process. Faster MRI can reduce patient discomfort and motion artifacts. Many reconstruction methods are used in this matter, like deep learning-based MRI reconstruction, parallel MRI, and compressive sensing. Among these techniques, the convolutional neural network (CNN) generates high-quality images with faster scan and reconstruction procedures compared to the other techniques. The Inception module proposed by Google inspires the algorithm of this study for MRI reconstruction. In other words, we introduce a new MRI U-Net modification by using the Inception module. Our method is more flexible and robust compared to the standard U-Net.
Brain MRI Technics Images Translation by Generative Adversarial Network
One of the most critical problems in medical imaging is having high-quality data on healthy and sick patients. Also, gathering and creating a useful dataset is very time-consuming and is not always cost-effective. Machine learning methods are the newest methods in image processing, especially in medical image processing for classification, segmentation, and translation. GAN (Generative Adversarial Networks) is a class of machine learning frameworks that we consider a solution to image-to-image translation problems and augmentation. One of GAN's applications is generating more realistic data for training and validation to improve the performance of the algorithm and evaluation. In this paper, we propose a high-quality image-to-image translation framework based on CycleGAN in a paired and unpaired model of translation from T1 (or T2) to T2 (or T1) weighted MRI (Magnetic Resonance Imaging) of brain images. For evaluation, we used a dataset that consisted of T1 and T2 images acquire...