CNN-based Deformable Registration Facilitates Fast and Accurate Air Trapping Measurements at Inspiratory and Expiratory CT - PubMed (original) (raw)

CNN-based Deformable Registration Facilitates Fast and Accurate Air Trapping Measurements at Inspiratory and Expiratory CT

Kyle A Hasenstab et al. Radiol Artif Intell. 2021.

Erratum in

Abstract

Purpose: To develop a convolutional neural network (CNN)-based deformable lung registration algorithm to reduce computation time and assess its potential for lobar air trapping quantification.

Materials and methods: In this retrospective study, a CNN algorithm was developed to perform deformable registration of lung CT (LungReg) using data on 9118 patients from the COPDGene Study (data collected between 2007 and 2012). Loss function constraints included cross-correlation, displacement field regularization, lobar segmentation overlap, and the Jacobian determinant. LungReg was compared with a standard diffeomorphic registration (SyN) for lobar Dice overlap, percentage voxels with nonpositive Jacobian determinants, and inference runtime using paired t tests. Landmark colocalization error (LCE) across 10 patients was compared using a random effects model. Agreement between LungReg and SyN air trapping measurements was assessed using intraclass correlation coefficient. The ability of LungReg versus SyN emphysema and air trapping measurements to predict Global Initiative for Chronic Obstructive Lung Disease (GOLD) stages was compared using area under the receiver operating characteristic curves.

Results: Average performance of LungReg versus SyN showed lobar Dice overlap score of 0.91-0.97 versus 0.89-0.95, respectively (P < .001); percentage voxels with nonpositive Jacobian determinant of 0.04 versus 0.10, respectively (_P_ < .001); inference run time of 0.99 second (graphics processing unit) and 2.27 seconds (central processing unit) versus 418.46 seconds (central processing unit) (_P_ < .001); and LCE of 7.21 mm versus 6.93 mm (_P_ < .001). LungReg and SyN whole-lung and lobar air trapping measurements achieved excellent agreement (intraclass correlation coefficients > 0.98). LungReg versus SyN area under the receiver operating characteristic curves for predicting GOLD stage were not statistically different (range, 0.88-0.95 vs 0.88-0.95, respectively; P = .31-.95).

Conclusion: CNN-based deformable lung registration is accurate and fully automated, with runtime feasible for clinical lobar air trapping quantification, and has potential to improve diagnosis of small airway diseases.Keywords: Air Trapping, Convolutional Neural Network, Deformable Registration, Small Airway Disease, CT, Lung, Semisupervised Learning, Unsupervised Learning Supplemental material is available for this article. © RSNA, 2021 An earlier incorrect version of this article appeared online. This article was corrected on December 22, 2021.

Keywords: Air Trapping; CT; Convolutional Neural Network; Deformable Registration; Lung; Semisupervised Learning; Small Airway Disease; Unsupervised Learning.

2021 by the Radiological Society of North America, Inc.

PubMed Disclaimer

Conflict of interest statement

Disclosures of conflicts of interest: K.A.H. No relevant relationships. J.T. No relevant relationships. N.Y. No relevant relationships. T.R. No relevant relationships. A.H. Grants to institution from GE Healthcare, Bayer, and KA Imaging; co-founder of and shareholder in Arterys.

Figures

Study design. A previously developed three-dimensional lobar segmentation convolutional neural network (CNN) (LungSeg) was applied to 9118 inspiratory and expiratory series pairs from the COPDGene Study, creating segmentations of the trachea and each lung lobe. We then trained our proposed lung deformable registration algorithm (LungReg) to perform expiratory-to-inspiratory registration using the CT images and corresponding lobar segmentations. LungReg was then evaluated across several technical and clinical diagnostic metrics. AUC = area under the receiver operating characteristic curve, COPD = chronic obstructive pulmonary disease, DIR-Lab = Deformable Image Registration Laboratory, GOLD = Global Initiative for Chronic Obstructive Lung Disease.

Figure 1:

Study design. A previously developed three-dimensional lobar segmentation convolutional neural network (CNN) (LungSeg) was applied to 9118 inspiratory and expiratory series pairs from the COPDGene Study, creating segmentations of the trachea and each lung lobe. We then trained our proposed lung deformable registration algorithm (LungReg) to perform expiratory-to-inspiratory registration using the CT images and corresponding lobar segmentations. LungReg was then evaluated across several technical and clinical diagnostic metrics. AUC = area under the receiver operating characteristic curve, COPD = chronic obstructive pulmonary disease, DIR-Lab = Deformable Image Registration Laboratory, GOLD = Global Initiative for Chronic Obstructive Lung Disease.

Flow diagram of loss functions incorporated into the training of the lung deformable image registration algorithm (LungReg). Inspiratory (I) and affine-registered expiratory (E) images are propagated through a three-dimensional (3D) U-Net convolutional neural network (CNN), gw(I,E), to predict a displacement field (u). The spatial transformation is then applied to affine-registered expiratory images using a spatial transformer to deformably register expiratory images to inspiratory images. Four loss function components point to the U-Net because they are used to optimize U-Net weights: cross-correlation for image similarity (ℒCC), displacement regularization for smooth deformations (ℒφ), Dice overlap score for alignment of anatomic structures (ℒseg), and percentage of voxels with nonpositive Jacobian determinants (ℒjac) to encourage transformation invertibility. Note the segmentations are only used during LungReg training and are not required during inference time. Black lines = forward propagation, blue lines = spatial transformations, orange lines = loss functions, ϕ = spatial transformation function.

Figure 2:

Flow diagram of loss functions incorporated into the training of the lung deformable image registration algorithm (LungReg). Inspiratory (I) and affine-registered expiratory (E) images are propagated through a three-dimensional (3D) U-Net convolutional neural network (CNN),gw(I,E), to predict a displacement field (u). The spatial transformation is then applied to affine-registered expiratory images using a spatial transformer to deformably register expiratory images to inspiratory images. Four loss function components point to the U-Net because they are used to optimize U-Net weights: cross-correlation for image similarity (ℒ_CC_), displacement regularization for smooth deformations (ℒφ), Dice overlap score for alignment of anatomic structures (ℒseg), and percentage of voxels with nonpositive Jacobian determinants (ℒjac) to encourage transformation invertibility. Note the segmentations are only used during LungReg training and are not required during inference time. Black lines = forward propagation, blue lines = spatial transformations, orange lines = loss functions, ϕ = spatial transformation function.

Three-dimensional (3D) U-Net convolutional neural network, gw(I,E), used to predict the displacement field defining the deformation. Input comprises a 192 × 192 × 192 × 2 array representing inspiratory (I) and affine-registered expiratory (E) images concatenated along the channel axis. Output is a 192 × 192 × 192 × 3 displacement field. The encoder consists of sequences of 3D convolutions with stride 2 and kernel size 3, each followed by a Leaky rectified linear unit (LeakyReLU) layer with parameter of 0.2. The decoder alternates between convolutions, Leaky rectified linear unit layers, and 3D upsampling.

Figure 3:

Three-dimensional (3D) U-Net convolutional neural network,gw(I,E), used to predict the displacement field defining the deformation. Input comprises a 192 × 192 × 192 × 2 array representing inspiratory (I) and affine-registered expiratory (E) images concatenated along the channel axis. Output is a 192 × 192 × 192 × 3 displacement field. The encoder consists of sequences of 3D convolutions with stride 2 and kernel size 3, each followed by a Leaky rectified linear unit (LeakyReLU) layer with parameter of 0.2. The decoder alternates between convolutions, Leaky rectified linear unit layers, and 3D upsampling.

Dice scores across algorithms and anatomic structures (top row) and paired differences with symmetric diffeomorphic registration (iterative) (SyN) (bottom row). α,β,γ correspond to the algorithm for performing deformable registration of lung CT (LungReg) with the cross-correlation loss, segmentation loss, and Jacobian loss, respectively. Significance and direction of paired differences are indicated by (+) and (−). Affine registration shows significantly lower Dice values across all algorithms and structures (P < .001). LungReg algorithms show a significantly larger Dice value than SyN for all structures, except the right middle lobe (RML) and right upper lobe (RUL), where LungRegα without the use of segmentations during training produced significantly lower Dice values for the RML (P < .001)and was not significantly different for the RUL (P = .718). LungRegα,β and LungRegα,β,γ with training segmentations showed a significant increase in Dice overlap score (P < .001). LL = left lung, LLL = left lower lobe, LUL = left upper lobe, RL = right lung, RLL = right lower lobe, TRA = trachea, Whole = both lungs.

Figure 4:

Dice scores across algorithms and anatomic structures (top row) and paired differences with symmetric diffeomorphic registration (iterative) (SyN) (bottom row). α,β,γ correspond to the algorithm for performing deformable registration of lung CT (LungReg) with the cross-correlation loss, segmentation loss, and Jacobian loss, respectively. Significance and direction of paired differences are indicated by (+) and (−). Affine registration shows significantly lower Dice values across all algorithms and structures (P < .001). LungReg algorithms show a significantly larger Dice value than SyN for all structures, except the right middle lobe (RML) and right upper lobe (RUL), where LungRegα without the use of segmentations during training produced significantly lower Dice values for the RML (P < .001)and was not significantly different for the RUL (P = .718). LungRegα,β and LungRegα,β,γ with training segmentations showed a significant increase in Dice overlap score (P < .001). LL = left lung, LLL = left lower lobe, LUL = left upper lobe, RL = right lung, RLL = right lower lobe, TRA = trachea, Whole = both lungs.

Case example comparing deformed images, segmentations, and displacement fields for the algorithm for performing deformable registration of lung CT (LungRegα,β,γ) and symmetric diffeomorphic registration (iterative) (SyN). Dashed white lines are overlaid inspiratory segmentations. Positive displacements (red) are posterior-to-anterior, left-to-right, and inferior-to-superior. Deformed expiratory images appear similar across LungRegα,β,γ and SyN. However, overlap of lung structures improves, especially for the right middle lobe. Displacement fields suggest similar anatomic transformations between algorithms, but with greater emphasis on the lung boundaries, as evidenced by the lung outline presence visible in each LungRegα,β,γ field.

Figure 5:

Case example comparing deformed images, segmentations, and displacement fields for the algorithm for performing deformable registration of lung CT (LungRegα,β,γ) and symmetric diffeomorphic registration (iterative) (SyN). Dashed white lines are overlaid inspiratory segmentations. Positive displacements (red) are posterior-to-anterior, left-to-right, and inferior-to-superior. Deformed expiratory images appear similar across LungRegα,β,γ and SyN. However, overlap of lung structures improves, especially for the right middle lobe. Displacement fields suggest similar anatomic transformations between algorithms, but with greater emphasis on the lung boundaries, as evidenced by the lung outline presence visible in each LungRegα,β,γ field.

Receiver operating characteristic curves and area under the receiver operating characteristic curves (AUCs) for each lung deformable registration algorithm for predicting Global Initiative for Chronic Obstructive Lung Disease (GOLD) stages with use of percentage emphysema low attenuation area, or %EM-LAA, and percentage air trapping–attenuation difference map, or %AT-ADM, as predictors. Algorithms showed near-identical performance for each respective GOLD stage. Areas under the receiver operating characteristic curves for the algorithm for performing deformable registration of lung CT (LungReg) were not significantly different from those for symmetric diffeomorphic registration (iterative) (SyN), which suggests that LungReg air trapping measurements could supplant SyN air trapping measurements.

Figure 6:

Receiver operating characteristic curves and area under the receiver operating characteristic curves (AUCs) for each lung deformable registration algorithm for predicting Global Initiative for Chronic Obstructive Lung Disease (GOLD) stages with use of percentage emphysema low attenuation area, or %EM-LAA, and percentage air trapping–attenuation difference map, or %AT-ADM, as predictors. Algorithms showed near-identical performance for each respective GOLD stage. Areas under the receiver operating characteristic curves for the algorithm for performing deformable registration of lung CT (LungReg) were not significantly different from those for symmetric diffeomorphic registration (iterative) (SyN), which suggests that LungReg air trapping measurements could supplant SyN air trapping measurements.

References

    1. Lowe KE, Regan EA, Anzueto A, et al. COPDGene® 2019: Redefining the Diagnosis of Chronic Obstructive Pulmonary Disease. Chronic Obstr Pulm Dis (Miami) 2019;6(5):384–399. -PMC -PubMed
    1. Sharifi H, Lai YK, Guo H, et al. Machine Learning Algorithms to Differentiate Among Pulmonary Complications After Hematopoietic Cell Transplant. Chest 2020;158(3):1090–1103. -PMC -PubMed
    1. Rosenow T, Ramsey K, Turkovic L, et al. Air trapping in early cystic fibrosis lung disease—Does CT tell the full story? Pediatr Pulmonol 2017;52(9):1150–1156. -PubMed
    1. Alter P, Orszag J, Kellerer C, et al. Prediction of air trapping or pulmonary hyperinflation by forced spirometry in COPD patients: results from COSYCONET. ERJ Open Res 2020;6(3):00092–2020. -PMC -PubMed
    1. Kligerman SJ, Henry T, Lin CT, Franks TJ, Galvin JR. Mosaic Attenuation: Etiology, Methods of Differentiation, and Pitfalls. RadioGraphics 2015;35(5):1360–1380. -PubMed

LinkOut - more resources