Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning - PubMed (original) (raw)
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
Takeru Miyato et al. IEEE Trans Pattern Anal Mach Intell. 2019 Aug.
Abstract
We propose a new regularization method based on virtual adversarial loss: a new measure of local smoothness of the conditional label distribution given input. Virtual adversarial loss is defined as the robustness of the conditional label distribution around each input data point against local perturbation. Unlike adversarial training, our method defines the adversarial direction without label information and is hence applicable to semi-supervised learning. Because the directions in which we smooth the model are only "virtually" adversarial, we call our method virtual adversarial training (VAT). The computational cost of VAT is relatively low. For neural networks, the approximated gradient of virtual adversarial loss can be computed with no more than two pairs of forward- and back-propagations. In our experiments, we applied VAT to supervised and semi-supervised learning tasks on multiple benchmark datasets. With a simple enhancement of the algorithm based on the entropy minimization principle, our VAT achieves state-of-the-art performance for semi-supervised learning tasks on SVHN and CIFAR-10.
Similar articles
- Deep virtual adversarial self-training with consistency regularization for semi-supervised medical image classification.
Wang X, Chen H, Xiang H, Lin H, Lin X, Heng PA. Wang X, et al. Med Image Anal. 2021 May;70:102010. doi: 10.1016/j.media.2021.102010. Epub 2021 Feb 22. Med Image Anal. 2021. PMID: 33677262 - Manifold adversarial training for supervised and semi-supervised learning.
Zhang S, Huang K, Zhu J, Liu Y. Zhang S, et al. Neural Netw. 2021 Aug;140:282-293. doi: 10.1016/j.neunet.2021.03.031. Epub 2021 Mar 26. Neural Netw. 2021. PMID: 33839600 - Generative Adversarial Training for Supervised and Semi-supervised Learning.
Wang X, Li J, Liu Q, Zhao W, Li Z, Wang W. Wang X, et al. Front Neurorobot. 2022 Mar 24;16:859610. doi: 10.3389/fnbot.2022.859610. eCollection 2022. Front Neurorobot. 2022. PMID: 35401139 Free PMC article. - A regularization-driven Mean Teacher model based on semi-supervised learning for medical image segmentation.
Wang Q, Li X, Chen M, Chen L, Chen J. Wang Q, et al. Phys Med Biol. 2022 Aug 30;67(17). doi: 10.1088/1361-6560/ac89c8. Phys Med Biol. 2022. PMID: 35970179 - Robustifying models against adversarial attacks by Langevin dynamics.
Srinivasan V, Rohrer C, Marban A, Müller KR, Samek W, Nakajima S. Srinivasan V, et al. Neural Netw. 2021 May;137:1-17. doi: 10.1016/j.neunet.2020.12.024. Epub 2021 Jan 9. Neural Netw. 2021. PMID: 33515855 Review.
Cited by
- Adversarial training collaborating hybrid convolution-transformer network for automatic identification of reactive lymphocytes in peripheral blood.
Mei L, Peng H, Luo P, Jin S, Shen H, He J, Yang W, Ye Z, Sui H, Mei M, Lei C, Xiong B. Mei L, et al. Biomed Opt Express. 2024 Aug 9;15(9):5143-5161. doi: 10.1364/BOE.525119. eCollection 2024 Sep 1. Biomed Opt Express. 2024. PMID: 39296391 Free PMC article. - Choroidal Optical Coherence Tomography Angiography: Noninvasive Choroidal Vessel Analysis via Deep Learning.
Zhu L, Li J, Hu Y, Zhu R, Zeng S, Rong P, Zhang Y, Gu X, Wang Y, Zhang Z, Yang L, Ren Q, Lu Y. Zhu L, et al. Health Data Sci. 2024 Sep 10;4:0170. doi: 10.34133/hds.0170. eCollection 2024. Health Data Sci. 2024. PMID: 39257642 Free PMC article. - Unsupervised Domain Adaptation for Inter-Session Re-Calibration of Ultrasound-Based HMIs.
Lykourinas A, Rottenberg X, Catthoor F, Skodras A. Lykourinas A, et al. Sensors (Basel). 2024 Aug 4;24(15):5043. doi: 10.3390/s24155043. Sensors (Basel). 2024. PMID: 39124090 Free PMC article. - The Approach to Sensing the True Fetal Heart Rate for CTG Monitoring: An Evaluation of Effectiveness of Deep Learning with Doppler Ultrasound Signals.
Hirono Y, Sato I, Kai C, Yoshida A, Kodama N, Uchida F, Kasai S. Hirono Y, et al. Bioengineering (Basel). 2024 Jun 28;11(7):658. doi: 10.3390/bioengineering11070658. Bioengineering (Basel). 2024. PMID: 39061740 Free PMC article. - Extracting fetal heart signals from Doppler using semi-supervised convolutional neural networks.
Hirono Y, Kai C, Yoshida A, Sato I, Kodama N, Uchida F, Kasai S. Hirono Y, et al. Front Physiol. 2024 Jul 8;15:1293328. doi: 10.3389/fphys.2024.1293328. eCollection 2024. Front Physiol. 2024. PMID: 39040082 Free PMC article.
LinkOut - more resources
Full Text Sources
Other Literature Sources
Miscellaneous