Mastering the game of Go with deep neural networks and tree search - PubMed (original) (raw)
. 2016 Jan 28;529(7587):484-9.
doi: 10.1038/nature16961.
Aja Huang 1, Chris J Maddison 1, Arthur Guez 1, Laurent Sifre 1, George van den Driessche 1, Julian Schrittwieser 1, Ioannis Antonoglou 1, Veda Panneershelvam 1, Marc Lanctot 1, Sander Dieleman 1, Dominik Grewe 1, John Nham 2, Nal Kalchbrenner 1, Ilya Sutskever 2, Timothy Lillicrap 1, Madeleine Leach 1, Koray Kavukcuoglu 1, Thore Graepel 1, Demis Hassabis 1
Affiliations
- PMID: 26819042
- DOI: 10.1038/nature16961
Mastering the game of Go with deep neural networks and tree search
David Silver et al. Nature. 2016.
Abstract
The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses 'value networks' to evaluate board positions and 'policy networks' to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.
Comment in
- Ready or Not, Here We Go: Decision-Making Strategies From Artificial Intelligence Based on Deep Neural Networks.
Dyster T, Sheth SA, McKhann GM 2nd. Dyster T, et al. Neurosurgery. 2016 Jun;78(6):N11-2. doi: 10.1227/01.neu.0000484053.82181.f6. Neurosurgery. 2016. PMID: 27191806 No abstract available. - Train artificial intelligence to be fair to farming.
Lin YP, Petway JR, Settele J. Lin YP, et al. Nature. 2017 Dec 21;552(7685):334. doi: 10.1038/d41586-017-08881-3. Nature. 2017. PMID: 29293217 No abstract available.
Similar articles
- Mastering the game of Go without human knowledge.
Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, Hubert T, Baker L, Lai M, Bolton A, Chen Y, Lillicrap T, Hui F, Sifre L, van den Driessche G, Graepel T, Hassabis D. Silver D, et al. Nature. 2017 Oct 18;550(7676):354-359. doi: 10.1038/nature24270. Nature. 2017. PMID: 29052630 - Google AI algorithm masters ancient game of Go.
Gibney E. Gibney E. Nature. 2016 Jan 28;529(7587):445-6. doi: 10.1038/529445a. Nature. 2016. PMID: 26819021 No abstract available. - A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play.
Silver D, Hubert T, Schrittwieser J, Antonoglou I, Lai M, Guez A, Lanctot M, Sifre L, Kumaran D, Graepel T, Lillicrap T, Simonyan K, Hassabis D. Silver D, et al. Science. 2018 Dec 7;362(6419):1140-1144. doi: 10.1126/science.aar6404. Science. 2018. PMID: 30523106 - [Deep Learning and AlphaGo].
Yoshida H. Yoshida H. Brain Nerve. 2019 Jul;71(7):681-694. doi: 10.11477/mf.1416201340. Brain Nerve. 2019. PMID: 31289242 Review. Japanese. - Recent Advances in General Game Playing.
Świechowski M, Park H, Mańdziuk J, Kim KJ. Świechowski M, et al. ScientificWorldJournal. 2015;2015:986262. doi: 10.1155/2015/986262. Epub 2015 Aug 24. ScientificWorldJournal. 2015. PMID: 26380375 Free PMC article. Review.
Cited by
- Human bias and CNNs' superior insights in satellite based poverty mapping.
Sarmadi H, Wahab I, Hall O, Rögnvaldsson T, Ohlsson M. Sarmadi H, et al. Sci Rep. 2024 Oct 2;14(1):22878. doi: 10.1038/s41598-024-74150-9. Sci Rep. 2024. PMID: 39358399 Free PMC article. - Quantum 2-Player Games and Realizations with Circuits.
Zhang J, Chen T, Deng W, Tong X, Zhang X. Zhang J, et al. Research (Wash D C). 2024 Sep 30;7:0480. doi: 10.34133/research.0480. eCollection 2024. Research (Wash D C). 2024. PMID: 39351071 Free PMC article. - AnyFace++: Deep Multi-Task, Multi-Domain Learning for Efficient Face AI.
Rakhimzhanova T, Kuzdeuov A, Varol HA. Rakhimzhanova T, et al. Sensors (Basel). 2024 Sep 15;24(18):5993. doi: 10.3390/s24185993. Sensors (Basel). 2024. PMID: 39338738 Free PMC article. - Dendrites endow artificial neural networks with accurate, robust and parameter-efficient learning.
Chavlis S, Poirazi P. Chavlis S, et al. ArXiv [Preprint]. 2024 Sep 13:arXiv:2404.03708v2. ArXiv. 2024. PMID: 39314509 Free PMC article. Preprint. - Machine Learning for RNA Design: LEARNA.
Runge F, Hutter F. Runge F, et al. Methods Mol Biol. 2025;2847:63-93. doi: 10.1007/978-1-0716-4079-1_5. Methods Mol Biol. 2025. PMID: 39312137
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources