Training and operation of an integrated neuromorphic network based on metal-oxide memristors (original) (raw)

Nature volume 521, pages 61–64 (2015)Cite this article

Subjects

Abstract

Despite much progress in semiconductor integrated circuit technology, the extreme complexity of the human cerebral cortex1, with its approximately 1014 synapses, makes the hardware implementation of neuromorphic networks with a comparable number of devices exceptionally challenging. To provide comparable complexity while operating much faster and with manageable power dissipation, networks2 based on circuits3,4 combining complementary metal-oxide-semiconductors (CMOSs) and adjustable two-terminal resistive devices (memristors) have been developed. In such circuits, the usual CMOS stack is augmented with one3 or several4 crossbar layers, with memristors at each crosspoint. There have recently been notable improvements in the fabrication of such memristive crossbars and their integration with CMOS circuits5,6,7,8,9,10,11,12, including first demonstrations5,6,12 of their vertical integration. Separately, discrete memristors have been used as artificial synapses in neuromorphic networks13,14,15,16,17,18. Very recently, such experiments have been extended19 to crossbar arrays of phase-change memristive devices. The adjustment of such devices, however, requires an additional transistor at each crosspoint, and hence these devices are much harder to scale than metal-oxide memristors11,20,21, whose nonlinear current–voltage curves enable transistor-free operation. Here we report the experimental implementation of transistor-free metal-oxide memristor crossbars, with device variability sufficiently low to allow operation of integrated neural networks, in a simple network: a single-layer perceptron (an algorithm for linear classification). The network can be taught in situ using a coarse-grain variety of the delta rule algorithm22 to perform the perfect classification of 3 × 3-pixel black/white images into three classes (representing letters). This demonstration is an important step towards much larger and more complex memristive neuromorphic networks.

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 51 print issues and online access

$199.00 per year

only $3.90 per issue

Buy this article

USD 39.95

Prices may be subject to local taxes which are calculated during checkout

Additional access options:

Figure 1: Memristor crossbar.

The alternative text for this image may have been generated using AI.

Figure 2: Pattern classification experiment (top-level description).

The alternative text for this image may have been generated using AI.

Figure 3: Pattern classification experiment (physical-level description).

The alternative text for this image may have been generated using AI.

Figure 4: Pattern classification experiment: results.

The alternative text for this image may have been generated using AI.

Similar content being viewed by others

References

  1. Mountcastle, V. B. The Cerebral Cortex (Harvard Univ. Press, 1998)
    Google Scholar
  2. Likharev, K. K. CrossNets: neuromorphic hybrid CMOS/nanoelectronic networks. Sci. Adv. Mater. 3, 322–331 (2011)
    Article CAS Google Scholar
  3. Likharev, K. K. Hybrid CMOS/nanoelectronic circuits: opportunities and challenges. J. Nanoelectron. Optoelectron. 3, 203–230 (2008)
    Article Google Scholar
  4. Strukov, D. B. & Williams, R. S. Four-dimensional address topology for circuits with stacked multilayer crossbar arrays. Proc. Natl Acad. Sci. USA 106, 20155–20158 (2009)
    Article ADS CAS Google Scholar
  5. Xia, Q. et al. Memristor-CMOS hybrid integrated circuits for configurable logic. Nano Lett. 9, 3640–3645 (2009)
    Article ADS CAS Google Scholar
  6. Chevallier, C. J. et al. 0.13μm 64Mb multi-layered conductive metal-oxide memory. Int. Solid-State Circuits Conf. 10, 260–261 (2010)
    Google Scholar
  7. Miyamura, M. et al. Programmable cell array using rewritable solid-electrolyte switch integrated in 90 nm CMOS. Int. Solid-State Circuits Conf. 11, 228–229 (2011)
    Google Scholar
  8. Kawahara, A. et al. An 8Mb multi-layered cross-point ReRAM macro with 443MB/s write throughput. Int. Solid-State Circuits Conf. 12, 432–434 (2012)
    Google Scholar
  9. Kim, G. H. et al. 32×32 crossbar array resistive memory composed of a stacked Schottky diode and unipolar resistive memory. Adv. Funct. Mater. 23, 1440–1449 (2013)
    Article CAS Google Scholar
  10. Kim, K.-H. et al. A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. Nano Lett. 12, 389–395 (2012)
    Article ADS CAS Google Scholar
  11. Yang, J. J., Strukov, D. B. & Stewart, D. R. Memristive devices for computing. Nature Nanotechnol. 8, 13–24 (2013)
    Article ADS CAS Google Scholar
  12. Liu, T. et al. A 130.7-mm 2-layer 32-Gb ReRAM memory device in 24-nm technology. IEEE J. Solid-State Circuits 49, 140–153 (2014)
    Article ADS Google Scholar
  13. Jo, S. H. et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297–1301 (2010)
    Article ADS CAS Google Scholar
  14. Chanthbouala, A. et al. A ferroelectric memristor. Nature Mater. 11, 860–864 (2012)
    Article ADS CAS Google Scholar
  15. Seo, K. et al. Analog memory and spike-timing-dependent plasticity characteristics of a nanoscale titanium oxide bilayer resistive switching device. Nanotechnology 22, 254023 (2011)
    Article ADS Google Scholar
  16. Ohno, T. et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nature Mater. 10, 591–595 (2011)
    Article ADS CAS Google Scholar
  17. Ziegler, M. et al. An electronic version of Pavlov’s dog. Adv. Funct. Mater. 22, 2744–2749 (2012)
    Article CAS Google Scholar
  18. Alibart, F., Zamanidoost, E. & Strukov, D. B. Pattern classification by memristive crossbar circuits with ex-situ and in-situ training. Nature Commun. 4, 2072 (2013)
    Article ADS Google Scholar
  19. Eryilmaz, S. B. et al. Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array. Front. Neurosci. 8, 205 (2014)
    Article Google Scholar
  20. Waser, R., Dittman, R., Staikov, G. & Szot, K. Redox-based resistive switching memories. Adv. Mater. 21, 2632–2663 (2009)
    Article CAS Google Scholar
  21. Wong, H. S. P. et al. Metal–oxide RRAM. Proc. IEEE 100, 1951–1970 (2012)
    Article CAS Google Scholar
  22. Hertz, J., Krogh, A. & Palmer, R. G. Introduction to the Theory of Neural Computation (Perseus, 1991)
    Google Scholar
  23. Pershin, Y. V. & Di Ventra, M. Experimental demonstration of associative memory with memristive neural network. Neural Netw. 23, 881–886 (2010)
    Article Google Scholar
  24. Hasler, J. & Marr, B. Finding a roadmap to achieve large neuromorphic hardware systems. Front. Neurosci. 7, 118 (2013)
    Article Google Scholar
  25. Kaneko, Y., Nishitani, Y. & Ueda, M. Ferroelectric artificial synapses for recognition of a multishaded image. IEEE Trans. Electron. Dev. 61, 2827–2833 (2014)
    Article ADS CAS Google Scholar
  26. Pi, S., Lin, P. & Xia, Q. Cross point arrays of 8 nm × 8 nm memristive devices fabricated with nanoimprint lithography. J. Vacuum Sci. Technol. B 31, 06FA02–1 (2013)
    Article Google Scholar
  27. Govoreanu, B. et al. Vacancy-modulated conductive oxide resistive RAM (VMCO-RRAM). IEDM Tech Dig. 10.2. 1–4 http://dx.doi.org/10.1109/IEDM.2013.6724599 (2013)
    Google Scholar
  28. LeCun, Y., Bottou, L., Orr, G. B. & Müller, K.-R. Efficient backprop. Lect. Notes Comput. Sci. 7700, 9–48 (2012)
    Article Google Scholar
  29. Schiffmann, W., Joost, M. & Werner, R. Optimization of the Backpropagation Algorithm for Training Multilayer Perceptrons https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.53.6869&rep=rep1&type=pdf (Technical Report, Institute of Physics, University of Koblenz, 1994)
  30. Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Neural Inf. Process. Systems 12, 1097–1105 (2012)
    Google Scholar

Download references

Acknowledgements

We acknowledge useful discussions with F. Alibart, I. Kataeva, W. Lu, L. Sengupta, S. Stemmer, and E. Zamanidoost. This work was supported by the AFOSR under the MURI grant FA9550-12-1-0038, by DARPA under contract number HR0011-13-C-0051UPSIDE via BAE Systems, and by the DENSO Corporation, Japan.

Author information

Author notes

  1. M. Prezioso, F. Merrikh-Bayat and B. D. Hoskins: These authors contributed equally to this work.

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of California at Santa Barbara, Santa Barbara, 93106, California, USA
    M. Prezioso, F. Merrikh-Bayat, B. D. Hoskins, G. C. Adam & D. B. Strukov
  2. Department of Physics and Astronomy, Stony Brook University, Stony Brook, 11794, New York, USA
    K. K. Likharev

Authors

  1. M. Prezioso
  2. F. Merrikh-Bayat
  3. B. D. Hoskins
  4. G. C. Adam
  5. K. K. Likharev
  6. D. B. Strukov

Contributions

M.P., F.M.-B., B.D.H., K.K.L., and D.B.S. designed the research. M.P., B.D.H., and G.C.A. performed fabrication and device testing. M.P. and F.M.-B. performed pattern classifier experiments. All authors discussed and interpreted results. M.P., K.K.L., and D.B.S. wrote the manuscript. K.K.L. and D.B.S. advised on all parts of the project.

Corresponding authors

Correspondence toM. Prezioso or D. B. Strukov.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

PowerPoint slides

Source data

Rights and permissions

About this article

Cite this article

Prezioso, M., Merrikh-Bayat, F., Hoskins, B. et al. Training and operation of an integrated neuromorphic network based on metal-oxide memristors.Nature 521, 61–64 (2015). https://doi.org/10.1038/nature14441

Download citation

This article is cited by

Editorial Summary

A neuromorphic network based on metal-oxide memristors

Building neuromorphic networks matching the cognitive complexity of their biological prototypes but with higher performance is one of the great challenges in computing. One promising approach to such devices — potentially simpler than those based on complex silicon circuits — combines complementary metal-oxide-semiconductors (CMOSs) with adjustable two-terminal resistive devices (memristors). Here Dmitri Strukov and colleagues demonstrate a transistor-free metal-oxide memristor network with low device variability that works as a single-layer perceptron. That is, it can learn to recognize imperfect 3 × 3 pixel black-and-white patterns as one of three letters of the alphabet. The strength of this approach is its scalability so that larger neuromorphic networks capable of more challenging tasks should be possible.

Associated content