Switched by input: power efficient structure for RRAM-based convolutional neural network

ABSTRACT

Convolutional Neural Network (CNN) is a powerful technique widely used in computer vision area, which also demands much more computations and memory resources than traditional solutions. The emerging metal-oxide resistive random-access memory (RRAM) and RRAM crossbar have shown great potential on neuromorphic applications with high energy efficiency. However, the interfaces between analog RRAM crossbars and digital peripheral functions, namely Analog-to-Digital Converters (ADCs) and Digital-to-Analog Converters (DACs), consume most of the area and energy of RRAM-based CNN design due to the large amount of intermediate data in CNN. In this paper, we propose an energy efficient structure for RRAM-based CNN. Based on the analysis of data distribution, a quantization method is proposed to transfer the intermediate data into 1 bit and eliminate DACs. An energy efficient structure using input data as selection signals is proposed to reduce the ADC cost for merging results of multiple crossbars. The experimental results show that the proposed method and structure can save 80% area and more than 95% energy while maintaining the same or comparable classification accuracy of CNN on MNIST.

References

1. J. Qiu et al., "Going deeper with embedded fpga platform for convolutional neural network," in ACM International Symposium on FPGA, 2016.
2. C. Zhang et al., "Optimizing fpga-based accelerator design for deep convolutional neural networks," in FPGA, 2015, pp. 161--170.
3. M. Hu et al., "Hardware realization of bsb recall function using memristor crossbar arrays," in DAC, 2012, pp.
4. B. Li et al., "Memristor-based approximated computation," in ISLPED. IEEE Press, 2013, pp. 242--247.
5. C. Xu et al., "Design implications of memristor-based rram cross-point structures," in DATE, 2011, pp. 1--6.
6. L. Xia et al., "MNSIM: Simulation platform for memristor-based neuromorphic computing system," in DATE, 2016, pp. 469--474.
7. K. Simonyan et al., "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
8. Y. Zhang et al., "Random telegraph noise analysis in alox/woy resistive switching memories," Applied Physics Letters, vol. 104, no. 10, p. 103507, 2014.
9. Y. LeCun et al., "The mnist database of handwritten digits," 1998.
10. M. Kim et al., "Bitwise neural networks," in ICML workshop, 2015.
11. J. Fieres et al., "Training convolutional networks of threshold neurons suited for low-power hardware implementation," in IJCNN. IEEE, 2006, pp. 21--28.
12. A. Krizhevsky et al., "Imagenet classification with deep convolutional neural networks," in NIPS, 2012, pp. 1097--1105.
13. F. Alibart et al., "High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm," Nanotechnology, vol. 23, no. 7, p. 075201, 2012.
14. D. Garbin et al., "Hfo2-based oxram devices as synapses for convolutional neural networks," TED, vol. 62, no. 8, pp. 2494--2501, Aug 2015.
15. M. Catanzaro et al., "Reconfigurable rram for lut logic mapping: A case study for reliability enhancement," in SOCC, 2012, pp. 94--99.
16. Y. Zhang et al., "Study of conduction and switching mechanisms in Al/AlOx/WOx/W resistive switching memory for multilevel applications," Applied Physics Letters, vol. 102, no. 23, p. 233502, 2013.
17. R. St Amant et al., "General-purpose code acceleration with limited-precision analog computation," in ISCA, 2014, pp. 505--516.
18. W.-H. Tseng et al., "A 960ms/s dac with 80db sfdr in 20nm cmos for multi-mode baseband wireless transmitter," in VLSI Circuits Digest of Technical Papers, 2014, pp. 1--2.
19. B. Li et al., "Merging the interface: Power, area and accuracy co-optimization for rram crossbar-based mixed-signal computing system," in DAC, 2015.
20. S. Han et al., "Learning both weights and connections for efficient neural networks," arXiv preprint arXiv:1506.02626, 2015.
21. S. Yu et al., "A low energy oxide-based electronic synaptic device for neuromorphic visual systems with tolerance to device variation," Advanced Materials, vol. 25, no. 12, pp. 1774--1779, 2013.
22. T. Tang et al., "Spiking neural network with rram: Can we use it for real-world application?" in DATE, 2015, pp. 860--865.