Walther Carballo-Hernández
ABSTRACT
The biological human brain works with spiking neural networks which computational complexity is simple, compensated by the high density connection between neurons. However, most of our research in artificial neural networks is based on simplified models that need many real values processing elements of complex computing which require too much silicon space, energy and slow learning convergence. Therefore, in this brief article, it is presented a proposal for a fully digital architecture on FPGAs optimized for a highly dense intersynaptic connection on an event-based quantized Sigma-Delta pulse coding for deep Convolutional Neural Networks. This article presents early results of an approach of neuromorphic hardware design for information or pixel luminosity changes coded in time, using Sigma-Delta modulation, the design of a Pulsed Arithmetic-Logic Unit for bitstream operations with quantized weights reducing memory, from 32 bits in floating point representation, down to 1 bit. This 32x memory reduction and binary operation cells in a systolic architectures makes the integration of deep learning models feasible for embedded design like smart cameras using FPGAs.
- A. Krizhevsky, I. Sutskever, and G. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," In Advances in neural information processing systems, pp. pp. 1097--1105, 2012. Google Scholar
Digital Library
- Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, 1998.Google Scholar
- M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, and Y. Bengio, "Binarized Neural Networks: Training Neural Networks with Weights and Activations Constrained to +1 or -1," arXiv:1602.02830, 2016.Google Scholar
- I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, "Quantized Neural Network: Training Neural Network with Low Precision Weight and Activations," arXiv:1609.07061, 2016.Google Scholar
- P. O'Connor and M. Welling, "Deep Spiking Networks," Neural and Evolutionary Computing. arXiv:1602.0832, February 2016.Google Scholar
- P. O'Connor and M. Welling, "Sigma-Delta Quantized Networks," International Conference on Learning Representations ICLR 2017, 2017.Google Scholar
- L. A. Camunas-Mesa, T. Serrano-Gotarredona, and B. Linares-Barranco, "Event-driven sensing and processing for high-speed robotic vision," Biomedical Circuits and Systems Conference (BioCAS), IEEE, pp. pp. 516--519, October 2014.Google Scholar
- D. Amani, "Capteur d'Images Événementiel Asynchrone à Échantil-lonnage Non-uniforme," Ph.D. dissertation, 2016.Google Scholar
- N. Kouvaras, "Operations on delta-modulated signals and their application in the realization of digital filters," Radio and Electronic Engineer, 1978.Google Scholar
- V. Salapura, "Neural Networks Using Bit Stream Arithmetic: a Space Efficient Implementation," Circuits and Systems, 1994. ISCAS '94., 1994 IEEE International Symposium on, 1994.Google Scholar
Index Terms
- Deep Learning Pulsed-based Convolutional Neuroprocessor Architecture on FPGAs
Recommendations
Associative Learning of Integrate-and-Fire Neurons with Memristor-Based Synapses
A memrsitor is a two-terminal electronic device whose conductance can be precisely modulated by charge or flux through it. In this paper, we present a class of memristor-based neural circuits comprising leaky integrate-and-fire (I & F) neurons and ...
Hebbian Learning Meets Deep Convolutional Neural Networks
Image Analysis and Processing – ICIAP 2019AbstractNeural networks are said to be biologically inspired since they mimic the behavior of real neurons. However, several processes in state-of-the-art neural networks, including Deep Convolutional Neural Networks (DCNN), are far from the ones found in ...
Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition
Deep-learning neural networks such as convolutional neural network (CNN) have shown great potential as a solution for difficult vision problems, such as object recognition. Spiking neural networks (SNN)-based architectures have shown great potential as ...
Comments