Towards new generation, biologically plausible deep neural network learning

Abstract

Artificial neural networks in their various different forms convincingly dominate machine learning of the present day. Nevertheless, the manner in which these networks are trained, in particular by using end-to-end backpropagation, presents a major limitation in practice and hampers research, and raises questions with regard to the very fundamentals of the learning algorithm design. Motivated by these challenges and the contrast between the phenomenology of biological (natural) neural networks that artificial ones are inspired by and the learning processes underlying the former, there has been an increasing amount of research on the design of biologically plausible means of training artificial neural networks. In this paper we (i) describe a biologically plausible learning method that takes advantage of various biological processes, such as Hebbian synaptic plasticity, and includes both supervised and unsupervised elements, (ii) conduct a series of experiments aimed at elucidating the advantages and disadvantages of the described biologically plausible learning as compared with end-to-end backpropagation, and (iii) discuss the findings which should serve as a means of illuminating the algorithmic fundamentals of interest and directing future research. Among our findings is the greater resilience of biologically plausible learning to data scarcity, which conforms to our expectations, but also its lesser robustness to additive, zero mean Gaussian noise.