How do neurons teach neurons?
Aggregate-label learning. For many decades, research on learning in neural networks, in particular with spiking neurons, has been obscured by the dogma that learning requires temporally precise teaching signals that instruct each neuron at which points in time it should modify its responses. We now know that this dogma is wrong. We have discovered that models of spiking neurons can perform previously unimaginable learning tasks without any time resolved teaching signals (Gütig, 2016). Already binary teaching signals, “aggregate-labels”, without temporal information, that merely instruct neurons to elicit fewer or more output spikes, enable neurons to discover reward predictive features within long streams of noise. Overcoming the dogma of temporally precise teaching signals, i.e. solving the fundamental temporal credit assignment problem (Sutton, 1988), has opened a vast field of new possibilities how learning neurons can interact with the external world and how neurons can teach neurons.
(A) Network architecture, grey: input layer; blue: student layer; red: teacher unit. (B) Voltage traces of 10 self-supervised student neurons after learning. Despite no external feedback, all student neurons discover and learn to respond to the blue input feature. Modified from Gütig (2016).
In a first step into this field of possibilities, we have discovered the concept of neural “self-supervision”: Feeding back simple projections of neural activity, such as its mean, as a teaching signal to groups of neurons allows simple neural networks to discover distributed ensembles of features even if their occurrences are rare and widely distributed across space and time (Fig. 2). In this part of the project we will test the neurobiological implementations of aggregate-label learning and neural self-supervision and hunt for their traces in human and animal learning behaviors. This part of our project also has a crucial theoretical component within which will combine and extend our theoretical advances to realize supervised and unsupervised learning, side-by-side, within deep multi-layer networks of spiking neurons.
This is placeholder text. To change this content, double-click on the element and click Change Content. Want to view and manage all your collections? Click on the Content Manager button in the Add panel on the left. Here, you can make changes to your content, add new fields, create dynamic pages and more. You can create as many collections as you need.
Your collection is already set up for you with fields and content. Add your own, or import content from a CSV file. Add fields for any type of content you want to display, such as rich text, images, videos and more. You can also collect and store information from your site visitors using input elements like custom forms and fields.
Be sure to click Sync after making changes in a collection, so visitors can see your newest content on your live site. Preview your site to check that all your elements are displaying content from the right collection fields.
Power in Numbers