We, and our partners spread across multiple institutions around the country, are studying the function of the brain, directly observing it at work, and tracking its activity in real-time as an animal learns. Combining state-of-the-art automated training procedures with next generation temporal focusing two-photon calcium imaging techniques, we are measuring how the activity of thousands of neurons in the brain correspond to an animal's experience of the visual world. By watching these patterns evolve over the course of learning, we can capture powerful clues about the learning rules that the brain uses to grasp new information. To dig deeper into the machinery of the brain, we are subsequently deploying massive-scale imaging techniques like synchrotron X-ray tomography on the same brains to image and reconstruct the connectome of a significant portion of the rodent's visual cortex. To date, a combined effort to characterize the function and anatomy of such a large portion of a brain has never been attempted.

Collecting data from brains is one thing, but how do we translate that data into a useful algorithm to solve a real problem? A crucial question for a neural network model is: does it have the same structure and function as networks of cells in the brain? The answer to this seminal question is extremely complicated. But with nature as our guide, we have a solid reference point in the connectome.

An understanding of the connectome is fundamental, long-term research. The remarkable diversity of life on planet earth means that there are many aspects of sensory systems and associated neural computations to explore. This is exactly why computer scientists are interested in this research – it provides new ways to think about problem solving and models of computation. While the clinical goals of this research are admirable, the promise of a new model of computation could be even more important. While current artificial intelligence software is impressive, it still comes up short in the contexts that matter most to the healthcare, defense, and intelligence communities: settings where decisions must be made from limited amounts of ambiguous data, and where those decisions must account for uncertainty. It is in precisely these contexts that humans excel. The human brain, arguably the most powerful known computational system, evolved to operate with extraordinary efficiency and accuracy in ambiguous, complicated environments. Initial work in building biologically-inspired neural networks has shown great promise, but the structure of today's artificial neural networks is a pale reflection of the complexity found in even relatively simple mammalian brains.

This work was supported by IARPA contract #D16PC00002, the Department of Defense (Army Research Laboratory) under the contract W911NF-18-1-0292, and the NVIDIA Corporation