A Single Math Model Explains Many Mysteries of Vision
But then the visual cortex goes to work. While the cortex and the retina are connected by relatively few neurons, the cortex itself is dense with nerve cells. For every 10 LGN neurons that snake back from the retina, there are 4,000 neurons in just the initial “input layer” of the visual cortex—and many more in the rest of it. This discrepancy suggests that the brain heavily processes the little visual data it does receive.
“The visual cortex has a mind of its own,” Shapley said.
For researchers like Young, Shapley, and Chariker, the challenge is deciphering what goes on in that mind.
Visual Loops
The neural anatomy of vision is provocative. Like a slight person lifting a massive weight, it calls out for an explanation: How does it do so much with so little?
Young, Shapley, and Chariker are not the first to try and answer that question with a mathematical model. But all previous efforts assumed that more information travels between the retina and the cortex—an assumption that would make the visual cortex’s response to stimuli easier to explain.
“People hadn’t taken seriously what the biology was saying in a computational model,” Shapley said.
Mathematicians have a long, successful history of modeling changing phenomena, from the movement of billiard balls to the evolution of spacetime. These are examples of “dynamical systems”—systems that evolve over time according to fixed rules. Interactions between neurons firing in the brain are also an example of a dynamical system—albeit one that’s especially subtle and hard to pin down in a definable list of rules.
LGN cells send the cortex a train of electrical impulses one-tenth of a volt in magnitude and one millisecond in duration, setting off a cascade of neuron interactions. The rules that govern these interactions are “infinitely more complicated” than the rules that govern interactions in more familiar physical systems, Young said.
Individual neurons receive signals from hundreds of other neurons simultaneously. Some of these signals encourage the neuron to fire. Others restrain it. As a neuron receives electrical pulses from these excitatory and inhibitory neurons, the voltage across its membrane fluctuates. It only fires when that voltage (its “membrane potential”) exceeds a certain threshold. It’s nearly impossible to predict when that will happen.
“If you watch a single neuron’s membrane potential, it’s fluctuating wildly up and down,” Young said. “There’s no way to tell exactly when it’s going to fire.”
The situation is even more complicated than that. Those hundreds of neurons connected to your single neuron? Each of those is receiving signals from hundreds of other neurons. The visual cortex is a swirling play of feedback loop upon feedback loop.
“The problem with this thing is there are a lot of moving parts. That’s what makes it difficult,” Shapley said.
Earlier models of the visual cortex ignored this feature. They assumed that information flows just one way: from the front of the eye to the retina and into the cortex until, voilà, vision appears at the end, as neat as a widget coming off a conveyor belt. These “feed forward” models were easier to create, but they ignored the plain implications of the anatomy of the cortex—which suggested “feedback” loops had to be a big part of the story.
“Feedback loops are really hard to deal with because the information keeps coming back and changes you, it keeps coming back and affecting you,” Young said. “This is something that almost no model deals with, and it’s everywhere in the brain.”
In their initial 2016 paper, Young, Shapley, and Chariker began to try and take these feedback loops seriously. Their model’s feedback loops introduced something like the butterfly effect: Small changes in the signal from the LGN were amplified as they ran through one feedback loop after another in a process known as “recurrent excitation” that resulted in large changes in the visual representation produced by the model in the end.
