Vision

Vision is the best understood of the senses and the one that occupies the largest share of the primate brain: roughly a third of human cerebral cortex is involved in processing visual information. It is also a showcase for the principles of this course. Phototransduction illustrates G-protein signalling with enormous amplification; the retina is a layered circuit that computes contrast and motion before any signal leaves the eye; the primary visual cortex builds new features such as orientation from simpler inputs; and higher areas divide the work into parallel streams. This lesson follows light from photon to perception.

Phototransduction

Light is detected by photoreceptors at the back of the retina: about 120 million rods, which are extremely sensitive and serve vision in dim light, and about 6 million cones, which work in daylight and come in three types sensitive to short, medium and long wavelengths, the basis of colour vision. Cones are concentrated in the fovea, a small pit at the centre of gaze where acuity is highest; rods dominate the periphery, which is why a faint star is easier to see slightly to one side of where you look.

Photoreceptors work backwards compared with most sensory cells. In the dark, cyclic GMP holds open cation channels in the outer segment, letting a steady dark current of sodium and calcium flow in, so the cell is depolarized to about −40 mV and releases glutamate continuously. Light hyperpolarizes it. The cascade is a classic G-protein pathway with enormous amplification. A photon isomerizes 11-cis retinal, bound to the protein opsin (in rods, rhodopsin), to all-trans retinal. The activated rhodopsin catalyses the activation of hundreds of molecules of the G protein transducin; each transducin activates a phosphodiesterase that hydrolyses cyclic GMP rapidly; the fall in cyclic GMP closes cation channels, and the cell hyperpolarizes, reducing glutamate release. A rod can detect a single photon, closing a few per cent of its open channels and producing a current change of about 1 picoampere, as Denis Baylor and colleagues showed in 1979.

Termination is just as important as activation. Rhodopsin kinase phosphorylates active rhodopsin, and the protein arrestin caps it; transducin hydrolyses its bound GTP; and the fall in calcium during light activates guanylyl cyclase to resynthesize cyclic GMP. Calcium feedback underlies light adaptation, which lets cones operate over about eight orders of magnitude of light intensity, from moonlight to bright sun, by continually adjusting their sensitivity. Defects in phototransduction proteins cause inherited retinal diseases such as retinitis pigmentosa, and gene therapy for one of them, a form of Leber congenital amaurosis caused by RPE65 mutations, was approved by the US Food and Drug Administration in 2017 and by Health Canada in 2020 as the first gene therapy for an inherited disease.

The retinal circuit

The retina is part of the brain, an outgrowth of the embryonic diencephalon, and it does substantial processing. Information flows vertically from photoreceptors to bipolar cells to retinal ganglion cells, whose axons form the optic nerve, and laterally through horizontal cells in the outer retina and amacrine cells in the inner retina. Light must pass through all these layers to reach the photoreceptors, an apparently backward arrangement that reflects how the eye develops; where the axons exit there are no photoreceptors, creating the blind spot.

Bipolar cells come in two broad classes. OFF bipolar cells express ionotropic glutamate receptors, so they are depolarized by glutamate and therefore by darkness. ON bipolar cells express the metabotropic receptor mGluR6, which closes a cation channel (TRPM1) when glutamate binds, so they are hyperpolarized by glutamate and depolarized when light reduces glutamate release. This sign inversion splits the visual signal into parallel ON and OFF channels from the first synapse, so increments and decrements of light are each signalled by an increase in firing. The two channels stay separate through the retina, the thalamus and into cortex.

Centre–surround receptive fields

In 1953 Stephen Kuffler, recording from cat retinal ganglion cells, found that each cell responds to light in a small circular region, the receptive field centre, and is opposed by light in a surrounding annulus. An ON-centre cell fires more when light falls on its centre and less when light falls on its surround; an OFF-centre cell does the opposite. Uniform illumination of both centre and surround produces only a weak response. The surround comes largely from lateral inhibition through horizontal cells. Ganglion cells therefore signal local contrast, differences in light between nearby points, rather than absolute brightness. This is efficient, because in natural scenes neighbouring points tend to have similar intensities, so transmitting differences removes redundancy, and it explains illusions such as Mach bands, in which bright and dark bands appear at the edges of luminance ramps.

A centre–surround receptive field is well described as a difference of two Gaussian functions: a narrow, strong excitatory centre minus a broad, weaker inhibitory surround. Applying this filter to an image enhances edges.

# Difference-of-Gaussians receptive field applied to a 1-D luminance profile with a step and a ramp.
import numpy as np
x = np.arange(-30, 31)                       # receptive field positions (arbitrary units)
def gauss(s): g = np.exp(-x**2 / (2 * s**2)); return g / g.sum()
dog = gauss(2.0) - gauss(6.0)                # narrow excitatory core minus equally weighted broad surround
print("DoG weights near the middle:", np.round(dog[27:34] * 100, 1))
print("sum of weights (response to uniform light):", round(dog.sum(), 3))

# luminance: dark plateau, a sharp step up, plateau, then a gentle ramp down
lum = np.concatenate([np.full(60, 20.0), np.full(60, 80.0), np.linspace(80, 40, 60), np.full(60, 40.0)])
resp = np.convolve(lum, dog, mode='same')
for i in list(range(44, 78, 3)) + list(range(110, 200, 8)):
    r = resp[i]
    bar = '#' * int(r / 1.5) if r > 0 else '-' * int(-r / 1.5)
    print(f"pos {i:3d} lum {lum[i]:5.1f}  response {r:+6.2f} {bar}")

DoG weights near the middle: [ 0.6  5.8 11.  13.3 11.   5.8  0.6]
sum of weights (response to uniform light): 0.0
pos  44 lum  20.0  response  -0.29 
pos  47 lum  20.0  response  -1.11 
pos  50 lum  20.0  response  -3.39 --
pos  53 lum  20.0  response  -8.31 -----
pos  56 lum  20.0  response -14.47 ---------
pos  59 lum  20.0  response  -3.99 --
pos  62 lum  80.0  response +14.10 #########
pos  65 lum  80.0  response +10.60 #######
pos  68 lum  80.0  response  +4.68 ###
pos  71 lum  80.0  response  +1.65 #
pos  74 lum  80.0  response  +0.47 
pos  77 lum  80.0  response  +0.10 
pos 110 lum  80.0  response  +0.08 
pos 118 lum  80.0  response  +0.92 
pos 126 lum  75.9  response  +0.34 
pos 134 lum  70.5  response  +0.01 
pos 142 lum  65.1  response  +0.00 
pos 150 lum  59.7  response  -0.00 
pos 158 lum  54.2  response  -0.00 
pos 166 lum  48.8  response  -0.02 
pos 174 lum  43.4  response  -0.46 
pos 182 lum  40.0  response  -0.77 
pos 190 lum  40.0  response  -0.05 
pos 198 lum  40.0  response  -0.00

The filter gives zero response on uniform plateaus, because centre and surround cancel, and responds strongly at the step, positively on the bright side and negatively on the dark side. Along the linear ramp the response is again close to zero, but it deviates at the corners where the ramp begins and ends, which is where people see the bright and dark Mach bands. In the real retina there are about 20 types of ganglion cell in mammals, each tiling the retina and extracting a different feature: ON and OFF transient and sustained responses, colour opponency, and direction of motion. Direction-selective ganglion cells, for example, owe their selectivity to asymmetric inhibition from starburst amacrine cells.

The pathway to cortex

Ganglion cell axons leave the eye in the optic nerve. At the optic chiasm, axons from the nasal half of each retina cross, so the left side of the brain receives information about the right visual field from both eyes. Most axons go to the lateral geniculate nucleus (LGN) of the thalamus; others go to the superior colliculus, which orients the eyes and head; to the pretectum, which controls the pupillary reflex; and, from a special class of ganglion cells containing the photopigment melanopsin, to the suprachiasmatic nucleus, which sets the circadian clock (lesson 10). The LGN has six layers in primates: magnocellular layers carry fast, high-contrast-sensitivity, colour-blind signals from large parasol ganglion cells; parvocellular layers carry slower, finer, red–green colour-opponent signals from midget ganglion cells; and koniocellular cells between the layers carry blue–yellow signals.

Primary visual cortex

From the LGN, axons project to primary visual cortex (V1, also called striate cortex) in the occipital lobe. In the late 1950s and 1960s David Hubel and Torsten Wiesel recorded from V1 in cats and monkeys and found that most neurons respond not to spots but to bars or edges of a particular orientation. Simple cells have elongated receptive fields with separate ON and OFF subregions, as if built by lining up several LGN centre–surround fields; complex cells respond to a correctly oriented bar anywhere within a larger receptive field, as if combining several simple cells. Neurons are organized in columns: neurons in a column perpendicular to the cortical surface share orientation preference, and preference rotates smoothly across the surface, interrupted by pinwheel centres. Inputs from the two eyes are segregated into alternating ocular dominance columns. Hubel and Wiesel shared the 1981 Nobel Prize, with Roger Sperry.

Orientation tuning is quantified by recording responses to gratings at many orientations. Tuning curves are well fitted by a circular function, the von Mises function, because orientation repeats every 180 degrees:

where \(\theta_0\) is the preferred orientation, \(A\) the peak response above baseline \(b\), and \(\kappa\) sets the tuning width.

# Fit a von Mises orientation tuning curve to synthetic spike counts from an invented V1 neuron.
import numpy as np
rng = np.random.default_rng(5)
angles = np.arange(0, 180, 15)                         # degrees
def vm(theta, b, A, kappa, th0):
    return b + A * np.exp(kappa * (np.cos(np.deg2rad(2 * (theta - th0))) - 1))
true = (4.0, 30.0, 2.0, 62.0)
counts = np.array([rng.poisson(vm(a, *true)) for a in angles for _ in range(10)]).reshape(len(angles), 10)
mean = counts.mean(axis=1)
best, best_err = None, np.inf                         # coarse grid search on kappa and preferred angle
for th0 in np.arange(0, 180, 1.0):
    for kappa in np.arange(0.5, 6.0, 0.1):
        shape = np.exp(kappa * (np.cos(np.deg2rad(2 * (angles - th0))) - 1))
        X = np.column_stack([np.ones_like(shape), shape])
        coef, *_ = np.linalg.lstsq(X, mean, rcond=None)   # linear fit of b and A
        err = np.sum((X @ coef - mean) ** 2)
        if err < best_err:
            best, best_err = (coef[0], coef[1], kappa, th0), err
b, A, kappa, th0 = best
half = np.rad2deg(np.arccos(1 + np.log(0.5) / kappa)) / 2   # half-width at half height
print("orientation  mean spikes")
for a, m in zip(angles, mean): print(f"   {a:3d} deg   {m:5.1f}")
print(f"fit: preferred {th0:.0f} deg, peak {A:.1f} above baseline {b:.1f}, kappa {kappa:.1f}, half-width {half:.0f} deg")

orientation  mean spikes
     0 deg     5.5
    15 deg     8.1
    30 deg    14.2
    45 deg    25.2
    60 deg    32.6
    75 deg    27.8
    90 deg    17.8
   105 deg     8.7
   120 deg     5.7
   135 deg     4.7
   150 deg     3.6
   165 deg     4.5
fit: preferred 62 deg, peak 29.5 above baseline 3.1, kappa 1.7, half-width 27 deg

Development and critical periods

Hubel and Wiesel also showed that closing one eye of a kitten during a critical period in the first few months of life permanently shifted cortical neurons to respond almost entirely to the open eye, and the deprived eye's ocular dominance columns shrank. The same closure in an adult cat had little effect. This experience-dependent competition, explained in part by Hebbian and BCM plasticity (lesson 6), has a direct clinical counterpart: amblyopia, reduced vision in one eye from childhood misalignment (strabismus), unequal focus or cataract, must be treated early, by correcting the problem and patching the stronger eye. Research on reopening plasticity in adults, by manipulating inhibition, perineuronal nets or neuromodulation, and on binocular treatments using video games, has expanded options for older children and adults.

Beyond V1: parallel streams

Beyond V1, visual information flows through more than 30 cortical areas in primates. In 1982 Leslie Ungerleider and Mortimer Mishkin proposed two broad streams: a ventral stream through V4 to inferotemporal cortex, specialized for recognizing objects (what), and a dorsal stream through MT (V5) to parietal cortex, specialized for spatial location and motion (where), later reinterpreted by Melvyn Goodale and David Milner as vision for action (how). Neurons in MT are selective for direction and speed of motion; damage to MT causes akinetopsia, an inability to perceive motion. In inferotemporal cortex, neurons respond to complex shapes, and patches such as the fusiform face area respond selectively to faces; damage causes prosopagnosia, an inability to recognize faces. Receptive fields grow larger and selectivity more complex along the ventral hierarchy, a design that inspired convolutional neural networks (CMPUT-466), whose internal representations, when trained on object recognition, resemble those recorded along the primate ventral stream.

Vision is also actively constructed. Perception depends on expectations and context, as illusions show; feedback connections from higher to lower areas are at least as numerous as feedforward ones; and the eyes move about three times a second in saccades, so the brain must stitch together a stable world from brief snapshots, suppressing vision during each saccade. Some patients with damage to V1 can point to or discriminate stimuli they report not seeing, a phenomenon called blindsight, probably supported by pathways through the superior colliculus and LGN that bypass V1.