Sensory Coding

Psychophysics Relates the Physical Properties of Stimuli to Sensations

Psychophysical Laws Govern the Perception of Stimulus Intensity

Psychophysical Measurements of Sensation Magnitude Employ Standardized Protocols

Sensations Are Quantified Using Probabilistic Statistics

Decision Times Are Correlated with Cognitive Processes

Physical Stimuli Are Represented in the Nervous System by Means of the Sensory Code

Sensory Receptors Are Responsive to a Single Type of Stimulus Energy

Multiple Subclasses of Sensory Receptors Are Found in Each Sense Organ

Neural Firing Patterns Transmit Sensory Information to the Brain

The Receptive Field of a Sensory Neuron Conveys Spatial Information

Modality-Specific Pathways Extend to the Central Nervous System

The Receptor Surface Is Represented Topographically in Central Nuclei

Feedback Regulates Sensory Coding

Top-Down Learning Mechanisms Influence Sensory Processing

An Overall View

SINCE ANCIENT TIMES HUMANS have been fascinated by the nature of sensory experience. The Greek philosopher Aristotle defined five senses—vision, hearing, touch, taste, and smell—each linked to specific sense organs in the body: the eyes, ears, skin, tongue, and nose (Figure 21–1). Pain was not considered to be a specific sensory modality but rather an affliction of the soul. Intuition, often referred to colloquially as a “sixth sense,” was something beyond the experience of classic sensory systems. Today neurobiologists are more likely to describe intuition as inferences derived from previous experience and thus the result of cognitive rather than sensory processes.

Figure 21-1 The major sensory modalities in humans are mediated by distinct classes of receptor neurons located in specific sense organs. Each class of receptor cell transforms one type of stimulus energy into electrical signals that are encoded as trains of action potentials. The principal receptor cells include photoreceptors (vision), chemoreceptors (smell, taste, and pain), thermal receptors, and mechanoreceptors (touch, hearing, balance, and proprioception). The classic five senses—vision, smell, taste, touch, and hearing—and the sense of balance are mediated by receptors in the eye, nose, mouth, skin, and inner ear, respectively. The other somatosensory modalities—thermal senses, pain, and proprioception—are mediated by receptors distributed throughout the body.

In this chapter we consider the organizational principles and coding mechanisms universal to all sensory systems. We define sensory information as neural activity originating from stimulation of receptor cells in specific parts of the body. These senses include the classic five senses plus a variety of modalities not recognized by the ancients but essential to bodily function: the somatic sensations of proprioception (posture and movement of our own body), pain, itch, and temperature; visceral sensations (both conscious and unconscious) necessary for homeostasis; and the vestibular senses of balance (the position of the body in the gravitational field) and head movement.

The extent to which features of sensory processing have been conserved in the course of human evolution seems nothing short of astonishing. In each of the sensory systems receptors provide the first neural representation of the external world. This information flows centrally to regions of the brain involved in cognition. The sensory pathways have both serial and parallel components, consisting of fiber tracts with thousands or millions of axons interrupted by synaptic relays comprising millions of neurons. Along the way information is transformed from relatively simple forms to the complex forms that are the basis of cognition. Sensory pathways are also recursive. The higher centers in the brain modify and structure the incoming flow of sensory signals by feeding information back to earlier stages of processing; thus percepts are shaped by internal as well as environmental factors.

In each sensory modality a specific type of stimulus energy is transformed into electrical signals by specialized receptors. The sensory information is transmitted to the central nervous system by trains of action potentials that represent particular aspects of the stimulus. The question that has intrigued philosophers and scientists alike is whether experienced sensations accurately reflect the stimuli that produce them or whether our knowledge of the world is inherently subjective and imprecise.

Modern thought about how knowledge is represented in the brain began with European philosophers of the 17th, 18th, and 19th centuries whose interest in sensation and perception was related to the question of human nature itself. The major division was between the empiricists, represented by John Locke, George Berkeley, and David Hume, and the idealists, including René Descartes, Immanuel Kant, and Georg Wilhelm Friedrich Hegel. Locke, the preeminent empiricist, advanced the idea that the mind at birth is a blank slate, or tabula rasa, void of any ideas. Knowledge is obtained through sensory experience—what we see, hear, feel, taste, and smell. In fact, Berkeley questioned whether there was any sensory reality beyond the experiences and knowledge acquired through the senses. He asked the now-famous question: Does a falling tree make a sound if no one is near enough to hear it?

The idealists responded that the human mind possesses certain innate abilities, including logical reasoning itself. The 18th-century German philosopher Immanuel Kant classified the five senses as categories of human understanding. He argued that perceptions were not direct records of the world around us but rather were products of the brain and thus depended on the architecture of the nervous system. Kant referred to these brain properties as a priori knowledge.

Thus in Kant’s view the mind was not the passive receiver of sense impressions envisaged by the empiricists. Rather the human mind had evolved to conform to certain universal conditions such as space, time, and causality. These conditions were independent of any physical stimuli detected by the body. For Kant and the idealists this meant that knowledge is based not only on sensory stimulation but also on the brain’s properties that organize sensory experience. If sensory experience is inherently subjective and personal, they said, it may not be subject to scientific analysis.

Psychophysics Relates the Physical Properties of Stimuli to Sensations

The modern study of sensation and perception began in the 19th century with the emergence of experimental psychology as a scientific discipline. The first psychologists—Ernst Weber, Gustav Fechner, Hermann Helmholtz, and Wilhelm Wundt—focused their experimental study of mental processes on sensation, which they believed was the key to understanding the mind. Their findings gave rise to the fields of psychophysics and sensory physiology.

Psychophysics describes the relationship between the physical characteristics of a stimulus and the attributes of the sensory experience. Sensory physiology examines the neural consequences of a stimulus—how the stimulus is transduced by sensory receptors and processed in the brain. Some of the most exciting advances in our understanding of perception have come from merging these two approaches in both human and animal studies. For example, functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) have been used in controlled experiments to identify regions of the human brain involved in the perception of pain.

Psychophysical Laws Govern the Perception of Stimulus Intensity

Early scientific studies of the mind focused not on the perception of complex qualities such as color or taste but on phenomena that could be isolated and measured precisely: the size, shape, amplitude, velocity, and timing of stimuli. Weber and Fechner developed simple experimental paradigms to study how and under what conditions humans are able to distinguish between two stimuli of different amplitudes. They quantified the intensity of sensations in the form of mathematical laws that allowed them to predict the relationship between stimulus magnitude and sensory discrimination.

For example, in 1834 Weber demonstrated that the sensitivity of a sensory system to differences in intensity depends on the absolute strength of the stimuli. We easily perceive that 1 kg is different from 2 kg, but it is difficult to distinguish 50 kg from 51 kg. Yet both sets differ by 1 kg. This relationship is expressed in the equation now known as Weber’s law:

where ΔS is the minimal difference in strength between a reference stimulus S and a second stimulus that can be discriminated, and K is a constant. This is termed the just noticeable difference or difference limen. It follows that the difference in magnitude necessary to discriminate between a reference stimulus and a second stimulus increases with the strength of the reference stimulus.

Fechner extended Weber’s law to describe the relationship between the stimulus strength (S) and the intensity of the sensation (I) experienced by a subject:

where S₀ is the threshold amplitude of the stimulus and K is a constant. Although Fechner’s law was widely accepted for nearly a century after its publication in 1860, his assumption that the intensity of sensation could be equated with the sum of equal increments in “just noticeable differences” turned out to be incorrect.

In 1953 S. S. Stevens demonstrated that, over an extended range of stimulation, subjective experience of sensation intensity is best described by a power function rather than by a logarithmic relationship. Stevens’ law states that:

For some sensory experiences, such as the sense of pressure on the hand, the relationship between the stimulus magnitude and the perceived intensity is linear, that is, a power function with a unity exponent .

The lowest stimulus strength a subject can detect is termed the sensory threshold. Thresholds are normally determined statistically by presenting a subject with a series of stimuli of random amplitude. The percentage of times the subject reports detecting the stimulus is plotted as a function of stimulus amplitude, forming a relation called the psychometric function (Figure 21–2). By convention, threshold is defined as the stimulus amplitude detected in half of the trials. Thresholds can also be determined by the method of limits, in which the subject reports the intensity at which a progressively decreasing stimulus is no longer detectable or an increasing stimulus is detectable. This technique is widely used in audiology to measure hearing thresholds.

Figure 21-2 The psychometric function defines the mathematical relationship between the amplitude of a stimulus and the intensity of the sensation felt by the subject.

A. The psychometric function plots the percentage of stimuli detected by a human observer as a function of the stimulus magnitude. Threshold is defined as the stimulus intensity detected on 50% of the trials. Psychometric functions are also used to measure the just noticeable difference between stimuli that differ in intensity, frequency, or other parametric properties.

B. Detection and discrimination thresholds depend on the criteria used by individual subjects in psychophysical tasks. An ideal observer correctly detects the presence and absence of stimuli with equal probability (curve b). An observer who is told to respond to the slightest indication of a stimulus reports many false positives when no stimuli occur and has low sensory thresholds (curve a). An observer who is told to respond only when very certain that a stimulus has occurred reports more hits than false positives and has high sensory thresholds (curve c).

The measurement of sensory thresholds is a useful diagnostic technique for determining sensory function in individual modalities. An elevated threshold may signal an abnormality in sensory receptors (such as loss of hair cells in the inner ear caused by aging or exposure to very loud noise), deficits in nerve conduction properties (as in multiple sclerosis), or a lesion in sensory-processing areas of the brain. Sensory thresholds may also be altered by emotional or psychological factors related to the conditions in which stimulus detection is measured.

Psychophysical Measurements of Sensation Magnitude Employ Standardized Protocols

The lasting importance of Fechner’s work was the development of formal quantitative methods for measuring sensory performance and mathematical techniques to analyze them. Three of his methods are still widely used, either exactly as he formulated them or in a modified form: (1) the method of constant stimuli, in which a fixed set of stimuli is presented repeatedly to obtain a statistical characterization of the behavior associated with each stimulus; (2) the method of limits, described earlier; and (3) the method of adjustment or reproduction, in which a subject adjusts a second stimulus to match or reproduce the intensity of the first one.

The next major methodological and conceptual developments in psychophysics came almost a century later when S. S. Stevens introduced the technique of magnitude estimation, whereby subjects use a numerical scale to rate the intensity of the sensations experienced by stimuli of different amplitude (Figure 21–3). Verbal reports of subjective experience are widely used because they are usually reliable and repeatable. Reliability is assessed by correlations between observers rating the same stimuli; repeatability is measured by correlations between responses from the same subject to similar stimuli.

Figure 21-3 The firing rates of sensory nerves encode the stimulus magnitude. The data in the two plots suggest that the neural coding of stimulus intensity is faithfully transmitted from peripheral receptors to cortical centers that mediate conscious sensation. (Adapted, with permission, from Mount-castle, Talbot, and Kornhuber 1966.)

A. The number of action potentials per second recorded from a touch receptor in the hand is proportional to the amplitude of skin indentation. Each dot represents the response of the receptor to pressure applied by a small probe. The relationship between the neural firing rate and the pressure stimulus is linear. This receptor does not respond to stimuli weaker than 200 μm, its touch threshold.

B. Estimates made by human subjects of the magnitude of sensation produced by pressure on the hand increase linearly as a function of skin indentation. The relation between a subject’s estimate of the intensity of the stimulus and its physical strength resembles the relation between the discharge frequency of the sensory neuron and the stimulus amplitude.

Stevens generalized the method of direct verbal reporting by defining four scales of measurement—the nominal, ordinal, interval, and ratio scales—and specifying appropriate methods for analyzing data of each type. On nominal scales items have names but not rank; examples are names of colors, tastes, and smells. On ordinal scales items are ranked with a logical range and ordered relationship to each other, but the intervals between them cannot be compared meaningfully. Verbal descriptors of pain intensity are an example of an ordinal scale. When a clinician says, “On a scale from 1 to 10 in which 10 is the worst imaginable pain, how would you rate your pain?” there is no suggestion that the pain associated with an 8 is twice as intense as the pain associated with a 4 or that the difference between 5 and 6 equals the difference between 1 and 2.

On interval scales distances but not ratios between values have meaning. Counts of the number of stimuli delivered in a session or estimates of the position of an object on a grid map are examples of interval scales. In ratio scales the concepts of rank, interval, and ratio all have valid meanings. Estimates of the perceived intensity of a stimulus are treated as ratio scales. Subjects are instructed to assign a number proportional to the perceived intensity when a stimulus is detected, and to report “zero” when they feel no stimulus. Subjects typically choose their own numerical scale within a session. The values measured during an experiment are then normalized to allow comparisons of stimulus ratings between subjects. These scales permeate modern statistics and are used widely beyond the field of experimental psychology for which they were developed.

Sensations Are Quantified Using Probabilistic Statistics

Decision theory offers another approach to measurement of sensations by using statistical methods to explain the variability of subjects’ responses or false reports. When subjects are pressed to detect the weakest possible stimuli, they give many false-positive responses; that is they respond affirmatively in catch trials in which no stimulus was presented. As a result, the psychophysical thresholds measured are very low (Figure 21–2B, blue curve). Conversely, when subjects are told to avoid false positives, their perceptual thresholds become quite elevated. Trials in which strict criteria are used yield higher than normal threshold values (Figure 21–2B, red curve).

In 1927 L. L. Thurstone proposed that the variability of sensations evoked by a stimulus could be represented as a normal or Gaussian probability function with a mean (m_s) and a standard deviation (σ_s):

This allowed him to use the mathematics of probability theory and statistical tables to predict the discriminability of pairs of stimuli that differed along a physical dimension such as intensity. He proposed to equate the physical distance between the amplitudes of two stimuli to a psychological scale value of inferred loudness called the discrimination index or d′. He equated the number of correct responses (hits) and error trials (false positives when one stimulus is confused with another) with the sensory overlap of the two stimuli. This allowed him to use statistical tables of the normal probability function to calculate d′ values (Box 21–1).

Decision theory methods were first applied to psychophysical studies in 1954 by the psychologists Wilson Tanner and John Swets. They developed a series of experimental protocols for stimulus detection that allowed accurate calculation of d′ as well as techniques for measurement of subjective bias during sensory testing. Their methods were initially developed for engineers studying the detection of very weak radar pulses reflected from distant airplanes. As the engineers lowered the threshold for detection they detected more radar pulses, but their apparatus gave more false positives because it was triggered more frequently by noise. Tanner and Swets hypothesized that subjects gave false-positive responses when the sensory noise exceeded their response threshold.

Signal detection theory has been widely applied in sensory discrimination tests that require the subject to make a binary choice. Threshold measurements are a good example. In a “yes-no” experiment the beginning and end of an observation interval are cued, and the subject is required to say whether or not a signal, such as a tone, was present. We can represent trial-to-trial fluctuations of the perceived stimulus intensity and that of the silent “noise” period as two overlapping Gaussian curves. The subject says “yes” when the signal exceeds a criterion (called a decision boundary) that has been set by the subject, and “no” when it does not. When the stimulus is very weak the neural signal it evokes is very small, and there is considerable overlap between the pure noise and the stimulus signal plus noise. Hence there is no decision boundary that allows error-free responses. Nevertheless, the mathematical formulation of the probability density function allows the experimenter to compensate for subjective differences in response criteria in calculations of d′ (Box 21–1).

Sensory thresholds can also be measured using a two-alternative forced-choice protocol in which there are two observation intervals. The subject is asked whether the stimulus occurred in the first or second interval. The two-interval procedure is widely used for measuring relative intensity or sensory quality because the results obtained are more accurate than verbal judgments and the responses required are simple. Subjects can also provide nonverbal responses in such tests using levers, buttons, or other manipulanda that allow accurate measurement of decision times. Such instrumented behaviors allow neuroscientists to measure sensory processes in experimental animals by training them to use these tools to make easy sensory judgments. Such techniques can be used to probe the sensory capabilities of animal subjects as the discrimination tasks become more difficult and to investigate the underlying neural mechanisms when electrophysiological and behavioral studies are combined in the same experiment.

Decision Times Are Correlated with Cognitive Processes

Another important quantitative measure of psychophysical behavior is the reaction time, which is the time taken to perform a perceptual task. Franciscus Donders was the first, in 1865, to measure the time required to respond to stimuli. He and others found that reaction times elicited by strong stimuli are shorter than those elicited by weak stimuli. Similarly, in forced-choice tasks the time required for a decision is shorter when the stimuli are clearly distinctive in intensity or quality than when they are near the discrimination threshold. Reaction times are widely used as measures of certainty of responses in humans and animals. They are often correlated with neural activity in sensory areas of the brain and in studies of sensory-triggered motor behaviors.

Reaction times are also used to evaluate cognitive function. The tasks illustrated in Figure 21–5 were devised by Anne Treisman to investigate the mechanisms of visual pattern recognition. The subjects were asked to locate a blue cross within an array of symbols. With some patterns the blue cross seems to “pop out,” but with others the array must be carefully examined to find the blue cross.

Figure 21-5 Reaction times are used to investigate the mechanisms of pattern recognition. (Modified, with permission, from Treisman 1991.)

A. Visual stimuli used to investigate the mechanisms of pattern recognition in humans. Subjects are asked to find the blue cross in each array. The task is easy with the array at left because the blue cross is the only blue item and therefore seems to “pop out.” Detection is harder with the middle array because all of the items are the same color, but only one has two line segments. In the array at right, detection is difficult because the blue cross has the same shape as eight of the items and the same color as another eight. Each of the items must be examined individually to find the right one.

B. The time needed (reaction time) to detect the blue cross in arrays 1 and 2 is independent of the total number of objects because the items are similar in color or shape, allowing all of the objects to be scanned together (parallel search).

C. The time needed to detect the blue cross in an array of items that vary in shape or color (as in array 3) increases in proportion to the number of items when they share at least one property (color or shape in this example) because the search must examine each item (serial search). On average, the target item is found halfway through the search. It takes twice as long to determine that an item is absent because all of the objects must be examined.

One explanation for this is that when the sought-after item differs from the other elements of the array in only one property we can scan quickly the entire array (a parallel search), but when it shares two or more properties we need to examine all of the elements one-by-one (a serial search). If this hypothesis is true, the reaction time should not depend on the number of elements in an array when the sought-after item differs in only one property, but it should increase in proportion to the number of elements if we must examine them individually. That is exactly the result obtained in such experiments. The same hypothesis also predicts that it should take twice as long to determine that an item is absent because we need to examine all elements in an array before concluding that a particular one is absent.

Subjects typically locate a sought-after item halfway through the search. The slope of the curve relating the search time to the number of elements in the array shows how long it takes to examine each element. Such experiments indicate that 30 to 50 ms is required to compare each element with the target item (Figure 21–5C). Knowing what kinds of visual features allow a parallel search and the reaction time for detecting features in a serial search provides important clues to the underlying neural mechanisms.

Physical Stimuli Are Represented in the Nervous System by Means of the Sensory Code

The psychophysical methods described in the previous section provide objective techniques for analyzing sensations evoked by particular stimuli. These quantitative measures have been combined with neurophysiological techniques to study the neural mechanisms that transform sensory signals into specific percepts.

This approach to the neural coding problem was pioneered by Vernon Mountcastle in the 1960s. He showed that neurophysiological recordings from individual sensory neurons in the peripheral and central nervous system provide a statistical description of the neural activity evoked by a physical stimulus. He then tested hypotheses to determine which quantitative aspects of the neural response might correspond to psychophysical measurements in sensory tasks, and just as important, which do not.

The study of neural coding of information is fundamental to understanding how the brain works. A neural code describes the relationship between the activity in a specified neural population and its functional consequences for the operations that follow. The sensory systems provide a useful avenue to the study of neural coding in the brain because both the input and output of these systems can be precisely defined and quantified. Experimenters can control the physical stimuli provided to sensory receptors and measure the resulting sensations evoked by them using a variety of psychophysical techniques. By recording neuronal activity at various stages of sensory processing, neuroscientists attempt to decipher the codes that convey information in peripheral nerves and in the brain, and analyze the transformation of signals along pathways in the cerebral cortex. Indeed, study of the details of neural coding may lead to insight into the coding principles that underlie cognition.

When analyzing sensory experience it is important to realize that our conscious sensations differ qualitatively from the physical properties of stimuli because, as Kant and the idealists predicted, the nervous system extracts only certain pieces of information from each stimulus while ignoring others. It then interprets this information within the constraints of the brain’s intrinsic structure and previous experience. Thus we receive electromagnetic waves of different frequencies, but we see them as colors. We receive pressure waves from objects vibrating at different frequencies, but we hear sounds, words, and music. We encounter chemical compounds floating in the air or water, but we experience them as smells and tastes. Colors, tones, smells, and tastes are mental creations constructed by the brain out of sensory experience. They do not exist as such outside the brain.

The dominant research strategy in sensory neuroscience is to follow the flow of sensory information from receptors toward the cognitive centers of the brain, attempting to understand the processing mechanisms that occur at each synaptic relay and how they shape our internal representation of the external world. The neural coding of sensory information is better understood at the early stages of processing than at later stages.