Approaches to the Study of Brain Evolution
Jon H. Kaas
3.1 Introduction
The variability in structure and function across the brains of present‐day vertebrates ranges from very simple nervous systems to an astonishing variety of complex forms that allow great differences in behavioral specializations and range (see Kaas, 2007). Understanding how these different nervous systems evolved from those of ancient ancestors both addresses a natural curiosity and helps us understand the relationship of brains to behavior. Such understanding may change the way we think about our place in the biological world and how we plan our future. This review considers some of the strategies we use to study brain evolution. Some of us are especially concerned about the evolution of the large and very impressive human brain, and the question of human origins has intrigued humans across cultures from at least the emergence of modern humans. While this question can be a focus for some, it is important to have a broader understanding of nervous systems, as comparisons reveal general principles, and provide another way of evaluating theories of brain organization and function for any extant species, including our own. In reality, much of what we know about human brains, for example, comes from comparative studies of other primates and, more broadly, other animals, together with assumptions about the significance of shared features and how human brains evolved. As we readily make such assumptions to support the validity of various lines of research—the current emphasis on mouse brains, for example—a broad understanding of brain evolution can be very helpful in guiding interpretations of results. For practical reasons, this review focuses on ways of studying the evolution of brains in mammals. There is no obvious starting point, but a reasonable start is with the brains of the first mammals and their fossil kin. The methods of study discussed here can be applied to other lines of evolution.
3.2 The Structure of the Mammalian Radiation
Studies of brain evolution largely depend on comparisons of traits observed in the brains of extant animals and on inferences made about brains from skull endocasts from the fossil record. For such studies, it is very important to have an understanding of the phylogeny of the group of interest, mammals in this case, that is reliable. Here we are fortunate in that the use of comparative molecular data has produced powerful evidence and considerable agreement on phylogenetic relationships, especially for mammals (e.g., Bininda‐Emonds et al., 2007; Murphy, Pevzner, & O’Brien, 2004). In addition, the fossil record has continued to improve, and has added critical time points to the understood molecular evidence for diversification and radiation.
All present‐day mammals appear to have evolved from a single source, with subsequent offspring differing from parents in various ways, gradually accumulating differences and ultimately producing the great mammalian radiation with six major branches and many sub‐branches (see Figure 3.1), leading to over 250 extant species of primates alone, as well as the thousands of species of the especially successful bat and rodent radiations. Traditional concepts of how mammals are related to each other via common ancestors at various times in the past were inferred from similarities and differences in body anatomy, often from bones as these were usually all that was preserved in extinct species. But there are long‐recognized problems with this approach as members of different lines of evolution may come to resemble each other as they adapt to similar environments (convergent evolution), and separate lines of evolution may continue to resemble each other as they retain ancestral features or specialize in similar ways (parallel evolution). For example, some investigators incorrectly concluded that members of one of the two major branches of the bat radiation (megabats or fruit bats) were really primates because of proposed similarities and not at all closely related to other bats, the microbats or echolocating bats, even though this meant that microbats and megabats would have evolved wings separately (see Kaas & Preuss, 1993 for discussion). As another example of confusion arising from existence of similar traits, it has been extremely difficult to determine the evolutionary relationships of a number of small, insect‐eating mammals because they closely resemble each other. As early mammals were small and insect‐eating, many of these shared morphological traits were retained or reacquired in these separate groups of mammals that we now know are not closely related, although they had been grouped as “insectivores.”
Although studies of anatomical features in extant species and fossils have provided much valuable information about evolutionary trees, modern phylogenetic trees are largely based on genetic similarities and differences stemming from studies of genetic material and gene products. As for the use of anatomical features, the more gene products considered in a study the better, for genes change (mutate) at different rates and are under variable selection pressure. In general, the phylogenetic trees based on genetic relationships are considered to be much more accurate than those based on a more limited range of anatomical features, while the times of branch points of different lines of evolution can be determined by referring to fossil evidence that can be dated and then estimating rates of genetic change. Recently, the oldest known skeleton of a eutherian mammal has been dated at about 125 million years old (Ji et al., 2002). Obviously, the origin of any species would predate the oldest discovered fossil.
The present consensus is that present‐day mammals all stem from an early mammalian ancestor some 250–280 million years ago (mya). Over time, mammals evolved to form six major surviving branches, the rare monotremes (platypus and echidna) that retained the primitive trait of egg‐laying, the marsupials (roughly 6–7 percent of extant species) with primitive, short gestation periods, and four branches of eutherian mammals with placentas and prolonged gestation. Within each of these major branches or clades, there have been many subsequent branches. According to modern revisions, microbats and megabats remain together, but in Laurasiatheria rather than in Euarchontoglires with primates. Shrews, hedgehogs, and moles remain as members of the original order Insectivora, now renamed Eulipotyphia in the Laurasiatherian Superorder, while tenrecs and golden moles are in Afrosoricida of a different Superorder, Afrotheria. Tree shrews (Scandentia) and elephant shrews (Macroscelidea), once thought to be closely related to insectivores, are now in separate superorders (Euarchontoglires and Afrotheria).
This revised phylogeny with times of origin is essential to accurately reconstructing the evolution of mammalian brains using a comparative, cladistic approach. A clade is any group of organisms that have all originated from a common ancestor. The common ancestor defines the clade. Thus, a clade is one branch of the evolutionary tree of any size. All mammals form a clade, and all primates form a smaller clade. The Insectivores of former phylogenetic trees do not form a clade. Likewise, fish or reptiles are useful terms, but they do not denote a clade, as all members based on a common ancestor are not included. Fish do not include all the vertebrates that evolved from fish, and reptiles do not include the birds that evolved from reptiles. Correctly identifying members of a clade is important because comparisons across members of clades are used to infer those traits that the common ancestor passed on to the extant members of the clade. For example, a variable feature of brains, the corpus callosum, is found in all eutherian (placental) mammals, but not in any noneutherian mammals (monotremes and marsupials). Thus, we infer that the corpus callosum emerged as a feature or character of the common ancestor of all eutherian mammals, but not before the branching of the marsupials from the eutherians. A misclassification of some marsupials, such as the marsupial mole, as eutherian would confound the process of identifying ancestral traits, as did the recent claim that megabats are primates, a claim that has now been rebutted by molecular (genetic) evidence.
3.3 What We Learn from the Fossil Record
Fossils are useful as they provide an opportunity to look directly into the past. What we learn about the evolution of mammalian brains from the fossils of extinct species is limited by the fact that the soft tissues of the body, especially the brain, seldom fossilize. Fortunately, brains of mammals fill the skull rather tightly, so that the skull cavity reflects the size and shape of the brain and, for larger brains, even the locations of some of the major fissures of the brain may be revealed, and such fissures may mark functional divisions of the brain (Kaas, 2009; Radinsky, 1976; Ronan et al., 2014; Welker, 1990; Zilles, Palomeros, Gallagher, & Amunts, 2013).
From the endocasts and reconstruction of the shapes of brains from the inner surface of brain cavities of fossilized skulls, we have a good overall concept of how brains changed in size and shape from early mammals over 150 million years ago to modern humans and other extant mammals (de Sousa & Wood, 2007; Jerison, 1973, 2007; Kielan‐Jaworowska et al., 2004). Early mammals were typically small and they had small brains, even for their small body size. Brains, like other body organs, increase in size with body size, but not at the same rate (Jerison, 1973; 2007). Thus, they tend to occupy proportionately less of the total body mass as bodies get bigger. Some mammals have larger brains than expected for body size, and this feature is called encephalization. The empirical quotient of the deviation from expected brain size based on body size is the encephalization quotient, with higher values suggesting greater brain capacity. However, at least in primates, body size is not a good predictor of brain size, as some primates have evolved smaller body sizes without a corresponding decrease in brain size, while other species have much larger males than females without a corresponding difference in brain size (Herculano‐Houzel & Kaas, 2011). The forebrain of early mammals was dominated by proportionately large olfactory structures, the olfactory bulb and the olfactory, piriform cortex. A small cap of neocortex was separated from piriform cortex by a shallow rhinal sulcus (dimple), which sometimes could be seen in the skull endocasts (Kielan‐Jaworoska,