Skip to main content

Open questions: Tackling Darwin’s “instincts”: the genetic basis of behavioral evolution


All of us have marveled at the remarkable diversity of animal behaviors in nature.

None of us has much idea of how these have evolved.

Like other inherited phenotypes, many behavioral traits of animals—predatory instincts, courtship rituals, and shelter building, to name but a few—have a genetic basis (Fig. 1). Genes, of course, don’t control behavior directly, but encode the vast array of molecules that establish the connectivity and physiology of the nervous system (to make no mention of those that form the tissues and organs in which neural circuits are embedded). What is the genetic basis by which seemingly complex behaviors have evolved?

Fig. 1.
figure 1

Evolving behaviors. Examples of genetically defined behaviors that differ between closely related species. Top: Differences in the use of wing “rowing” in the male courtship routine of drosophilid species, illustrated by chronophotography [17]. Bottom: Differences in burrow construction between Peromyscus deer mice species [10] (image credit: Brant Peterson and Hopi Hoekstra)

This grand question covers a multitude of issues. Is there a subset of genes that have “special” roles in dictating behavioral evolution, or does behavior evolve along many different trajectories? What is the relative contribution of structural and regulatory genetic changes (influencing protein function and gene expression, respectively) to this process? What part(s) of the nervous system do such genetic changes impact: sensory input channels, central processing circuit and/or in locomotor pathways that directly control actions? Do answers to these questions vary among taxa?

Why are these questions timely?

Technical and intellectual reasons. We’re getting very good at observing and quantifying behavior. Cameras and computers do much of the job, revealing behaviors that previously escaped detection by the human eye or which would be too laborious to measure manually [1]. With large-scale population surveys and comparative genomics, we’re also more attuned to the intra- and interspecific behavioral [2] and genetic variation [3] that exists around the world. We also simply now know a lot about the genetic and neural basis of behavior of well-established genetic model systems, such as the mouse Mus musculus, the fly Drosophila melanogaster and the worm Caenorhabditis elegans. These models provide valuable points of comparison with related species that display obvious (and presumably evolutionarily significant) variants in their behaviors. Importantly, with new genome editing methods (such as CRISPR/Cas9 [4]) and optogenetic and thermogenetic tools for turning on and off neurons at will [5], we’re potentially able to manipulate molecules and circuits with precision in such non-traditional model species. This ability will allow us to determine causal relationships—and not simply correlations—between genetic and behavioral variation.

Why should neurobiologists care about these questions?

Despite impressive advances, we’re still a long way from understanding the genetic and neural basis of even simple behaviors. Seymour Benzer’s seminal work in the 1960s with D. melanogaster showed how forward genetics can identify key components of specific behaviors, such as courtship or circadian activity [6]. Despite the doors this work opened, the opposing view of Benzer’s contemporary, Jerry Hirsch, that behaviors are too complex to be reducible to the action of single (or a few) genes, is of course largely true. In flies and worms, large-scale genetic screens for loci underlying, for example, embryonic segmentation or axon guidance have been extraordinarily fruitful [7]. By contrast, behavioral screens are much harder to perform and, because there are many uninteresting ways in which a particular behavior can be disrupted, truly informative mutations are likely to be rare. Indeed, most characterized examples affect genes that have non-pleiotropic functions in peripheral sensory systems, such as specific olfactory receptors [2, 8]. Thus, a comparative, evolutionary approach that takes advantage of naturally occurring, phenotypically consequent genetic variants can offer a complementary way to identify molecular determinants of behavior. These may open new doors into explorations of the underlying neural circuits.

Why should evolutionary biologists care about these questions?

Although behavioral traits are critical to animals’ survival and reproduction, the challenge of high-throughput and robust quantification of these often-complex phenotypes has hampered exploration of their evolution, especially by comparison with studies of, for example, morphological characteristics [9]. This is changing: the current intersection of phenotyping and genomic technologies is rapidly increasing our ability to link behavioral variation with specific regions of the genome, through quantitative trait locus and association mapping [2]. Importantly, these advances not only allow determination of the genetic architecture of behavioral variations (as in burrow building by Peromyscus mice; Fig. 1) [10]), but can also allow us to nail the causal gene(s) and genetic variant(s), as exemplified by studies of drosophilid courtship song [11] and C. elegans’ sensitivity to environmental gases [12, 13].

These tools can also enrich our ability to understand the evolutionary processes that govern behavioral divergence. Reverse genetic approaches, in which regions of the genome are identified because they carry signatures of selection, are often a starting point for this endeavor [14]. While the behavioral impact of such candidate regions has rarely been characterized experimentally, deepened knowledge of nervous systems and technical advances now make hypothesis generation and testing much more easily achieved. The increased ability to integrate evolutionary processes (e.g., mutation, selection, genetic drift) with neurobiological mechanisms will translate into a richer understanding of behavioral divergence. This, in turn, may help us understand to what extent selective versus non-selective forces are responsible for the evolution of behavior.

What questions (and challenges) are there in the long-term?

Most initial traction on the problem of behavioral evolution is likely to come from studies of simple innate actions that can be reproduced in the laboratory. However, once efficient assays are established, we can begin to tackle behaviors that are shaped by internal state and experience, in essence trying to understand the evolutionary plasticity of neural plasticity. What remains a looming challenge is to place our favorite behavioral differences within an ecological context and understand the impact of these differences on fitness. This will be difficult for many reasons: fitness effects may be extremely small (though still impactful on an evolutionary scale) and many of our current model species’ behaviors in nature are poorly appreciated. Insights may thus only come from large-scale, field-based studies. Serious consideration should also be given to concerted efforts to establish new model species [15] which retain a close relationship with current model systems but have better-understood ecologies.

In his landmark book [16], Darwin opened his chapter on “Instinct” with a characteristically cautious statement: “I have nothing to do with the origin of the primary mental powers, any more than I have with that of life itself”. Over 150 years later, we might now be cautiously optimistic that an understanding of the evolution of at least some “mental powers” is within our reach.


  1. Robie AA, Seagraves KM, Egnor SE, Branson K. Machine vision methods for analyzing social interactions. J Exp Biol. 2017;220(Pt 1):25–34.

    Article  PubMed  Google Scholar 

  2. Bendesky A, Bargmann CI. Genetic contributions to behavioural diversity at the gene-environment interface. Nat Rev Genet. 2011;12(12):809–20.

    CAS  PubMed  Google Scholar 

  3. Savolainen O, Lascoux M, Merila J. Ecological genomics of local adaptation. Nat Rev Genet. 2013;14(11):807–20.

    Article  CAS  PubMed  Google Scholar 

  4. Hsu PD, Lander ES, Zhang F. Development and applications of CRISPR-Cas9 for genome engineering. Cell. 2014;157(6):1262–78.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Bernstein JG, Garrity PA, Boyden ES. Optogenetics and thermogenetics: technologies for controlling the activity of targeted cells within intact neural circuits. Curr Opin Neurobiol. 2012;22(1):61–71.

    Article  CAS  PubMed  Google Scholar 

  6. Weiner J. Time, love, memory: a great biologist and his quest for the origins of behavior. New York. London: Knopf; 1999.

    Google Scholar 

  7. St Johnston D. The art and design of genetic screens: Drosophila melanogaster. Nat Rev Genet. 2002;3(3):176–88.

    Article  CAS  PubMed  Google Scholar 

  8. Cande J, Prud’homme B, Gompel N. Smells like evolution: the role of chemoreceptor evolution in behavioral change. Curr Opin Neurobiol. 2012;23(1):152–8.

    Article  PubMed  Google Scholar 

  9. Stern DL. Evolution, development and the predictable genome. Greenwood Village (Colorado): Roberts and Company Publishers; 2011.

    Google Scholar 

  10. Hu CK, Hoekstra HE. Peromyscus burrowing: a model system for behavioral evolution. Semin Cell Dev Biol. 2017;61:107–14.

    Article  PubMed  Google Scholar 

  11. Ding Y, Berrocal A, Morita T, Longden KD, Stern DL. Natural courtship song variation caused by an intronic retroelement in an ion channel gene. Nature. 2016;536(7616):329–32.

    Article  CAS  PubMed  Google Scholar 

  12. Persson A, Gross E, Laurent P, Busch KE, Bretes H, de Bono M. Natural variation in a neural globin tunes oxygen sensing in wild Caenorhabditis elegans. Nature. 2009;458(7241):1030–3.

    Article  CAS  PubMed  Google Scholar 

  13. McGrath PT, Rockman MV, Zimmer M, Jang H, Macosko EZ, Kruglyak L, Bargmann CI. Quantitative mapping of a digenic behavioral trait implicates globin variation in C. elegans sensory behaviors. Neuron. 2009;61(5):692–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Vitti JJ, Grossman SR, Sabeti PC. Detecting natural selection in genomic data. Annu Rev Genet. 2013;47:97–120.

    Article  CAS  PubMed  Google Scholar 

  15. Sommer RJ. The future of evo-devo: model systems and evolutionary theory. Nat Rev Genet. 2009;10(6):416–22.

    CAS  PubMed  Google Scholar 

  16. Darwin C. On the origin of species by means of natural selection, or the preservation of favoured races in the struggle for life. London: John Murray; 1859.

    Book  Google Scholar 

  17. Cande J, Andolfatto P, Prud’homme B, Stern DL, Gompel N. Evolution of multiple additive loci caused divergence between Drosophila yakuba and D. santomea in wing rowing during male courtship. PLoS One. 2012;7(8):e43888.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references


We are grateful to Nicolas Gompel, Benjamin Prud’homme, Brant Peterson, and Hopi Hoekstra for providing images. We thank Thomas Auer, Margarida Cardoso-Moreira, Sophie Martin, and Lucia Prieto-Godino for comments on the manuscript. JRA was supported by a post-doctoral fellowship from Novartis Foundation for medical-biological Research (12A14). Research in RB’s laboratory is supported by the University of Lausanne, an ERC Consolidator Grant (615094), and the Swiss National Science Foundation.

Authors’ contributions

JRA and RB wrote the article. Both authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Richard Benton.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arguello, J.R., Benton, R. Open questions: Tackling Darwin’s “instincts”: the genetic basis of behavioral evolution. BMC Biol 15, 26 (2017).

Download citation

  • Published:

  • DOI: