Published June 17, 2025 | Version v1
Conference paper Open

A New Cognitive Ethnography

Authors/Creators

  • 1. Department of Cognitive Science, University of California San Diego

Description

A paradigm shift is underway in the behavioral sciences. Recent advances in computational theories of mind are reshaping the theoretical foundations of cognitive science. Deep learning and predictive processing are driving incredible advances in the capabilities of artificial intelligence as well as providing a theoretical basis for the unification of many previously dispersed sub-fields. Over the past five decades, I have developed a method for the study of cognition in the wild. This method identifies empirical observations of the fine-scale details of naturally occurring ongoing activity as instances of theoretical concepts. As theories of cognition evolved and as the sites of my ethnographic fieldwork changed, the cognitive ethnography method continued to bring cognitive theory to the interpretation of real-world activity. Theorizing and speculation about how real-world cognitive processes might appear if the mind were assumed to be a predictive processing system have already begun. Where previous work has theorized the applicability of predictive processing to real-world cognition, the analyses presented here offer a first empirical glimpse of what cognition viewed through this framework actually entails in the fine structure of naturally occurring activity. In this paper, I propose a novel theoretical framework that combines predictive processing with multimodality and the continuous coupling of organism to environment. I use this framework as the underlying theory in the cognitive ethnographic analysis of two dozen vignettes of real-world activity.

Files

a-new-cognitive-ethnography.pdf

Files (19.5 MB)

Name Size Download all
md5:a8d6968e52d38eb3578d8f8f35f63ae2
19.5 MB Preview Download

Additional details

References

  • Agha, A. (2007). Language and Social Relations. Cambridge University Press.
  • Alač, M., & Hutchins, E. (2004). I see what you are saying: Action as cognition in fMRI brain mapping practice. Journal of Cognition and Culture, 4(3), 629–661.
  • Barbasi, A. (2002). Linked: the new science of networks. Perseus Publishing.
  • Barsalou, L. (2010). Grounded Cognition: Past, Present, and Future. In Topics in Cognitive Science (pp. 716–724).
  • Bateson, G. (1972). Steps to an Ecology of Mind. University of Chicago Press.
  • Beer, R. (2008). The Dynamics of Brain-Body-World systems: a status report. In P. Calvo & A. Gomila (Eds.), Handbook of Cognitive Science: An embodied approach (pp. 99–120). Elsevier.
  • Benetti, S., Ferrari, A., & Pavani, F. (2023). Multimodal processing in face-to-face interactions: A bridging link between psycholinguistics and sensory neuroscience. Front. Hum. Neurosci, 17.
  • Bourdieu, P. (1977). Outline of a Theory of Practice. Cambridge University Press.
  • Brooks, R. (1991). Intelligence without representation. Artificial Intelligence, 47, 139–159.
  • Calvo, P., & Gomila, T. (2008). Handbook of cognitive science: An embodied approach. Elsevier.
  • Campbell, M., & Cunnington, R. (2017). More than an Imitation Game: Top-down modulation of the Human Mirror System. Neuroscience and Behavioral, Reviews(75, 195–202.
  • Cangelosi, A., & Parisi, D. (2002). Simulating the evolution of language. Springer-Verlag.
  • Churchland, P., Ramachandran, V., & Sejnowski, T. (1994). A Critique of Pure Vision. In C. Koch & J. Davis (Eds.), Large-scale neuronal theories of the brain (pp. 23–60). MIT Press.
  • Clark, A. (1998). Being There: Putting brain, body, and world together again. MIT Press.
  • Clark, A. (2001). Mindware: An Introduction to the Philosophy of Cognitive Science. Oxford University Press.
  • Clark, A. (2016). Surfing Uncertainty: Prediction, action, and the embodied mind. Oxford University Press.
  • Clark, A. (2023). The Experience Machine: How our minds predict and shape reality. Pantheon Books.
  • Clark, A., & Chalmers, D. (1998). The Extended Mind (pp. 7–19). Analysis.
  • Coco, M., & Dale, R. (2014). Cross-recurrence quantification analysis of categorical and continuous time series: an R package. Frontiers in Psychology, 5(510).
  • Cole, M. (1996). Cultural Psychology: A once and future discipline. Basic Books.
  • Daniels, H., Cole, M., & Wertsch, J. (2007). The Cambridge companion to Vygotsky. Cambridge University Press.
  • Dasen, P. (1972). Cross-Cultural Piagetian Research: A Summary. Journal of Cross-Cultural Psychology, 3(1), 23–39.
  • Dreyfus, H. (1982). Husserl: Intentionality and cognitive science. MIT Press.
  • Duncker, K. (1945). On problem solving. Psychological Monographs, 58, 1–270.
  • Dupuy, J. (2000). The mechanization of the mind: On the origins of cognitive science. Princeton University Press.
  • Enfield, N. (2006). Social consequences of common ground. In N. Enfield & S. Levinson (Eds.), Roots of human sociality: Culture, cognition and interaction (pp. 399–430). Berg Publishers.
  • Engel, A., Stewart, O. G., & Paulo, E. (2010). Directive Minds: How dynamics shapes cognition (J., Ed.; pp. 219–244). MIT Press.
  • Fauconnier, G. (1997). Mappings in Thought and Language. Cambridge University Press.
  • Fauconnier, G., & Turner, M. (2002). How We Think. Basic Books.
  • Flash, T., & Berthoz, A. (2021). Space-time geometries for motion and perception in the brain and the arts. Nature.
  • Fouse, A., Weibel, N., Hutchins, E., & Hollan, J. (2011). ChronoViz: A system for supporting navigation of time-coded data. Extended Abstracts of CHI 2011, SIGCHI Conference on Human Factors in Computing Systems.
  • Friston, K. (2008). Heirarchical Models in the Brain. PLOS Computational Biology.
  • Gallese, V., & Lakoff, G. (2005). The Brain's Concepts: The role of the sensori-motor system in conceptual knowledge. Cognitive Neuropsychology, 22(3), 455–479.
  • Gentner, D., & Stevens, A. (1983). Mental Models. Lawrence Erlbaum Assoc.
  • Gibbs, R. (2006). Embodiment and Cognitive Science. Cambridge University Press.
  • Gibson, J. (1986). The Ecological Approach to Visual Perception. Lawrence Erlbaum Associates.
  • Gilbert, J., Blaser, M., Caporaso, J., Jansson, J., Lynch, S., & Knight, R. (2018). Current Understanding of the Human Microbiome. Nat Med, 24(4), 392–400.
  • Glucksberg, S. (1964). Functional fixedness: Problem solution as a function of observing responses. Psychonomic Science, 1, 117–118.
  • Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. Harvard University Press.
  • Goldin-Meadow, S. (2006). Meeting other minds through gesture: How children use their hands to reinvent language and distribute cognition. In N. Enfield & S. Levinson (Eds.), Roots of human sociality: Culture, cognition and interaction (pp. 353–373). Berg Publishers.
  • Goodwin, C. (1994). Professional Vision. American Anthropologist, 96(3), 606–633.
  • Goodwin, C. (2000). Action and Embodiment within Situated Human Interaction. Journal of Pragmatics.
  • Goodwin, C. (2007). Environmentally Coupled Gestures. In S. Duncan, J. Cassell, & E. Levy (Eds.), Gesture and the Dynamic Dimension of Language (pp. 195–212). John Benjamins.
  • Goody J. (1977). The Domestication of the Savage Mind. Cambridge University Press.
  • Greeno, J. (1998). The situativity of knowing, learning, and research. American Psychologist, 53(1), 5–26.
  • Greeno, J., & Moore, J. (1993). Situativity and symbols: Response to Vera and Simon. Cognitive Science, 17, 49–59.
  • Grice, H. (1975). Logic and Conversation. In P. Cole & J. Morgan (Eds.), Syntax and Semantics (pp. 43–58). Academic Press.
  • Havelange, V., Lenay, C., & Stewart, J. (2003). Les représentations: mémoire externe et objets techniques. In Intellectica (pp. 115–131).
  • Havelange, V., Lenay, C., & Stewart, J. (2003). Les représentations: Mémoire externe et objets techniques. Intellectica, 35, 115–131.
  • Hazlehurst, B., & Hutchins, E. (1998). The emergence of propositions from the coordination of talk and action in a shared world (Vol. 13, pp. 373–424).
  • Heidegger, M. (1962). Being and time. Harper.
  • Holder, B., & Hutchins, E. (2001). What Pilots learn about autoflight while flying on the line. Proceedings of the 11th International Symposium on Aviation Psychology.
  • Hollan, J., Hutchins, E., & Weitzman, L. (1984). STEAMER: An interactive inspectable simulation-based training system. AI Magazine, 5(2), 15–27.
  • Holland, D., & Quinn, N. (1987). Cultural Models in Language and Thought. Cambridge University Press.
  • Hurford, J., Studdert-Kennedy, M., & Knight, C. (1998). Approaches to the evolution of language: Social and cognitive bases. Cambridge University Press.
  • Hurley, S. (1998). Consciousness in action. Harvard University Press.
  • Hutchins, E. (1980). Culture and Inference: A Trobriand Case Study. Harvard University Press.
  • Hutchins, E. (1995). Cognition in the Wild. MIT Press.
  • Hutchins, E. (1995). How a cockpit remembers its speeds. Cognitive Science, 19, 265–288.
  • Hutchins, E. (2005). Material Anchors for Conceptual Blends. Journal of Pragmatics, 37, 1555–1577.
  • Hutchins, E. (2006). The distributed cognition perspective on human interaction. In N. Enfield & S. Levinson (Eds.), Roots of human sociality: Culture, cognition and interaction (pp. 375–398). Berg Publishers.
  • Hutchins, E. (2007). Measuring change in pilots' conceptual understanding of autoflight. Proc. of the 14th International Symposium on Aviation Psychology, 281–286.
  • Hutchins, E. (2010). Cognitive Ecology. In Topics in Cognitive Science (pp. 705–715).
  • Hutchins, E. (2010). Enaction, imagination, and insight. In J. Stewart, O. Gappene, E. Paolo, & Enaction (Eds.), Towards a New Paradigm in Cognitive Science (pp. 425–450). MIT Press.
  • Hutchins, E. (2012). Concepts in practice as sources of order. Mind, Culture and Activity, 19(3), 314–323.
  • Hutchins, E., & Hazlehurst, B. (1995). How to invent a lexicon: The development of shared symbols in interaction. In N. Gilbert & R. Conte (Eds.), Artificial societies: The computer simulation of social life (pp. 157–189). UCL Press.
  • Hutchins, E., & Hazlehurst, B. (2002). Auto-organization and emergence of shared language structure. In A. Cangelosi & D. Parisi (Eds.), Simulating the evolution of language (pp. 279–305). Springer-Verlag.
  • Hutchins, E., & Holder, B. (2000). Conceptual models for understanding an encounter with a mountain wave. Proceedings of HCI-Aero 2000.
  • Hutchins, E., & Johnson, C. (2009). Modeling the emergence of language as an embodied collective cognitive activity. Topics in Cognitive Science, 1, 523–546.
  • Hutchins, E., & Klausen, T. (1986). Distributed cognition in an airline cockpit. In Y. E. Middleton (Ed.), Cognition and communication at work (pp. 15–34). Cambridge University Press.
  • Hutchins, E., & McCandless, T. (1982). Manboard: A graphic display program for training relative motion concepts [Technical Report TN 82-10,]. Navy Personnel Research.
  • Hutchins, E., & Nomura, S. (2011). Collaborative Construction of Multimodal Utterances. In J. Streeck, C. Goodwin, & C. LeBaron (Eds.), Embodied Interaction: Language and body in the material world (pp. 29–43). Cambridge University Press.
  • Hutchins, E., & Palen, L. (1997). Constructing Meaning from Space, Gesture, and Speech. In L. Resnick, R. Saljo, C. Pontecorvo, B. Burge, & tools Discourse (Eds.), and reasoning: Essays on situated cognition (pp. 23–40). Springer-Verlag.
  • Hutchins, E., Middleton, C., & Newsome, W. (2009). Conceptualizing spatial relations in flight training. Proceedings of the 15th International Symposium on Aviation Psychology.
  • Hutchins, E., Nomura, S., & Holder, B. (2006). The ecology of language practices in worldwide airline flight deck operations. Proceedings of the 28th Annual Conference of the Cognitive Science Society, 363–368.
  • Hutchins, E., Weibel, N., Emmenegger, C., Fouse, A., & Holder, B. (2013). An integrative approach to understanding flight crew activity. Journal of Cognitive Engineering and Decision Making, 353–376.
  • Ingold, T. (2000). The perception of the environment: Essays in livelihood, dwelling, and skill. Routledge.
  • Johnson, M. (1987). The Body in the Mind: the bodily basis of meaning, imagination, and reason. University of Chicago Press.
  • Kelso, S. (1995). Dynamical Patterns. MIT Press.
  • Kugler, P., & Turvey, M. (1987). Information, Natural Law, and the Self-assembly of Rhythmic Movement. Earlbaum.
  • Kutas, M., & Federmeier, K. (1998). Minding the body. Psychophysiology, 35, 135–150.
  • Lakoff, G., & Nuñez, R. (2000). Where Mathematics Comes From: How the embodied mind brings mathematics into being. Basic Books.
  • Langacker, R. (1987). Foundations of Cognitive Grammar (Vol. 1). Stanford University Press.
  • Latour, B. (2005). Reassembling the Social: An introduction to Actor-Network-Theory. Oxford University Press.
  • Lave, J. (1988). Cognition in practice: Mind, mathematics and culture in everyday life. Cambridge University Press.
  • Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge University Press.
  • Lee, D. D. (1949). Being and value in a primitive culture. Journal of Philosophy, 48, 401–415.
  • Malinowski, B. (1965). Coral Gardens and their Magic: The language of magic and gardening (Vol. 2). Indiana University Press.
  • Malone T., & Bernstein M. (2015). Handbook of Collective Intelligence. MIT Press.
  • Marchese, F. (2011). Exploring the Origins of Tables for Information Visualization. 15th International Conference on Information Visualisation, 395–402.
  • Maturana, H., & Varela, F. (1987). The Tree of Knowledge: Biological roots of human understanding. New Science Library.
  • McNeill, D. (1992). Hand and Mind: What Gestures Reveal about Thought. University of Chicago Press.
  • McNeill, D. (2005). Gesture and thought. University of Chicago Press.
  • Minsky M., & Papert S. (1988). Perceptrons. MIT Press.
  • Murphy, K. (2004). Imagination as Joint Activity: The case of architectural interaction. Mind, Culture, and Activity, 11(4), 267–278.
  • Núñez, R., & Sweetser, E. (2010). With the Future Behind Them: Convergent Evidence From Aymara Language and Gesture in the Crosslinguistic Comparison of Spatial Construals of Time. Cognitive Science, 30(3), 401–450.
  • Newell, A., & Simon, H. (1972). Human Problem Solving. Prentice Hall.
  • Noë, A. (2004). Action in perception. MIT Press.
  • Nomura, S., Hutchins, E., & Holder, B. (2006). The Uses of Paper in Commercial Airline Flight Operations. Proceedings of the 2006 ACM Conference on Computer Supported Cooperative Work, CSCW 2006, 249–258.
  • O'Regan, J., & Noë, A. (2002). A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences, 24(5).
  • Palmer, E., Hutchins, E., Ritter, R., & Cleemput, I. (1993). Altitude deviations: breakdowns of an error tolerant system. NASA Technical Memorandum, 108788.
  • Pea, R. (1996). Practices of distributed intelligence and designs for education. In G. Salomon (Ed.), Distributed cognitions: Psychological and educational considerations (pp. 47–87). Cambridge University Press.
  • Pfeifer, R., & Bongard, J. (2007). How the body shapes the way we think: A new view of intelligence. MIT Press.
  • Port, R., & Gelder, T. (1995). Mind as Motion: Explorations in the dynamics of cognition. MIT Press.
  • Rogoff, B. (2003). The cultural nature of human development. Oxford University Press.
  • Rowlands, M. (2006). Body language: Representation in action. MIT Press.
  • Rumelhart, D. (1975). Notes on a schema for stories (D. G. Collins, Representation, & Understanding, Eds.). Academic Press.
  • Rumelhart, D., & McClelland, J. (1986). Parallel Distributed Processing. MIT Press.
  • Rumelhart, D., Smolenski, P., McClelland, J., & Hinton, G. (1986). Schemata and sequential thought processes in PDP models. In J. McClelland & D. Rumelhart (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition (Vol. 2). MIT Press.
  • Schegloff, E. (1984). On some gestures' relation to talk. In M. Atkinson & J. Heritage (Eds.), Structures of Social Action (pp. 266–296). Cambridge University Press.
  • Senft, G. (1986). Kilivila: The Language of the Trobriand Islanders. Mouton de Gruyter.
  • Simon, H., & Kaplan, C. (1989). Foundations of Cognitive Science (M. Posner, Ed.). MIT Press.
  • Smith, L. (2005). Action alters shape perception. Cognitive Science.
  • Spivey, M. (2007). The Continuity of Mind. Oxford University Press.
  • Sporns, O., & Zwi, D. (2004). The small world of the cerebral cortex. Neuroinformatics, 2, 145–162.
  • Stewart, J., Gapenne, O., & Paolo, E. (2010). Enaction: Toward a new Paradigm for Cognitive Science. MIT Press.
  • Streeck, J. (1993). Gesture as Communication I: Its Coordination with Gaze and Speech. Communication Monographs, 60(4), 275–299.
  • Streeck, J., Goodwin, C., & LeBaron, C. (2011). Embodied Interaction: Language and body in the material world. Cambridge University Press.
  • Suchman, L. (1987). Plans and Situated Actions: The problem of human-machine communication. Cambridge University Press.
  • Suleyman, M., & Bhaskar, M. (2023). The Coming Wave: Technology, Power, and the Twenty-First Century's Greatest Dilemma. Crown Books.
  • Sunstein, C. (2007). Infotopia: How many minds produce knowledge. Oxford University Press.
  • Surowiecki, J. (2004). The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations. Doubleday.
  • Talmy, L. (1995). Fictive motion in language and perception. In P. Bloom, M. Peterson, L. Nadel, & M. Garrett (Eds.), Language and Space (pp. 307–384). MIT Press.
  • Thelen, E., & Smith, L. (1994). A Dynamical Systems Approach to the Development of Cognition and Action. MIT Press.
  • Theureau, J. (2015). Le Cours D'Action: L'Enaction & L'Expérience. Octares Editions.
  • Thompson, E. (2007). Mind in life: Biology, phenomenology, and the sciences of mind. Harvard University Press.
  • Turnbaugh, P., Ley, R., Hamady, M., Fraser-Liggett, C., Knight, R., & Gordon, J. (2007). The Human Microbiome Project. Nature, 449(18), 804–810.
  • Varela, F., Thompson, E., & Rosch, E. (1991). The embodied mind: Cognitive science and human experience. MIT Press.
  • Vygotsky, L. (1978). Mind in Society. Harvard University Press.
  • Weibel, N., Fouse, A., Emmenegger, C., Kimmich, S., & Hutchins, E. (2012). Let's look at the cockpit: Exploring mobile eye-tracking for observational research on the flight deck. Proc. ETRA 2012, ACM Symposium on Eye Tracking Research and Applications.
  • Wertsch, J. (1985). Vygotsky and the Social Formation of Mind. Harvard University Press.
  • Wilson, S., Saygin, A., Sereno, M., & Iacoboni, M. (2004). Listening to speech activates motor areas involved in speech production. Nature Neuroscience, 701–702.