The Tales We Spin: Navigating Cognitive Biases in AI and Us
Written on
Chapter 1: Understanding Our Complex Minds
The human brain is a remarkably intricate structure, composed of billions of neurons that form trillions of connections. This complex, nonlinear dynamical system is arguably the most sophisticated entity in the universe, capable of performing a vast range of tasks that allow us to adapt and thrive in a continually changing world.
While this complexity may be well-known, its implications are often underestimated. Our identities and behaviors are deeply intertwined with the brain's intricate workings. This article examines the tension between the elusive processes that shape us and our ongoing quest for coherent narratives about our thoughts and actions. This struggle mirrors the concept of explainability in artificial intelligence research.
Psychological phenomena, such as cognitive biases, arise from this conflict, with cognitive dissonance and confabulation highlighting the limits of our self-understanding.
“You didn’t notice, but your brain did.”
Over a century after Sigmund Freud introduced the idea of the subconscious, the notion that we are not fully in control of our actions has been somewhat integrated into our self-perception. Given the brain's complexities, it is only natural that we often lack awareness of our motivations. The twentieth century marked a significant shift in acknowledging that much more occurs within us than what we consciously recognize. We navigate our lives largely on autopilot, directing our conscious attention only to urgent matters, as discussed in my earlier article on attention.
The brain processes immense amounts of information every second, while our conscious awareness is limited to a mere fraction of it, most of which has been heavily filtered. A myriad of processes—whether motor, cognitive, or emotional—operate automatically and beyond our active control.
Moreover, decision-making is not as straightforward as it seems. Research shows that if we exclude emotional components, decision-making becomes nearly impossible, as demonstrated in Antonio Damasio’s book, The Strange Order of Things.
Our decisions are often influenced by our bodily sensations and the gut-brain connection, which utilizes distributed, parallel processing through specialized types of neurons. The phrase "trust your gut" has a much more scientific basis than one might think.
Danny Kahneman and Amos Tversky highlight how biases from priming or anchoring can significantly impact our decision-making under uncertainty, typically without our awareness. Remarkably, even when we recognize these biases, we remain susceptible to their influence.
Section 1.1: AI and the Challenge of Explainability
Researchers in artificial intelligence and computational neuroscience grapple with a similar issue, encapsulated in the term explainability. How can we elucidate the emerging intelligence of AI systems and ensure these systems can articulate their decision-making processes while elucidating their information processing?
Modern machine learning techniques enable computers to develop their algorithms autonomously based on given inputs and desired outputs, as detailed in Pedro Domingos’ The Master Algorithm.
As a result, understanding machine behaviors becomes increasingly challenging, even as their performance improves across various tasks like image classification, speech recognition, and playing games like Go. Often, machines excel without human intervention or direct instruction, leading to Frederick Jelinek's famous remark during his work on speech recognition software: "Every time I fire a linguist, the performance of the speech recognizer goes up."
In large neural networks, computations are distributed throughout the entire system, and in deep networks, through multiple hidden layers with nonlinear activation functions. This distribution makes tracing specific computations nearly impossible, complicating our understanding of how these networks achieve tasks such as distinguishing between cats and dogs.
As explored in my previous articles, this approach contrasts sharply with the symbolic manipulations employed by traditional Turing machines, which are based on rational, step-by-step reasoning. Thus, we may encounter fundamental limitations in understanding information processing in complex distributed systems.
Section 1.2: The Importance of Narrative
It is not surprising that humans face their own challenges in explainability regarding the brain's information processing. Behaviorists have long treated the brain as a black box, focusing on input-output relationships. However, we consistently seek narratives that clarify our actions and motivations. But how reliable are these narratives?
Human beings evolved from animals, which function perfectly well without constant storytelling about their motivations. Yet, we developed the ability to communicate, both with others and internally. Social and sexual pressures likely spurred the evolution of advanced cognitive abilities such as language. A significant portion of our cognitive resources is devoted to social interactions and gossip about those around us.
The neocortex, where our language capabilities reside, is the most recent addition to our neural framework. It constitutes a large portion of our brain's mass, but it does not always dominate or "know" all that happens within us.
This leads to a functional perspective of the brain: when one region seeks information about another, signals must travel between regions. This process is neither efficient nor necessary for all brain functions, further complicating our understanding of brain activity. Consequently, the regions attempting to provide explanations may lack access to crucial information, as seen in split-brain patients.
Despite these challenges, we persist in attempting to explain our actions. Maintaining a coherent narrative is beneficial and appears essential for social functionality. However, we must recognize that not all our thoughts and narratives hold truth; indeed, many are far from it.
The Stories We Tell Ourselves
“If a given idea has been held in the human mind for many generations, as almost all our common ideas have, it takes sincere and continued effort to remove it; and if it is one of the oldest we have in stock, one of the big, common, unquestioned world ideas, vast is the labor of those who seek to change it.”
? Charlotte Perkins Gilman
Humans possess a strong inclination toward narratives. We interpret the world through stories, constantly seeking agency and causality. This desire leads us to construct coherent narratives about ourselves, attributing our behaviors to specific reasons.
However, as we have seen, the brain engages in extensive distributed, parallel processing, making it challenging to articulate and understand our motivations. Consequently, we often resort to speculation regarding the underlying causes of our actions.
Furthermore, addressing contradictory beliefs is inherently difficult. This results in many individuals carrying significant inconsistencies in their thoughts, often requiring rigorous critical thinking to resolve. Evolutionarily, maintaining a perfectly coherent worldview may not be essential for survival, allowing our brains to tolerate contradictions unless they become disruptive.
Stories wield the power to shape our perceptions, often without our conscious awareness. The brain continually predicts sensory inputs, leading to variations in how we perceive the world based on our expectations—cognitive scientists refer to this phenomenon as a controlled hallucination. As we strive to simplify the complex realities of life into digestible narratives, these stories can significantly alter our perception of reality.
Cognitive Biases and Dissonance
“He had very few doubts, and when the facts contradicted his views on life, he shut his eyes in disapproval.”
? Hermann Hesse, The Fairy Tales of Hermann Hesse
In 1957, Leo Festinger introduced the term cognitive dissonance to describe the discomfort experienced when confronted with information that contradicts our beliefs. Often, this discomfort operates subconsciously, prompting us to devise alternative explanations for unwelcome events or dismiss them altogether.
Our instinct to avoid cognitive dissonance keeps us anchored in our established worldviews, leading us to subconsciously overlook contradictory information or reframe distressing thoughts. Self-justification is a common coping mechanism for cognitive dissonance, manifesting both internally and externally.
In Mistakes Were Made, But Not By Me, Carol Tavris illustrates numerous instances where avoidance of cognitive dissonance has resulted in severe consequences, including moral declines in fascist societies, misguided actions by the US government during the Vietnam War, and biases in the legal system regarding exonerating evidence.
Many decisions, particularly moral ones, are made without a clear understanding of their foundations. This lack of awareness often leads to excessive self-justification, as individuals adjust their worldviews to restore coherence and minimize cognitive dissonance.
A classic example is the smoker who may subconsciously recognize the irrationality of their habit yet finds ways to justify it—“I just enjoy it” or “It’s not that harmful.” Many can relate to this situation, regardless of the behavior in question.
Split-Brain Patients and Confabulation
The brain’s attempts to justify its actions can sometimes reach a level of absurdity, particularly evident in split-brain patients whose corpus callosum—connecting the left and right hemispheres—has been severed (Michael Gazzaniga, a pioneer in this research, explains this concept in a brief video).
Without communication between the hemispheres, each processes information independently. The left hemisphere, housing the language center, cannot access information processed in the right visual field. When a stimulus is shown only to the left visual field, the right hemisphere reacts but cannot articulate the reasons behind it. Therefore, when asked about their reaction, the patient fabricates an explanation on the spot.
This phenomenon illustrates how we often create narratives to rationalize our behavior, even when lacking crucial information. Instead of acknowledging uncertainty, we tend to invent justifications, retroactively framing these explanations as confident insights—a process known as confabulation, which we all engage in regularly.
What Can We Learn from This?
While exploring cognitive biases can be unsettling (with Wikipedia listing 157 varieties), it is said that knowledge is the first step toward wisdom. Observing the ongoing commentary we maintain about ourselves in light of our complex inner lives can foster greater awareness of our biases and confabulations.
I strive to recognize instances where I catch myself rationalizing my behavior or struggling to justify my actions. However, I remain realistic about my capacity to overcome these biases. They are deeply ingrained within us, and even Daniel Kahneman acknowledges the challenges in addressing them.
Lastly, the AI community should also consider these insights. Our biases reveal a tendency to overestimate our ability to explain our actions, which raises concerns regarding the capacity of machines to articulate their behaviors. While we desire transparency from AI systems, we must be cautious not to anthropomorphize them too much.
After all, we wouldn’t want robotic overlords engaged in endless confabulation.
The first video, The Stories We Tell Ourselves | David Mathews | TEDxBinghamtonUniversity, explores how our personal narratives shape our understanding and decision-making processes.
The second video, How the Stories We Tell Ourselves Can Make or Break Us - Letters From Esther, examines the profound impact of our self-narratives on our lives and choices.