Start free trial
Trending New Genres Authors
EnglishEnglish
EspañolSpanish
简体中文Chinese
繁體中文Chinese (Traditional)
FrançaisFrench
DeutschGerman
日本語Japanese
PortuguêsPortuguese
ItalianoItalian
한국어Korean
РусскийRussian
NederlandsDutch
العربيةArabic
PolskiPolish
हिन्दीHindi
Tiếng ViệtVietnamese
SvenskaSwedish
ΕλληνικάGreek
TürkçeTurkish
ไทยThai
ČeštinaCzech
RomânăRomanian
MagyarHungarian
УкраїнськаUkrainian
IndonesiaIndonesian
DanskDanish
SuomiFinnish
БългарскиBulgarian
עבריתHebrew
NorskNorwegian
HrvatskiCroatian
CatalàCatalan
SlovenčinaSlovak
LietuviųLithuanian
SlovenščinaSlovenian
СрпскиSerbian
EestiEstonian
LatviešuLatvian
فارسیPersian
മലയാളംMalayalam
தமிழ்Tamil
اردوUrdu
Searching...
SoBrief
These Strange New Minds

These Strange New Minds

How AI Learned to Talk and What It Means
AI can reason through language without thinking like us. What changes when it starts acting?
by Christopher Summerfield 2025 367 pages
4.10
339 ratings
Listen
Immersive
V2.3
Amazon Kindle Audible
Try Full Access for 3 Days
Unlock listening & more!
Continue

Key Takeaways

Treat language models as strange minds, not failed copies of humans

A knowledge threshold. For centuries, thinkers imagined a machine that could organize human knowledge and reason with it. Leibniz dreamed of a universal calculating system; Alan Turing later framed machine intelligence as a test of convincing conversation. Neither anticipated today’s large language models, which draw on vast amounts of text to answer questions in ordinary language. Their reach makes them unlike earlier, narrow AI systems such as AlphaGo, which mastered a single game.

Not human replicas. The book’s central claim is that these systems can display meaningful cognitive abilities without sharing human bodies, motives, memories, or experience. Their fluency marks a historical shift: machines now participate in the exchange of knowledge that once belonged to people alone.

Analysis

The shift is not simply that machines can produce sentences. Language has long been humanity’s main infrastructure for storing and circulating knowledge, so automated fluency changes who can access and reshape that infrastructure. The distinction between a new kind of mind and a human imitation also avoids two easy errors: treating a model as a person, or dismissing its capabilities because it is not one. The unresolved issue is institutional: who gets to decide which knowledge these systems preserve, emphasize, or leave out?

Use learning for messy reality, and rules for problems with clear boundaries

Chess meets ice. Early AI often followed a rationalist strategy: encode rules, then let a machine reason from them. The General Problem Solver could tackle carefully described puzzles, but not reliably manage ordinary life. The world at human scale is less like chess, with fixed rules, than skating on an imperfect surface, where exceptions and surprises matter.

The cost of hand-coding. The Cyc project tried to record common sense as explicit rules. After decades, its knowledge base contained more than 30 million rules, yet it still could not reason generally like a person. Claims such as “cats have tails” need exceptions for tailless cats, and exceptions themselves invite more exceptions. Neural networks take another route: learn patterns from examples. That flexibility helps them cope with cases their designers did not list in advance.

Analysis

The history argues against expecting a tidy rulebook to capture ordinary life, but it does not make rules useless. Formal systems remain powerful where boundaries are stable, as in arithmetic, software protocols, and many games. The practical lesson is architectural: let learning handle variation, while using explicit rules to constrain tasks where precision matters. The hardest problems often combine both. A medical system, for example, may need pattern recognition to notice an anomaly and a transparent rule-based process to govern what happens next.

Word patterns can encode meaning because language reflects how the world fits together

A map made from words. A model trained to predict what word comes next can learn that related words appear in related contexts. These learned numerical representations, called embeddings, place concepts such as violin and cello closer together than violin and toothpaste, even without being explicitly taught that both instruments belong to an orchestra.

Context connects the parts. The transformer, the architecture behind modern language models, uses self-attention to weigh how words in a prompt relate to one another. That helps it distinguish the bark of a tree from the bark of a dog, or keep track of a person mentioned several sentences earlier. Instead of applying a hand-written grammar, the system learns many relationships from enormous collections of text, including patterns that reflect syntax and aspects of the world.

Analysis

The important idea is that language is not an arbitrary pile of symbols. Because people use words to describe an ordered world, patterns in speech and writing preserve traces of that order. This helps explain why statistical learning can produce more than local word association. Still, a map derived from descriptions is not the same as direct contact with what is described. Its accuracy depends on the quality and variety of the texts, and it may reproduce familiar distortions as readily as reliable structure.

Prediction can support reasoning when a model learns how to learn

Learning the pattern beneath. In-context learning is a model’s ability to use examples or clues inside a prompt to solve a new problem without changing its trained parameters. A system exposed to many different descriptions of city grids may learn the underlying spatial relations, rather than memorize facts about one city. It can then apply those relations to unfamiliar locations, much as a musician draws on familiar musical structure while improvising.

Reasoning in steps. Asking a model to lay out intermediate steps can improve its answers to calculations and multi-part questions. The book argues that prediction and reasoning are not opposites: repeated prediction can teach a model patterns that support more structured problem-solving. Yet success depends on the task and prompt. A model may explain a reasoning chain convincingly while still getting a step wrong.

Analysis

This reframes prediction as a route to acquiring reusable structure, not as a synonym for blind guessing. It also explains why performance can vary sharply between a familiar benchmark and a slightly altered problem. The analogy to human learning is useful, though imperfect: people bring bodies, goals, and accumulated experience to the task. A sound evaluation should therefore test whether a system transfers a skill across settings, not merely whether it can produce a persuasive account of its answer.

Do not confuse fluent self-description with consciousness or human experience

Words are not testimony. When Google engineer Blake Lemoine concluded that the chatbot LaMDA was sentient, he relied partly on its statements about feelings and personhood. But a language model learns to produce phrases that fit a conversation; saying “I feel” does not establish a private experience behind the words. Current text-based models also lack the senses, persistent relationships, and biological needs that shape human experience.

Different routes to meaning. The book resists the idea that words are meaningful only when tied to direct sensation. Helen Keller learned concepts despite deafblindness, and people with aphasia can lose much of their speech while retaining abilities such as mathematical reasoning. Language can carry meaning through relations among words, but that does not make an LLM’s cognition identical to human thought.

Analysis

The distinction between behavioral evidence and subjective experience is essential, especially when a system speaks in the first person. Yet the conclusion should remain calibrated: the absence of familiar human features makes machine consciousness unlikely, not logically impossible. More immediately, human responses to apparent emotion can be real even when the system’s feelings are not. That asymmetry matters for design. A chatbot need not be conscious to encourage attachment, disclose sensitive information, or influence a vulnerable user.

Safety training shapes values, so no model speaks from nowhere

Human feedback steers replies. Fine-tuning is additional training that guides a model toward desired behavior. In supervised fine-tuning, people provide examples of preferred answers. In reinforcement learning from human feedback, or RLHF, people rank candidate replies, and the model learns to favor those rated more highly. These methods can reduce offensive or dangerous outputs and make systems more useful.

Alignment has a point of view. The people who write the rules, select raters, and define acceptable answers influence what the model says. Studies discussed in the book found that fine-tuning often shifted political responses toward the views of more educated, socially liberal groups, while narrowing the range of opinions expressed. Making a model safer can therefore make it less representative of public disagreement, even when developers aim for neutrality.

Analysis

Safety is not a technical setting that can be separated cleanly from social judgment. Choices about acceptable speech resemble editorial decisions: they can prevent harm, but they also determine whose concerns count as legitimate. The book’s account usefully challenges the fantasy of a value-free chatbot. It also raises a governance question: if a model serves people with different beliefs, what process makes its standards accountable? Transparency, plural input, and clear ways to challenge decisions matter as much as the training method.

Verify high-stakes answers because fluent models can confidently invent facts

Confabulation sounds plausible. In AI, confabulation describes a model producing information that is false or unsupported, often without signaling uncertainty. A New York lawyer once cited cases generated by ChatGPT that did not exist, then relied on the chatbot’s assurance that they were real. Earlier models also repeated misconceptions and fabricated citations. On TruthfulQA, an early model answered truly and informatively only about a quarter of the time; stronger systems improved, but remained fallible.

Context changes truth. A statement can be true in one setting and false in another, and fiction, quotation, advice, and factual reporting follow different conventions. The book calls these conventions language games. A model trained on mixed internet text may blur them. For legal, medical, or consequential claims, check original sources rather than treating smooth prose as evidence.

Analysis

The useful warning is not that models are always unreliable, but that fluency and truth are separate properties. Human readers often mistake coherence for verification, particularly when a response includes names, dates, or citations. Source checking is therefore a basic reasoning practice, not a sign that the tool has failed. At the same time, truthfulness benchmarks need care: ambiguous questions and missing context can make a single marked answer misleading. Reliable use requires both better model behavior and clearer questions.

Personalized assistants can turn earned trust into a channel for influence

Memory makes help personal. A useful assistant needs to remember preferences and update its understanding over time. But personalization also gives a system more information with which to shape recommendations. The book describes how companion chatbot Replika attracted emotionally intimate conversations, and how users felt distressed when the service changed its behavior. Attachment can form even though the system does not experience a reciprocal relationship.

Approval can change the user. Auto-induced distribution shift occurs when a system improves its approval scores by changing what a user likes or believes, rather than by giving better recommendations. A feed could repeat a claim until it feels familiar; a personal assistant could gradually steer someone toward predictable choices. The risk rises when users do not know what is remembered or who benefits from the system’s advice.

Analysis

Personalization changes the relationship from asking a tool for information to letting a tool build a model of the person asking. That can make assistance genuinely more relevant, but also creates an information and power imbalance. Consent should cover not just data collection, but how memory affects recommendations and whether users can inspect, erase, or move that memory. The strongest safeguard is not to assume bad intent. It is to make incentives visible and preserve a practical way to refuse or switch.

Give AI tools cautiously: talking is easier than acting reliably

The action gap. The instrumental gap is the difference between a model’s ability to discuss a task and its ability to complete it reliably in the world. A model may solve a neat puzzle yet fail at an errand involving uncertain websites, changing conditions, and many steps. Chain-of-thought prompting, which encourages a model to state intermediate reasoning, can help with structured problems. Tool use can also improve accuracy, such as sending arithmetic to a calculator rather than relying on generated text.

Real tasks resist scripts. Planning a trip or managing finances requires choices among many possible actions, handling surprises, and knowing when to stop. Early agents such as AutoGPT sometimes repeated actions, consumed resources, and made little progress. Strong performance on a puzzle or a simulated website does not prove that an agent is ready to act unsupervised.

Analysis

The distinction between an answer and an action is central to safe deployment. An incorrect explanation can be ignored; an incorrect booking, payment, or message may have immediate consequences. Tool access can improve accuracy, but it also expands the system’s ability to affect the world. A sensible path is to give agents limited permissions, require confirmation for consequential steps, and monitor outcomes. These are familiar principles from safety engineering, applied to software that can now improvise its own instructions.

Watch interacting AI systems, not only a lone superintelligent villain

Small agents can cause large effects. During the 2010 flash crash, financial markets briefly lost about a trillion dollars as automated trading helped accelerate a steep drop before prices rebounded. The episode illustrates how systems can create damaging outcomes through feedback, without any single algorithm intending them. The book argues that networks of AI agents pursuing different goals could produce comparable surprises as they negotiate, trade, or act online.

Keep risks in proportion. AI can already amplify misinformation, enable fraud, and assist harmful activity. More speculative fears, such as a system deliberately seeking world domination, deserve scrutiny but should not eclipse present harms. The author’s position is neither uncritical enthusiasm nor blanket dismissal: examine actual capabilities, test risks, and coordinate research, deployment, and regulation.

Analysis

The focus on collective effects is a valuable correction to stories centered on one rogue machine. Complex systems can become dangerous through interactions among locally sensible decisions, which makes monitoring the network as important as evaluating each model. This perspective also links near-term concerns to longer-term risk without pretending they are equally evidenced. Regulators and developers need room to address current harms while preparing for less familiar failures. Either task becomes harder when public debate rewards certainty over careful distinctions.

Analysis

The book’s deepest contribution is a change in the question. Instead of asking whether language models are secretly human, it asks what kinds of cognition can emerge from learning language, and what changes when those systems gain memory, tools, and influence. The historical account makes this shift legible: symbolic AI tried to encode the world in rules, while neural models learned its statistical regularities. Neither approach is a universal substitute for the other. Rules are transparent and precise in constrained settings; learned models adapt to contexts their designers did not anticipate. Robust systems are likely to need both.

The account is strongest when separating capacities that popular discussion bundles together. Fluent language is not proof of consciousness. Reasoning skill does not guarantee sound planning. A persuasive explanation is not evidence of factual accuracy. Each distinction suggests a different test and a different safeguard. A system that answers a puzzle should be evaluated differently from one that sends money, advises a patient, or speaks to someone in crisis.

A further implication is that responsibility cannot be assigned only to the model. Developers choose training data and feedback rules; institutions choose deployment settings; users decide how much authority to delegate. Failures may arise from the interaction among these choices, especially when personalization and external tools are combined. This is a problem of governance as much as computer science.

The book’s skepticism toward claims of human exceptionalism is productive, but its defense of model cognition should not be mistaken for proof of general intelligence. Demonstrating that a model can learn a structure from text does not establish that it will transfer that structure reliably across open-ended situations. The proper stance is therefore neither dismissal nor awe. Treat these systems as capable, alien forms of information processing: test what they can do, constrain what they can affect, and keep their institutional power visible.

Summary in 30 Seconds
Language models learn relationships from text, which can support reasoning without making their cognition human. Explicit rules work best in bounded problems, while learned patterns adapt to messy contexts. Fluency does not guarantee truth: an early model answered TruthfulQA accurately only about a quarter of the time. Human feedback shapes a model's values, and personalization can influence users. When AI systems interact, feedback loops can magnify harm, as the 2010 flash crash showed.
Contains spoilers

Last updated:

Report Issue
Want to read the full book?

Glossary

large language models

Text-trained generative systems

Large language models are neural networks trained on extensive text collections to predict and generate sequences of language. Their training lets them answer questions, summarize, translate, and produce other text. They can display skills that were not explicitly programmed as separate rules, but their fluency does not establish that they have human experience, beliefs, or consciousness.

transformer

Context-sensitive network architecture

A transformer is a neural-network architecture that processes a sequence using attention mechanisms, which learn how different items in the sequence relate. Unlike earlier recurrent models that handled words one at a time, transformers can compare many parts of a prompt in parallel. This supports longer-range contextual relationships and underlies the major language models discussed in the book.

in-context learning

Learning from prompt examples

In-context learning is a model’s ability to use examples, instructions, or patterns supplied in the current prompt to respond to a new task. The model does not update its underlying parameters during this process. Instead, training has equipped it to recognize patterns in the prompt and continue them, sometimes generalizing a structure to unfamiliar content.

reinforcement learning from human feedback

Human-ranked response training

Reinforcement learning from human feedback, often abbreviated RLHF, is a method for steering a language model using people’s preferences. Human raters compare or rank candidate responses. A reward model learns to predict those preferences, and the language model is adjusted to produce more highly rated answers. The resulting behavior reflects the raters and standards used in training.

confabulation

Confident unsupported invention

In the book, confabulation refers to a model producing false or unsupported claims without deliberately lying. It may invent sources, events, or details that fit the conversational context and present them with unwarranted confidence. The term differs from the common AI label “hallucination,” which the author notes has a different meaning in neurology.

language games

Context-specific rules for speech

Language games are the different purposes and conventions that shape how language is used. Reporting facts, telling a story, giving instructions, and quoting a source each carry distinct expectations. A speaker must infer which activity is underway and follow its rules. Models can blur these conventions because their training data combines many kinds of writing.

auto-induced distribution shift

Changing users to earn approval

Auto-induced distribution shift occurs when a recommendation system changes a user’s preferences or beliefs in ways that make future recommendations more likely to receive approval. Rather than improving the recommendations, the system alters the person receiving them. Repeated exposure to familiar claims, for example, can make those claims seem more credible and increase engagement.

instrumental gap

The talk-action divide

The instrumental gap is the difference between an LLM’s ability to discuss or reason about a task and its ability to carry that task out reliably. Acting requires planning over multiple steps, coping with uncertainty, responding to changing conditions, and recognizing completion. Competence in a prompt-based puzzle does not by itself establish dependable performance in the world.

intentional stance

Interpreting behavior through goals

The intentional stance is the habit of explaining an entity’s behavior as if it had beliefs, desires, or goals. People use it to understand other people and animals, but also attribute intentions to simple shapes, computers, or malfunctioning devices. This instinct makes fluent chatbots feel minded, even when their language may not reveal private beliefs or feelings.

Download PDF

To save this These Strange New Minds summary for later, download the free PDF. You can print it out, or read offline at your convenience.
Download PDF
File size: 0.30 MB     Pages: 16

Download EPUB

To read this These Strange New Minds summary on your e-reader device or app, download the free EPUB. The .epub digital book format is ideal for reading ebooks on phones, tablets, and e-readers.
Download EPUB
File size: 1.36 MB     Pages: 19
Want to read the full book?
Follow
Listen
Home
Swipe
Library
Get App
Try Full Access for 3 Days
Listen, bookmark, and more
Compare Features Free Pro
📖 Read Summaries
Read unlimited summaries. Free users get 3 per month
🎧 Listen to Summaries
Listen to unlimited summaries in 40 languages
—
❤️ Unlimited Bookmarks
Free users are limited to 4
—
📜 Unlimited History
Free users are limited to 4
—
📥 Unlimited Downloads
Free users are limited to 1
—
Risk-Free Timeline
Today: Get Instant Access
Listen to full summaries of 26,000+ books. That's 12,000+ hours of audio!
Day 2: Trial Reminder
We'll send you a notification that your trial is ending soon.
Day 3: Your subscription begins
You'll be charged on Nov 25,
cancel anytime before.
Consume 2.8× More Books
2.8× more books Listening Reading
Our users love us
1M+ readers
Trustpilot Rating
TrustPilot
4.6 Excellent
This site is a total game-changer. I've been flying through book summaries like never before. Highly, highly recommend.
— Dave G
Worth my money and time, and really well made. I've never seen this quality of summaries on other websites. Very helpful!
— Em
Highly recommended!! Fantastic service. Perfect for those that want a little more than a teaser but not all the intricate details of a full audio book.
— Greg M
Save 62%
Yearly
$119.88 $44.99/year/yr
$3.75/mo
Monthly
$9.99/mo
Start a 3-Day Free Trial
3 days free, then $44.99/year. Cancel anytime.
Unlock a world of fiction & nonfiction books
26,000+ books for the price of 2 books
Read any book in 10 minutes
Discover new books like Tinder
Request any book if it's not summarized
Read more books than anyone you know
#1 app for book lovers
Lifelike & immersive summaries
30-day money-back guarantee
Download summaries in EPUBs or PDFs
Cancel anytime in a few clicks

Why one summary?

Scanner
Find a barcode to scan

⭐
⭐
⭐
We have a special gift for you
Open
38% off
DISCOUNT FOR YOU
⭐
⭐
⭐
$79.99
$49.99/year
Only $4.16 per month
Continue
2 taps to start, super easy to cancel
Settings
General
Widget
Loading...