Start free trial
Trending New Genres Authors
EnglishEnglish
EspañolSpanish
简体中文Chinese
繁體中文Chinese (Traditional)
FrançaisFrench
DeutschGerman
日本語Japanese
PortuguêsPortuguese
ItalianoItalian
한국어Korean
РусскийRussian
NederlandsDutch
العربيةArabic
PolskiPolish
हिन्दीHindi
Tiếng ViệtVietnamese
SvenskaSwedish
ΕλληνικάGreek
TürkçeTurkish
ไทยThai
ČeštinaCzech
RomânăRomanian
MagyarHungarian
УкраїнськаUkrainian
IndonesiaIndonesian
DanskDanish
SuomiFinnish
БългарскиBulgarian
עבריתHebrew
NorskNorwegian
HrvatskiCroatian
CatalàCatalan
SlovenčinaSlovak
LietuviųLithuanian
SlovenščinaSlovenian
СрпскиSerbian
EestiEstonian
LatviešuLatvian
فارسیPersian
മലയാളംMalayalam
தமிழ்Tamil
اردوUrdu
Searching...
SoBrief
The Deep Learning Revolution

The Deep Learning Revolution

Neural networks were written off for decades. How a stubborn minority revived them and changed AI.
by Terrence J. Sejnowski 2018 352 pages
3.72
500+ ratings
Listen
Immersive
V2.3
Amazon Kindle Audible
Try Full Access for 3 Days
Unlock listening & more!
Continue

Key Takeaways

Let data teach machines what hand-written rules cannot anticipate

Learning beats scripting. Deep learning is a form of machine learning in which layered networks learn patterns from examples rather than relying on programmers to specify every rule. In 2005, Sebastian Thrun’s team trained Stanley, a self-driving car, by driving it through desert terrain and letting it learn how sensory inputs predicted steering. It completed a difficult 132-mile course. The same shift improved translation: instead of matching isolated word combinations, a deep network learned relationships across whole sentences.

One method, many tasks. Learning algorithms can be reused across problems, from recognizing speech to screening medical images. Their success depends on abundant examples, computing power, and persistence. The promise is not that machines understand the world as people do, but that they can extract useful structure from experience where hand-coded rules become unwieldy.

Analysis

The central distinction is between specifying a solution and specifying a process for learning solutions. That shift makes software adaptable, but it also changes where errors enter: a hand-built rule can be inspected directly, while a learned pattern depends on the data and training objective. The book's examples make a strong case for learning in messy domains, yet the approach is not automatically superior. Where rules are stable, explicit, and safety-critical, transparent programming may remain easier to verify.

Do not mistake effortless human skills for easy computational problems

Perception is hard. Early AI researchers found it easier to program computers to prove mathematical theorems than to recognize ordinary objects. A project to make a robot stack blocks looked simple, but even a controlled toy world proved difficult to describe with rules. Real scenes add changes in lighting, shape, position, and occlusion. Human vision conceals this complexity because it works so quickly and automatically.

A long winter. Frank Rosenblatt's perceptron could classify patterns, but a single layer could only separate categories with a simple linear boundary. Marvin Minsky and Seymour Papert's mathematical critique highlighted this limitation, and many researchers took it as a reason to abandon neural networks. A small group continued working on multilayer learning. It took decades of better algorithms, data, and computers for their approach to succeed at practical vision and speech tasks.

Analysis

The contrast between theorem proving and vision exposes a bias in how intelligence is judged. Deliberate reasoning feels impressive, while perception feels basic, yet evolution has had far longer to refine the latter. This resembles a broader engineering lesson: performance that looks effortless can hide enormous computational work. The historical account also cautions against treating a model's limitation as a verdict on an entire research program. Still, persistence alone is not evidence of eventual success; the crucial change was that new methods and resources made testable progress possible.

Train networks to discover useful internal representations

Learning hidden features. A perceptron adjusts connection weights when its classification is wrong. More capable networks add hidden layers, where combinations of inputs can become useful features. Backpropagation of errors, or backprop, is a learning method that sends information about output mistakes backward through the network so weights at earlier layers can be adjusted.

A voice from letters. In the 1980s, Terry Sejnowski and Charles Rosenberg trained NETtalk to turn written English into speech. Given a window of letters, it learned to predict the sound associated with the center letter. It initially produced babble, then increasingly accurate pronunciation, including for unfamiliar words. The same general approach helped predict protein structures from amino-acid sequences, showing that learned representations could help in fields far beyond language.

Analysis

Representation learning matters because real problems rarely arrive with their important features neatly labeled. A network can build intermediate patterns that make a difficult distinction easier, much as a scientist develops a useful measurement before testing a theory. NETtalk also complicates the idea that language depends only on explicit rules: statistical regularities can produce convincing pronunciation. That is not a complete account of language, however. Pronunciation performance does not establish that a network captures meaning, intention, or the social setting in which words are used.

Give deep networks enough layers, data, and computing power to scale

A breakthrough in vision. Convolutional networks apply small filters across an image, learning local features and combining them through successive layers. Their organization resembles the visual system, where early neurons respond to simple edges and later areas respond to more complex patterns. Pooling helps preserve recognition when an object shifts position.

Scale changed the results. In 2012, Geoffrey Hinton and his students used AlexNet to classify images from ImageNet, a database containing more than 15 million labeled images. Its error rate improved dramatically over earlier approaches, and subsequent systems pushed benchmark errors lower still. The networks were not simply cleverer than their predecessors. They could be trained on far more examples using far more computing power. The Neural Information Processing Systems community helped sustain research across fields during the long period before these conditions arrived.

Analysis

The lesson is not merely that bigger models win. Scale became useful because the network architecture and training method could make use of additional examples and computation. This is an empirical engineering achievement, not proof that every increase in size improves every task. Benchmark scores also reflect the categories chosen and the data collected. The resemblance between convolutional networks and visual cortex is scientifically suggestive, but it does not show that brains use the same learning procedure or that artificial systems reproduce biological vision in full.

Choose learning methods that match the structure of the problem

Learning without labels. Independent component analysis, or ICA, is an unsupervised method that separates statistically independent sources from mixed data. In the cocktail-party problem, several microphones record overlapping voices; ICA can separate the voices without being told what each speaker said. Applied to brain recordings, it can distinguish neural signals from eye movements and other artifacts.

Learning with uncertainty. A Boltzmann machine is a network of interacting units that uses probabilities and hidden units to learn patterns. Its training compares activity when examples guide the network with activity when it runs freely. This lets it learn internal structure and generate plausible examples, rather than simply return a label. Together, these methods illustrate that learning is not one universal recipe: different tasks call for different signals, objectives, and assumptions.

Analysis

These examples broaden the meaning of learning beyond memorizing labeled answers. ICA is useful when the sources are hidden but their mixtures contain structure; a probabilistic network is useful when several interpretations remain possible. The common thread is that the algorithm exploits regularities in data. That strength has a boundary: statistical independence is not the same as causal independence, and a generated sample can look plausible without being true. A method's assumptions should therefore be treated as part of its result, not as invisible technical details.

Use delayed rewards to teach machines sequences of decisions

Credit across time. Temporal difference learning is a reinforcement-learning method that updates estimates of future reward by comparing what was expected with what happened next. It addresses the temporal credit assignment problem: when a reward arrives after many actions, which earlier choices helped produce it?

Learning by playing. Gerald Tesauro's TD-Gammon learned backgammon through self-play. It began with largely random moves, but after many games reached a level comparable to top human players and developed strategies that challenged accepted wisdom. The same basic reward-learning principle appears in the brain: dopamine neurons signal differences between expected and received rewards. AlphaGo combined reward learning with deep networks and self-play to master Go, a game whose vast number of possible positions defeats exhaustive search.

Analysis

Delayed reward turns experience into strategy without requiring a teacher to label every move. The connection to dopamine offers a valuable bridge between artificial learning and neuroscience, especially because it predicts how reward signals change with expectation. But the reward is only a proxy for what matters. A system optimized for a poorly chosen target can become adept at achieving that target while missing the human purpose behind it. Reward design is therefore an ethical and scientific choice, not just a parameter setting.

Expect AI to reorganize work, not simply erase it

Jobs change shape. Automation often removes routine tasks while creating new work around systems, maintenance, and services. The book notes that ATMs reduced some teller duties without ending teller employment; customer-facing work and ATM repair expanded. Amazon's growth also created jobs in distribution and delivery even as online retail pressured local stores.

Learning must continue. Machine learning can support diagnosis, legal document review, and personalized education, but better technology does not guarantee adoption. A sleep-monitoring system described in the book offered faster, cheaper analysis, yet clinics had little incentive to replace a billable process. As occupations change, workers need new skills throughout their lives. Online courses can widen access, while digital tutors may free teachers to focus on encouragement and individual support.

Analysis

The useful question is not whether a task can be automated, but who gains time, income, and bargaining power when it is. The book's examples show why technical superiority alone does not dislodge an incumbent: reimbursement, regulation, and established workflows shape adoption. Claims that automation creates replacement jobs deserve the same scrutiny as claims that it destroys them. The transition depends on whether new roles are accessible to displaced workers and whether education systems can provide training at the pace industries change.

Audit training data and objectives before trusting algorithmic judgments

Data carry history. A neural network learns from the examples and goals supplied to it. Facial-recognition systems trained on databases containing many more white faces can make more errors on underrepresented groups. A loan model can also use location or other proxies to reproduce discrimination, even if it never receives a protected characteristic directly.

Objectives carry choices. If a system is trained only to maximize profit, it will favor whatever signals improve profit, not necessarily fairness or public welfare. The book argues that fairness must be included in the objective, but someone still has to decide how competing aims should be weighted. Medical tools offer a more hopeful model of partnership: a network and human experts can make complementary errors, producing better results together than either alone.

Analysis

Algorithmic fairness is not achieved simply by removing sensitive fields. Proxy variables, uneven samples, and institutional history can preserve patterns of exclusion. Nor is fairness a single measurable target that can be installed without debate. The strength of the book's framing is that it locates responsibility in data and objective design rather than treating a model as a neutral judge. Human collaboration can improve outcomes, but human reviewers also need the authority and time to challenge machine recommendations.

Treat superhuman performance as a skill, not proof of a general mind

Many intelligences. AlphaGo's ability to defeat champions demonstrated extraordinary skill within Go, not competence across every human activity. The book describes intelligence as a collection of abilities shaped by different environments: bats use echolocation, while bacteria can move toward food by responding to chemical patterns. A single ranking of intelligence misses these specialized adaptations.

The explanation remains open. Researchers can identify neural correlates of consciousness, patterns of brain activity associated with awareness, but correlation alone does not explain how experience arises. Deep networks can perform striking tasks while remaining difficult to interpret, and the biological brain includes bodies, motivation, and learning systems beyond pattern recognition. The book's deeper claim is that AI can reveal principles of intelligence without settling what intelligence or consciousness ultimately are.

Analysis

A domain-specific champion can be more capable than a person at one task without possessing a general human-like mind. This distinction helps avoid both inflated promises and reflexive dismissal. Comparing intelligence across species also requires care: echolocation and chemical navigation solve different problems, so human-centered tests will systematically undervalue them. The consciousness discussion adds an important restraint. Behavioral success is evidence of capability, but it is not by itself evidence of subjective experience or a complete account of the mechanisms that produce it.

Analysis

The book's most persuasive achievement is its double narrative. It tells how neural networks moved from the margins of artificial intelligence into practical use, and how that history changed the questions researchers ask about brains. Stories of the long research winter give technical progress human stakes: the breakthroughs did not arrive as a sudden flash of genius, but when persistent ideas met improved algorithms, abundant data, and much stronger computers. The examples, from NETtalk to AlphaGo, show how a shared learning principle can appear in very different tasks.

The argument is strongest when it distinguishes learning from hand-coded rules and demonstrates the engineering consequences. It is more tentative when the similarity between artificial networks and biological systems is treated as a clue to brain function. A convolutional network can resemble the visual hierarchy without proving that the cortex learns through backpropagation. Indeed, the book notes that biological feedback does not neatly supply the error signals that backprop requires. The distinction matters: AI can be inspired by biology and useful as a model without being a literal replica.

The account also makes clear that technical performance is embedded in institutions. A diagnostic system can be more accurate yet remain unused when payment structures reward existing routines. A model can be powerful yet reproduce the biases in its training data or the priorities in its objective. These are not side issues to the science. They determine which capabilities become products, who benefits, and which errors are tolerated.

The book's lasting contribution is therefore neither a promise of artificial general intelligence nor a claim that the brain has been decoded. It offers a research strategy: build systems that learn, study what they can and cannot do, and use those results to sharpen questions about biological intelligence. The central discipline is humility. Human intuition about what is easy, what is intelligent, and what is impossible has repeatedly been a poor guide to both machines and nature.

Summary in 30 Seconds
Machines learn patterns from examples better than they follow hand-written rules. Neural networks were dismissed after single-layer limits were exposed, then returned once hidden layers, error backpropagation, huge datasets, and fast hardware arrived. Delayed-reward learning taught programs to master games. Automation tends to reshape jobs rather than erase them. Training data and objectives carry bias, so audit them. Superhuman skill in one task is not general intelligence, and consciousness remains unexplained.
Contains spoilers

Last updated:

Report Issue

Review Summary

3.72 out of 5
Average of 500+ ratings from Goodreads and Amazon.

The Deep Learning Revolution receives mixed reviews (3.74/5). Critics argue it's more memoir than technical guide, offering limited deep learning instruction and focusing heavily on Sejnowski's career and colleagues. However, supporters praise it as an excellent historical overview of neural networks' evolution, detailing key pioneers and developments from the 1980s onward. Readers appreciate the balance between accessible storytelling and technical context, though some found explanations unclear. The book effectively traces how AI shifted from rule-based systems to neural networks, though it requires patience and some technical background to fully appreciate.

Your rating:
4.28
88 ratings
Want to read the full book?

Glossary

Deep learning

Learning through layered networks

A branch of machine learning that trains neural networks with multiple layers to extract patterns from data. Early layers can represent simple features, while later layers combine them into more complex patterns useful for tasks such as image recognition, speech recognition, and translation.

Perceptron

Single-layer pattern classifier

A simple neural-network model that combines weighted inputs and classifies a pattern into one of two categories. Its learning algorithm adjusts the weights when it makes an error. A perceptron can solve problems whose categories can be separated by a linear boundary, but not more complex distinctions.

Backpropagation of errors

Backward error-based weight updates

A learning method for multilayer networks. It compares a network's output with the desired answer, then propagates information about the error backward through the layers to calculate how connection weights should change. Repeating this process across examples can improve the network's performance.

Independent component analysis

Separating mixed statistical sources

An unsupervised learning method that separates signals into components that are as statistically independent as possible. It can, for example, separate voices mixed together in microphone recordings without knowing what the speakers said. It has also been used to distinguish brain signals from artifacts in EEG recordings.

Boltzmann machine

Probabilistic network learning patterns

A neural-network model made of interacting units whose states are probabilistic. It uses hidden units to learn internal patterns and can be trained by comparing network activity when examples are constrained with activity when the network runs freely. Its learned structure can support both classification and generation of examples.

Temporal difference learning

Updating estimates from reward differences

A reinforcement-learning method that updates an estimate of future reward by comparing a current prediction with a revised estimate based on the next state and any reward received. It helps assign credit to earlier actions even when the final reward arrives much later.

Convolutional network

Network for spatial pattern learning

A neural network architecture that applies learned filters across local regions of an image or other structured input. Successive layers combine these local features into increasingly complex patterns. Pooling can help preserve recognition when a feature shifts position within the input.

Generative adversarial network

Competing networks generate examples

A system with two neural networks trained in competition. One network generates synthetic examples, while the other tries to distinguish them from real examples. Their opposing objectives can produce increasingly realistic samples, including images, without requiring every training example to carry a category label.

Spike-timing-dependent plasticity

Synaptic change based on spike order

A form of synaptic plasticity in which the change in connection strength depends on the timing of spikes in two connected neurons. Repeated input spikes arriving shortly before an output spike can strengthen a synapse, while the reverse timing can weaken it. The book presents this as a possible mechanism for learning sequences.

Predictive coding

Perception through prediction errors

A model of perception in which higher levels of the brain generate predictions about incoming sensory information. Lower levels pass forward the mismatches between prediction and input, allowing the system to update its interpretation. The approach connects prior expectations with ongoing sensory signals.

Neural correlates of consciousness

Brain activity linked to awareness

The neural structures and patterns of activity associated with conscious experience. Researchers seek these correlates by comparing brain activity with what a person reports perceiving. Finding a correlation is an important step, but it does not by itself establish that the activity causes the experience or explain how awareness arises.

About the Author

Terrence Joseph Sejnowski is the Francis Crick Professor at The Salk Institute for Biological Studies, directing the Computational Neurobiology Laboratory. As a Howard Hughes Medical Institute Investigator, his pioneering research spans neural networks and computational neuroscience. At UC San Diego, he serves as Professor of Biological Sciences with adjunct positions in Neurosciences, Psychology, Cognitive Science, and Computer Science. He directs the Institute for Neural Computation and the Crick-Jacobs Center for Theoretical and Computational Biology. His collaborations with figures like Geoffrey Hinton helped establish foundational deep learning concepts, including work on Boltzmann machines and backpropagation algorithms.

Download PDF

To save this The Deep Learning Revolution summary for later, download the free PDF. You can print it out, or read offline at your convenience.
Download PDF
File size: 0.34 MB     Pages: 16

Download EPUB

To read this The Deep Learning Revolution summary on your e-reader device or app, download the free EPUB. The .epub digital book format is ideal for reading ebooks on phones, tablets, and e-readers.
Download EPUB
File size: 1.43 MB     Pages: 19
Want to read the full book?
Follow
Listen1 mins
Home
Swipe
Library
Get App
Try Full Access for 3 Days
Listen, bookmark, and more
Compare Features Free Pro
📖 Read Summaries
Read unlimited summaries. Free users get 3 per month
🎧 Listen to Summaries
Listen to unlimited summaries in 40 languages
—
❤️ Unlimited Bookmarks
Free users are limited to 4
—
📜 Unlimited History
Free users are limited to 4
—
📥 Unlimited Downloads
Free users are limited to 1
—
Risk-Free Timeline
Today: Get Instant Access
Listen to full summaries of 26,000+ books. That's 12,000+ hours of audio!
Day 2: Trial Reminder
We'll send you a notification that your trial is ending soon.
Day 3: Your subscription begins
You'll be charged on Nov 24,
cancel anytime before.
Consume 2.8× More Books
2.8× more books Listening Reading
Our users love us
1M+ readers
Trustpilot Rating
TrustPilot
4.6 Excellent
This site is a total game-changer. I've been flying through book summaries like never before. Highly, highly recommend.
— Dave G
Worth my money and time, and really well made. I've never seen this quality of summaries on other websites. Very helpful!
— Em
Highly recommended!! Fantastic service. Perfect for those that want a little more than a teaser but not all the intricate details of a full audio book.
— Greg M
Save 62%
Yearly
$119.88 $44.99/year/yr
$3.75/mo
Monthly
$9.99/mo
Start a 3-Day Free Trial
3 days free, then $44.99/year. Cancel anytime.
Unlock a world of fiction & nonfiction books
26,000+ books for the price of 2 books
Read any book in 10 minutes
Discover new books like Tinder
Request any book if it's not summarized
Read more books than anyone you know
#1 app for book lovers
Lifelike & immersive summaries
30-day money-back guarantee
Download summaries in EPUBs or PDFs
Cancel anytime in a few clicks

Why one summary?

Scanner
Find a barcode to scan

⭐
⭐
⭐
We have a special gift for you
Open
38% off
DISCOUNT FOR YOU
⭐
⭐
⭐
$79.99
$49.99/year
Only $4.16 per month
Continue
2 taps to start, super easy to cancel
Settings
General
Widget
Loading...