Key Takeaways
Treat important intangibles as uncertainty with observable consequences
Measurement means learning. Hubbard defines measurement as observations that reduce uncertainty in a quantitative way. It does not require an exact answer, or even a conventional numeric object: a probability that a lawsuit will succeed can be a measurement if it improves what is known. This reframes the usual objection that quality, reputation, or security is too vague to measure. If an idea matters, it should affect something observable, directly or indirectly.
Indirect evidence counts. Eratosthenes inferred Earth's circumference from noon shadows in Alexandria and Syene. A 7.2-degree difference implied the cities were separated by one-fiftieth of Earth's circumference. His estimate came within roughly 3 percent of the modern value. He did not need to circle the planet; he needed observations that revealed something about it.
This definition usefully separates measurement from certainty, a distinction that matters in science as much as management. A thermometer is valuable despite error; a business estimate can be valuable for the same reason. The harder question is whether an observable consequence is a valid proxy for the concept. A rise in referrals may indicate stronger reputation, but could also reflect discounts or a competitor's failure. Measurement begins with observability, but trustworthy interpretation requires testing the link between proxy and phenomenon.
Define the decision before choosing a metric
Start with the choice. A measurement has decision value only if it could change an action. Asking for the value of clean water or information technology may disguise a false choice, since few decision makers are considering abandoning either altogether. The practical question may be whether a specific upgrade, policy, or investment is worth its cost.
Make vague terms concrete. The clarification chain is: if something matters, it must be detectable; if detectable, it can be observed in some amount; and an amount can be measured. At the Department of Veterans Affairs, managers initially treated security as a general aspiration. They clarified it as reducing the frequency or severity of defined events, including virus attacks and unauthorized access, then linked those events to costs such as lost work and legal exposure. Of 99 model variables for water-system upgrades, only one warranted further measurement.
Decision-first thinking is a useful defense against metric accumulation. It asks managers to specify what an indicator is for, the alternatives it informs, and the point at which action changes. This resembles good experimental design: a measure is chosen to discriminate among explanations or options, rather than simply because data are available. The method also exposes political choices hidden inside definitions. A team can agree on what counts as a security incident while still disagreeing about which losses deserve priority.
Break difficult estimates into smaller questions you can actually answer
Decompose before surrendering. A Fermi decomposition estimates an uncertain quantity by expressing it as a product or sum of simpler parts. To estimate piano tuners in Chicago, Enrico Fermi prompted students to consider population, households with pianos, tuning frequency, and the number of pianos a tuner could handle. The resulting estimate was imperfect, but it replaced helplessness with a reasoned range and exposed which assumptions mattered most.
The structure improves judgment. Experiments by Donald MacGregor and J. Scott Armstrong found that decomposition especially improved highly uncertain estimates. In some tests, breaking a question into no more than five components reduced estimation error by factors of 10 or even 100. Sometimes the breakdown itself reveals that experts know more than their first broad estimate suggested. It can also identify the specific component where fresh observation would pay off.
Decomposition is powerful because it turns a global guess into a set of inspectable claims. Each part can be challenged, sourced, or measured separately, which makes assumptions visible to colleagues instead of burying them in a single confident number. The approach also resembles fault-tree analysis in engineering, where a system-level failure is traced to component failures. Its danger is false precision: several weak estimates multiplied together can look impressively structured while remaining weak. The component ranges and their relationships still need honest scrutiny.
Calibrate your confidence before trusting expert forecasts
Confidence can be tested. Calibration means that judgments given a stated probability come true at about that frequency. Someone who says 90 percent certainty on many questions should be right about 90 percent of the time. In Hubbard's training data, 927 participants initially placed the correct answer inside their 90 percent ranges only about 53 percent of the time, on average, a sign of widespread overconfidence.
Practice changes performance. Training uses repeated questions, feedback, explicit consideration of reasons an estimate might be wrong, and equivalent bets. In an equivalent bet, a person compares a payoff conditional on their estimate being right with a random payoff at the stated probability. Hubbard reports that roughly 80 percent of participants reached ideal calibration after training. A 1997 test of IT forecasts also found trained analysts better calibrated than untrained CIO clients.
Calibration shifts debate from who sounds authoritative to whose confidence tracks outcomes. It is particularly useful where forecasts recur, because performance can be checked and improved over time. The equivalent-bet exercise is a clever way to make probability feel consequential rather than decorative. Still, calibration on trivia does not guarantee expertise in a new domain, nor does it ensure that the underlying questions are well framed. It disciplines confidence, not the entire forecasting process. Domain knowledge and sound definitions remain essential.
Measure the uncertainties that could change your decision
Price the chance of being wrong. Expected opportunity loss, or EOL, is the probability of choosing incorrectly multiplied by the cost of that mistake. The expected value of information, or EVI, is the reduction in EOL that a measurement is expected to produce. The expected value of perfect information, or EVPI, sets the upper limit on what perfect knowledge could be worth, so an actual study should cost less.
Attention often goes elsewhere. Across more than 80 analyses, Hubbard found that only a small fraction of thousands of modeled variables justified further measurement. Yet high-value variables were often neglected, while familiar, easy-to-measure items received attention. In one insurance firm, intensive function-point estimates for software effort were no closer to actual effort than managers' initial estimates. The effort was large; its information value was effectively nil.
This economic lens explains why a less precise measurement can be more useful than a polished report: value depends on whether uncertainty affects a choice, not on how sophisticated the analysis looks. It also gives teams a principled stopping rule, which can prevent analysis from becoming ritual. The calculation is only as good as its loss model, however. If a decision-maker omits reputational, safety, or distributional consequences, the apparent value of information may be systematically understated. The model should make such value judgments explicit.
Begin with a small random sample, not a costly census
A few observations can matter. The Rule of Five says a random sample of five gives a 93.75 percent chance that the population median lies between the sample's smallest and largest values. The interval may be wide, but it can still improve on a much wider initial uncertainty. Random selection matters: a large biased sample may mislead more than a small representative one. Measurement errors also differ. Random error can often be quantified; systematic bias does not simply disappear when more observations are added.
Instrument the important variables. For Marine Corps fuel planning, researchers attached GPS units and fuel-flow meters to three trucks and gathered about 500,000 data rows under different road conditions. The improved model roughly halved forecast error and was estimated to save at least $50 million per year per Marine Expeditionary Force. The key factors included route surface and vehicle familiarity, not only combat intensity.
Sampling is not a shortcut around rigor. It is a way to spend effort where each additional observation is informative, while random selection makes remaining error more assessable. The fuel study illustrates a further principle: better instruments can reveal causes that experienced planners overlook. But the Rule of Five applies specifically to the median and does not make five observations sufficient for every question. Rare events, highly varied populations, and biased selection require different designs. The right sample is defined by the decision and the data-generating process.
Update evidence with prior knowledge instead of worshipping p-values
Evidence needs a starting point. Bayesian analysis combines prior uncertainty with the likelihood of observed evidence to produce an updated probability. In a product test example, the prior chance of first-year profit is 40 percent. A successful test occurs in 80 percent of profitable cases, but also in 30 percent of unprofitable ones. After a successful test, the profit probability rises to 64 percent; after failure, it falls to 16 percent.
A p-value answers another question. It estimates how likely the observed result, or a more extreme one, would be if there were no real effect. It does not directly tell a decision maker the probability that the claim is true. Emily Rosa's therapeutic-touch experiment produced 44 percent correct guesses across 280 trials, no better than chance. Even a statistically significant result would need interpretation alongside prior plausibility and the consequences of acting.
Bayesian reasoning makes assumptions visible instead of pretending that evidence arrives in a vacuum. It is especially useful when samples are small and relevant prior knowledge is substantial. The product-test example shows why an apparently positive signal can remain weak when false positives are common. Priors can also encode prejudice or institutional habit, so they should be documented and challenged, not treated as sacred. A transparent range of reasonable priors is often more informative than a single number presented as neutral.
Turn expert judgment into a repeatable model and test its record
Structure beats unaided intuition. Paul Meehl's research compared expert judgments with statistical models and found that simple models often performed as well or better across tasks such as diagnosis, academic performance, and forecasting. A Lens Model turns experts' repeated judgments into a consistent formula: identify relevant factors, collect judgments across scenarios, and use regression to model how those factors shaped estimates. The formula removes inconsistency, even if the expert supplied the useful factors.
Avoid confidence theater. The book distinguishes this evidence-based modeling from arbitrary scorecards that multiply subjective ratings such as strategic fit or risk. Such scales can compress important differences and create an illusion of precision. A model should be judged against outcomes, not by how formal or reassuring its process feels. In one Life Technologies example, a model built from expert inputs reduced revenue-forecast error by 76 percent on 16 products not used to build it.
The most productive view of expertise is neither worship nor dismissal. Experts often recognize relevant signals, while repeated human judgment can apply those signals inconsistently. A model can preserve domain insight and make its use more stable. The crucial test is out-of-sample performance: predictions should be evaluated on cases excluded from model construction. Without that safeguard, a seemingly accurate formula may merely memorize its examples. The book's emphasis on measured track records is a strong antidote to confidence that rises simply because an analysis looks rigorous.
Measure preferences through choices, trade-offs, and behavior
Value is revealed in decisions. Preferences can be measured by what people say, but actual choices often provide stronger evidence. Willingness to pay estimates value by asking what people would sacrifice for a benefit; revealed preference examines what they already spend money or time on. To value a reduction in danger, for example, researchers can study how people trade money for small reductions in their own risk rather than pretending life has a single universal price.
Make competing priorities explicit. Utility curves represent combinations of outcomes a decision-maker considers equally valuable, such as quality and timeliness. They can clarify trade-offs without claiming that everyone shares the same preferences. Hubbard also warns that survey response scales can change answers simply by dividing options differently. In a happiness study described in the book, income and life events were related to self-reported happiness, allowing comparison of their effects in income terms.
Putting a value on a public benefit does not mean reducing human worth to a price. It makes allocation choices visible when budgets are limited and every alternative has consequences. Revealed behavior can correct the gap between stated ideals and actual priorities, but it is not a perfect window into preference: constraints, unequal resources, and limited information shape choices. A person who cannot afford a safer option has not necessarily chosen to value safety less. Good valuation needs both behavioral evidence and a careful account of the choices available.
Use digital traces and prediction markets as measurement instruments
Ordinary technology leaves useful traces. GPS, online searches, sales records, and activity monitors can reveal patterns that once required expensive surveys or manual observation. The book describes research in which Google searches for flu symptoms anticipated reported outbreaks by about a week. For Marine Corps fuel analysis, GPS and flow meters exposed route conditions as a major driver of consumption.
Markets aggregate forecasts. Prediction markets let participants buy and sell claims about future events; a share price can represent the market's implied probability. Historical comparisons in the book found some markets reasonably well calibrated, though performance varied by design and incentives. These tools are not magic. Their signals still require checks for selection bias, changing populations, and what the platform actually records.
Digital exhaust can function as a sensor for collective behavior, making measurement cheaper and more continuous. The same scale creates risks: online participants may not represent a population, platforms change their algorithms, and convenient data can be mistaken for relevant data. Prediction markets add a useful incentive to aggregate dispersed knowledge, but market prices are forecasts, not facts. Their strongest role is as one instrument among several, with performance checked against real outcomes and privacy considered from the outset.
Analysis
The book's deepest contribution is not the assertion that every concept has a number hidden inside it. It is the discipline of connecting observation to a consequential choice. That move turns measurement from a reporting habit into an economic intervention: a measurement is worth doing when its expected reduction in decision loss exceeds its cost. The framework is therefore simultaneously epistemic and managerial. It asks what is known, how well it is known, and whether improving that knowledge can alter action.
Its narrative examples make this abstract claim persuasive. Eratosthenes extracts planetary scale from shadows; Emily Rosa designs a cheap test of an extraordinary claim; Marine Corps analysts discover that convoy routes, rather than the most dramatic combat variables, dominate fuel uncertainty. Each case shows how reframing a problem can make observation possible and expose assumptions that experience had concealed.
The main limitation lies in the transition from an observable proxy to a valid construct. A business may count referrals as reputation, but referrals can move for reasons unrelated to reputation. A calibrated expert can describe uncertainty consistently, yet consistency does not establish that the expert's causal model is correct. The book's strongest safeguard is iterative validation, including comparison with actual outcomes, but organizations need to make that a continuing practice rather than a one-time study.
A second tension concerns commensuration. Translating environmental benefits, health risks, or employee quality into money can reveal trade-offs that implicit decision-making hides. It can also privilege what is readily priced and obscure rights, unequal exposure, or obligations that should not be traded away. Quantification does not settle values; it clarifies where value judgments enter. Properly used, the methods do not replace judgment. They make judgment accountable, revisable, and more honest about uncertainty.
Review Summary
How to Measure Anything receives mixed reviews. Many praise its insights on quantifying intangibles and reducing uncertainty in decision-making. Readers appreciate the practical tools, statistical concepts, and historical examples provided. However, some find the book dense, repetitive, and overly focused on mathematics. Critics argue it oversimplifies complex issues and mocks managers excessively. While some consider it a must-read for business decision-makers, others find it tedious and lacking in depth. Overall, the book is valued for its unique perspective on measurement but criticized for its writing style and presentation.
People Also Read
Glossary
Applied Information Economics
Decision-focused measurement methodA decision-analysis method that connects measurement to consequential choices. It defines the decision and its uncertainties, quantifies current knowledge, estimates the value of further information, measures high-value uncertainties, then supports a risk-return choice. Measurement efforts are repeated only while their expected decision value justifies their cost.
Fermi decomposition
Break an estimate into partsA way to estimate an unfamiliar quantity by expressing it as simpler components that can be estimated or observed separately. Estimating the number of piano tuners in a city, for example, can be decomposed into population, piano ownership, tuning frequency, and tuner capacity. The breakdown makes assumptions visible and reveals which unknowns matter most.
Calibration
Match confidence to accuracyA property of probability judgments in which events assigned a particular confidence level occur at roughly that frequency. If a forecaster gives many estimates at 80 percent confidence, about 80 percent should prove correct. Calibration can be evaluated through repeated questions with known answers and improved through feedback and structured estimation techniques.
Rule of Five
Estimate a median from fiveA nonparametric sampling rule: for a genuinely random sample of five observations, there is a 93.75 percent chance that the population median falls between the sample's smallest and largest values. It can provide a useful range when prior uncertainty is large, but it estimates the median, not every feature of a population.
Expected Value of Information
Value of reduced decision lossThe expected value of information is the reduction in expected opportunity loss produced by a measurement. Expected opportunity loss is the probability of making a suboptimal choice multiplied by the cost of that choice relative to the best alternative. The value of perfect information is an upper bound on what additional measurement should cost.
Measurement Inversion
Measure what receives little attentionA recurring pattern in which the variables receiving the most measurement effort are often less decision-relevant than uncertainties that organizations have ignored. Familiar, easy-to-measure factors can dominate reports even when uncertain benefits, adoption, or failure risks have greater potential to change the decision.
Monte Carlo simulation
Model many uncertain scenariosA computer method that repeatedly draws values from probability distributions for uncertain inputs, calculates an outcome for each combination, and summarizes the resulting range of outcomes. It allows decision-makers to estimate probabilities of losses or other thresholds when a calculation using single point estimates would conceal risk.
Lens Model
Model expert judgment consistentlyA method for converting repeated expert judgments into a statistical model. Experts identify relevant factors and assess many scenarios; regression then estimates how the factors relate to their judgments. The resulting formula applies those judgments consistently and can be tested against outcomes to see whether it improves forecasts.
Bayesian inversion
Update beliefs from evidenceA calculation that uses prior probabilities and the likelihood of observations under competing possibilities to determine updated probabilities after evidence arrives. It reverses a question that may be difficult, such as the probability a claim is true given an observation, into likelihoods that may be easier to estimate.
FAQ
What's How to Measure Anything about?
- Measuring Intangibles: The book by Douglas W. Hubbard focuses on the idea that anything, including intangibles like employee morale or the value of information, can be measured.
- Decision-Oriented Framework: It presents a universal approach to measurement that emphasizes the importance of measurements in informing decisions.
- Practical Applications: Hubbard shares real-world examples from various fields to illustrate how measurement can lead to better decision-making.
Why should I read How to Measure Anything?
- Overcoming Measurement Myths: The book is essential for anyone who believes that certain aspects of their business or life cannot be quantified.
- Improving Decision-Making: By learning the methods outlined, readers can enhance their decision-making processes.
- Accessible to All: Hubbard's writing is designed to be accessible to non-experts, making complex statistical concepts understandable.
What are the key takeaways of How to Measure Anything?
- Measurement is Possible: Hubbard emphasizes that even the most elusive intangibles can be quantified with the right approach.
- Applied Information Economics (AIE): Introduces a structured approach to measurement focusing on defining decisions and assessing uncertainty.
- Calibration of Estimates: Discusses the importance of calibrating estimates to improve accuracy in decision-making.
What are the best quotes from How to Measure Anything and what do they mean?
- “Anything can be measured.”: Encourages readers to rethink their assumptions about what can and cannot be quantified.
- “It’s better to be approximately right than to be precisely wrong.”: Highlights the importance of making informed estimates rather than waiting for perfect data.
- “If you understand it, you can model it.”: Emphasizes the connection between understanding a concept and the ability to measure it.
What is the Applied Information Economics (AIE) method in How to Measure Anything?
- Structured Measurement Approach: AIE is a systematic method for measuring uncertainties and making decisions based on those measurements.
- Five-Step Process: Consists of defining the decision, determining current knowledge, computing the value of additional information, applying relevant measurement instruments, and making a decision.
- Focus on Uncertainty Reduction: Treats measurement as a way to reduce uncertainty, allowing for more informed choices.
How does Douglas W. Hubbard define measurement in How to Measure Anything?
- Reduction of Uncertainty: Measurement is defined as "a quantitatively expressed reduction of uncertainty based on one or more observations."
- Probabilistic Nature: Acknowledges that measurements do not need to eliminate uncertainty entirely; they simply need to reduce it.
- Observable Consequences: Tied to observable outcomes, meaning that if something can be detected in any way, it can be measured.
What is the Monte Carlo method mentioned in How to Measure Anything?
- Simulation Technique: A statistical technique that uses random sampling to estimate the probability of different outcomes in a process that cannot easily be predicted.
- Application in Risk Analysis: Allows decision-makers to model uncertainty and assess risks by simulating thousands of scenarios based on varying inputs.
- Historical Context: Named after the Monte Carlo Casino due to its reliance on random sampling, similar to games of chance.
What is the Expected Value of Information (EVI) in How to Measure Anything?
- Definition of EVI: The reduction in expected opportunity loss (EOL) that results from obtaining additional information before making a decision.
- Importance in Decision-Making: Helps prioritize measurement efforts by quantifying the potential benefits of reducing uncertainty.
- Calculation: Calculated as the difference between the EOL before and after obtaining new information.
How does How to Measure Anything address biases in decision-making?
- Human Biases: Discusses common human biases, such as overconfidence and selection bias, that can affect decision-making.
- Calibration Techniques: Introduces calibration techniques to help individuals better assess their own uncertainty and improve their estimates.
- Importance of Training: Emphasizes the need for training to mitigate biases and enhance the accuracy of decision-making processes.
What is the significance of Bayesian analysis in How to Measure Anything?
- Updating Beliefs: Allows decision-makers to update their beliefs based on new evidence, integrating prior knowledge with new information.
- Practical Applications: Provides examples of how Bayesian methods can be applied to real-world problems, enhancing the understanding of uncertainty.
- Comparison to Traditional Methods: Contrasts Bayesian methods with traditional statistical approaches, highlighting their flexibility and applicability in uncertain situations.
How does Hubbard define the value of information in How to Measure Anything?
- Expected Value of Perfect Information (EVPI): Defined as the maximum amount a decision-maker should be willing to pay for information that would eliminate uncertainty.
- Cost-Benefit Analysis: Assessed through a cost-benefit lens, weighing the potential benefits of reducing uncertainty against the costs of obtaining that information.
- Real-World Examples: Provides case studies where the value of information was calculated, demonstrating its practical significance.
How can I apply the concepts from How to Measure Anything in my work?
- Identify Key Decisions: Start by clearly defining the decisions you need to make and the uncertainties involved.
- Use Bayesian Methods: Implement Bayesian analysis to refine your estimates as new data becomes available.
- Measure Incrementally: Apply the principle of incremental measurement by starting with small, manageable data collection efforts.
Download PDF
Download EPUB
.epub digital book format is ideal for reading ebooks on phones, tablets, and e-readers.