Spotting Statistical Manipulation: Baselines, Charts, and Risk

By Better Wise

From the Source to the Number

Last lesson you learned to walk a claim back to its source: the dataset, the document, the on-record statement. This lesson starts where that walk ends. You are holding the original numbers, and they still do not quite match the headline. Usually the gap comes from three moves: a chosen baseline, a chart that magnifies, and a risk stated in relative terms.

By the end of this lesson you should be able to:

• Name the baseline a claim uses and ask what a fairer one would show.
• Read a chart's axis, units and time window before reading its shape.
• Convert a relative risk into absolute terms using the base rate.
• Keep percent and percentage points apart.

Note: Everything below is a reading skill for claims you meet in the wild. The aim is to recognise a distorted number and correct it, not to build one.

Baselines: The Comparison You Were Not Shown

A number on its own says nothing. "Theft rose 40%" is a comparison. The real questions are: 40% of what, and starting when.

Worked example. A local page posts: "Downtown theft up 40% since 2020." You find the police table behind it:

Measured from 2020, theft is up 40%. Measured from 2019, it is down 30%. Both figures are arithmetically correct. The 2020 baseline is the one that produces the alarming sentence, because 2020 was a year when shops were closed and almost nothing was reported.

_____ is a poor baseline here because activity that year was unusually low, so any later recovery looks dramatic.

Watch for these baseline choices: a pandemic or crisis year, a record-high or record-low year, a single month compared with a whole year, or a raw count where the population has changed. If a city's population grew by a third, raw counts can rise while the rate per person falls.

The check is short: ask why this start date, and ask what the same series looks like across ten years rather than two.

Charts: When the Picture Exaggerates the Change

A chart is a claim about proportion. Read the axis before you read the shape.

Worked example. A post shows two bars for a public budget, with the vertical axis running from 98 to 106 (millions).

The y-axis starting at 98 instead of 0 removed almost all the distance between the bars, so an ordinary 4% change filled the frame.

Three more patterns worth naming:

• Dual axes. Two y-axes with independent scales can make two unrelated series appear to move together.
• Area and volume. A circle drawn with twice the diameter carries four times the area, so the picture overstates the change.
• The window. A line chart that begins at a trough or ends at a peak tells a different story than the full series. Rolling averages chosen after the fact do the same.

A truncated axis is not automatically dishonest. Financial index charts rarely start at zero, and that is a legitimate convention. The test is whether the axis is labelled and whether the visual difference is roughly proportional to the real one.

Relative Versus Absolute Risk: The Same Fact, Two Feelings

Worked example. A health story reports that a drug "doubles the risk" of a rare side effect. You find the study: the rate goes from 1 in 10,000 to 2 in 10,000.

• Relative risk: doubled, a 100% increase.
• Absolute risk: one extra case per 10,000 people taking the drug.

Both describe the same data. The first sentence sounds alarming, the second sounds small, and a reader needs both to judge.

To move from relative to absolute you need the base rate, the starting figure in the population. When a story gives you a percentage but no base rate, you are missing the half of the sentence that tells you whether it matters.

The same confusion appears with percentage points. Approval moving from 4% to 6% is a rise of 2 percentage points, and it is also a 50% increase. Both are true. A statement that does not say which one it means is telling you something about its intent.

Check Your Recall

Front: What is a baseline in a statistical claim? Back: The starting value or comparison group the change is measured against.

Front: Why can a pandemic year make a misleading baseline? Back: Activity was unusually low, so any later recovery looks like a dramatic rise.

Front: Two questions to ask about any 'up X%' claim Back: Compared to what, and starting when?

Front: What does a truncated y-axis do to a chart? Back: It cuts off the bottom of the scale, so small differences fill the frame and look large.

Front: Are truncated axes always dishonest? Back: No. They are common and legitimate for index levels; the test is a labelled axis and a visual change proportional to the real one.

Front: What is the problem with dual y-axes? Back: Two independent scales can make unrelated series look correlated.

Front: Define relative risk Back: The size of a change compared with the starting rate, so doubling is a 100% increase.

Front: Define absolute risk Back: The actual number of people affected, such as 1 in 10,000 rising to 2 in 10,000.

Front: What do you need to convert relative risk into absolute risk? Back: The base rate in the population.

Front: Difference between percent and percentage points Back: Going from 4% to 6% is +2 percentage points but a +50% relative increase.

Front: Why does the end date of a series matter as much as the start date? Back: Stopping at a peak or trough changes the story the same data tells.

Front: Why divide a raw count by the population? Back: Raw counts grow with population, so per-person rates are needed to compare places or years fairly.

Front: When is a number decoration rather than evidence? Back: When it has no denominator, no stated comparison, and no way to check it against a population.

Field Notes for the Next Suspicious Number

Field Notes for the Next Suspicious Number

flowchart TD
    Claim["📰<br/>Claim with a number"] --> Base["📏<br/>Baseline: compared to what?"]
    Base --> Axis["📊<br/>Chart: what do the axes say?"]
    Axis --> Risk["⚖️<br/>Risk: relative or absolute?"]
    Risk --> Verdict["✅<br/>Informative or decoration?"]

• Always hunt for the denominator before you judge the size.
• Ask why this start date and why this end date, out loud if you have to.
• Read the axis labels and units before the shape of the line.
• Convert any relative risk by multiplying it by the base rate.
• Say "percentage points" whenever you mean the difference between two percentages.
• Common mistake: treating every truncated axis as fraud. Some are standard practice, and crying foul on a normal chart costs you credibility.
• Common mistake: assuming a large relative change is a large real change. A 100% rise in a 1-in-a-million event is still rare.
• Rule of thumb: if the claim gives you no number you can check against a population, treat it as a question rather than a fact.

What to Carry Forward

You now have three levers that explain most of the distance between raw data and a startling headline: the baseline behind the percentage, the axis under the chart, and the choice between relative and absolute risk.

• A number needs a comparison, a denominator, and a time window.
• The chart's axis is part of the claim, not decoration around it.
• Relative risk creates alarm; absolute risk tells you whether alarm is warranted.
• A truncated axis or a relative figure is a prompt to look further, not proof of bad faith.

Next, you will put these checks into a repeatable five-minute routine for verifying a suspicious claim before you share it.