🔍 Evidence Selection and Stress-Testing Sources

By Better Wise

In the last lesson you built a claim and propped it up with three reasons, and you named the assumptions hiding under each one. Reasons are the skeleton; this lesson is about muscle. A reason without evidence is just an opinion with good posture, and evidence chosen carelessly will collapse the moment a reader pushes on it.

By the end of this lesson you can:

  • Apply four tests to any candidate piece of evidence
  • Choose between two pieces of evidence that both look usable
  • Run a counterexample test on your own reason before someone else does
  • Check the source behind a statistic, quote, or study summary
  • Decide whether to keep, narrow, or replace evidence in your capstone draft

Four Tests Strong Evidence Has to Pass

Before you read the tests, do this: open your draft and pick one reason. Write down every piece of evidence you could possibly attach to it, even the weak ones. Two minutes, no judging. Now compare your list to the tests below.

1. Relevance. The evidence must support this reason, not the general topic. If your claim is that a campus writing center reduces late-night panic, a statistic about national tutoring enrollment is on topic and still useless.

2. Specificity. Numbers, dates, names, and direct cases beat vague gestures. "Many students struggle" is a mood. "In the fall survey, 41 of 120 first-year students reported missing at least one deadline" is evidence.

3. Representativeness. Does the group in the evidence match the group in your claim? A study of graduate students does not support a claim about first-year writers.

4. Source quality. Who produced it, when, and what did they have to gain? A vendor's own report on its product is evidence of what the vendor says, not of what is true.

flowchart TD
    A["📄<br/>Candidate evidence"] --> B{"Relevant to<br/>this reason?"}
    B -- No --> X["❌<br/>Cut or replace"]
    B -- Yes --> C{"Specific and<br/>representative?"}
    C -- No --> Y["⚠️<br/>Narrow the claim"]
    C -- Yes --> D{"Source survives<br/>a lateral check?"}
    D -- No --> X
    D -- Yes --> E["✅<br/>Keep and cite"]

Practice. Take your weakest item from the two-minute list. Which of the four tests does it fail first?

Check your answer

Most weak evidence fails relevance or representativeness first. If it fails relevance, cut it: no amount of polish fixes off-target evidence. If it fails representativeness, either narrow your claim or go find a matching sample. Naming the first failure tells you which repair to make.

Picking Between Two Good Candidates

You will often have two usable pieces and room for one. Rank them on four axes, in this order: closer to your specific reason, closer to your specific population, closer to the original source, more recent where recency matters.

Note the third axis. A peer-reviewed study is usually stronger than a news article summarizing that study, and the news article is stronger than a social post quoting the news article. Each hop away from the original loses detail and gains distortion. When you can reach the original, reach for it.

Practice. You want to show that spaced practice beats cramming for your own writing course. Candidate A: a 2019 news piece about a memory study. Candidate B: the 2019 study itself. Candidate C: a classmate saying cramming worked for them. You have room for one. Which do you take, and where does each of the others go?

Check your answer

Take B, the original study, and check its population and task. A is a fallback only if B is paywalled, and you should cite the study through it carefully. C is an anecdote: useful as a hook or as the view you are arguing against, never as proof of a general pattern.

The Counterexample Test

Now the part most writers skip. Take your reason and ask: does a single real case exist that would make this false? If yes, your reason is too broad, and one sharp reader can dismantle it.

A counterexample does not destroy your argument. It usually tells you where to put a qualifier. "Spaced practice improves retention" becomes "spaced practice improves retention _____" once you find a case where it did not.

_____ is the source-level version of this move: you leave the page and ask what independent sources say about the source itself. Who funds it, who cites it, who disputes it.

Three other source checks worth thirty seconds each:

  • Date. Is this claim still true, or was it true in 2011?
  • Sample. How many cases, and who was left out?
  • Interest. Does the source benefit from the conclusion?

Evidence drawn only from cases likely to support your view is _____, and readers notice when a paragraph cites three examples that all agree.

Practice. Your reason: "Group work always improves student writing." Run the counterexample test, then rewrite the reason with a qualifier.

Check your answer

Counterexample: a group project where one student wrote everything and the others learned nothing. Rewrite: "Structured group work with individual accountability often improves revision quality." You kept the idea and removed the word that made it breakable.

What to Note About Each Source

For every source you plan to use, record four things next to your draft: author or institution, date, the population or sample, and where you found it. This is not bureaucracy. It is what lets you, a peer reviewer, or an AI critic verify the evidence before it enters your final text. Evidence you cannot trace back is evidence you cannot defend.

Field Notes: The Traps I See in Nearly Every Draft

  • The vivid anecdote wins too often. One striking story feels like proof and is not. Use it to open a paragraph, then bring in the pattern evidence.
  • Two sources that agree are not two sources. Trace them; they usually share one origin.
  • Do not delete the counterexample. Naming it and answering it is stronger than pretending it does not exist.
  • Numbers are not automatically objective. A percentage with no denominator is decoration.
  • Write your qualifier before someone demands one. _____ are tools, not hedges.
  • If the evidence is weak, weaken the claim instead of inflating the evidence. A smaller true claim beats a large false one every time.
  • Record the trail as you go. Reconstructing sources at the deadline is where citation errors and accidental plagiarism are born.

What to Carry Forward

Strong evidence passes four tests, survives a counterexample, and comes from a source you have checked rather than inherited. When it fails, you have three moves: keep and qualify, narrow the claim, or replace the evidence with _____ material closer to the question.

In the next lesson, AI as Critic, you will point a critique engine at exactly this material, asking it to attack your evidence and surface the counterexamples you missed, while you keep the final call about what stays in your draft.

Strong evidence is _____ to the specific reason, not merely to the general topic. Lateral reading means leaving a source to check _____. A single story used as proof of a general pattern is an _____. A controlled comparison supports a _____ claim better than a correlation does.

Front: What are the four tests of strong evidence? Back: Relevance, specificity, representativeness, and source quality.

Front: What does relevance require of evidence? Back: That it supports this specific reason, not just the general topic.

Front: Why is a single anecdote weak proof? Back: One case can illustrate a claim but cannot establish a general pattern.

Front: How do you run the counterexample test? Back: Ask whether one real case exists that would make the reason false, then narrow the claim if it does.

Front: What is a representative sample? Back: A group that matches the population the claim is about.

Front: What is selection bias in evidence gathering? Back: Collecting only cases likely to support your view, then treating them as the pattern.

Front: Primary vs secondary source Back: Primary is the original study, dataset, or document; secondary summarizes or reports on it.

Front: What is lateral reading? Back: Leaving a source to check what independent sources say about it and its claims.

Front: Which four details should you record for each source? Back: Author or institution, date, population or sample, and where you found it.

Front: What does a counterexample usually tell you to do? Back: Add a qualifier or narrow the claim rather than discard the whole reason.

Front: What is a qualifier in an argument? Back: A limiter such as some, often, or in this case that makes the claim match the evidence.

Front: Which evidence beats a news article summarizing a study? Back: The original study itself, because each hop from the source loses detail and gains distortion.

Front: Why is a percentage without a denominator weak? Back: It hides how many cases it is based on, so its weight cannot be judged.