AI as a Question Generator, Not an Authority

By Better Wise

You now have a framed decision, scored alternatives, a pre-mortem, and a stated confidence level. The last tool in the kit is also the easiest one to misuse. In earlier lessons you aimed your skepticism at your own reasoning: you attacked your plan in the pre-mortem and you checked whether the decision was reversible before spending more time on it. This lesson points that same skepticism at an outside helper.

By the end you should be able to:

• Explain why "ask the model what to do" is a category error for consequential choices
• Write three prompt patterns that surface missing questions, counterarguments, and blind spots
• Triage what comes back into questions worth answering, claims worth verifying, and noise
• Keep written evidence that the final call was yours

Where AI Fits in a Decision Loop

A language model is very good at producing plausible text quickly and very bad at knowing which claims are true about your specific situation. It has no access to your constraints, your risk tolerance, or the facts on the ground unless you supply them, and it will still sound equally confident when it invents them.

That produces two predictable failure modes:

The productive use sits between them. You ask for the questions you did not think to ask, for the strongest case against your leading option, and for the failure modes you cannot see from inside your own framing. The model widens your view; you keep the pen.

flowchart TD
    A["📝<br/>Write your own case first"] --> B["❓<br/>Prompt for missing questions"]
    B --> C["🔍<br/>Triage the output"]
    C --> D["⚖️<br/>Decide and record rationale"]
    D --> E["🔁<br/>Review the process later"]

Three Prompt Patterns That Widen Your View

Each pattern below asks for questions or arguments, not for a verdict. Give the model your decision statement, your constraints, and your success criteria from Module 1 so the output is anchored to your actual problem.

1. The missing-questions prompt. Write: "Here is my decision, my constraints, and my two options. Do not recommend an option. List the ten questions I have not asked that would most change the choice, and say for each one what a bad answer would imply." This is the highest-value prompt in the set, because the questions you failed to ask are exactly what you cannot generate alone.

2. The steelman-against prompt. Write: "Argue against Option A as strongly as the evidence allows. For each attack, name the assumption it depends on and the evidence that would confirm or kill it." Forcing the model to state assumptions turns rhetoric into things you can check.

3. The pre-mortem amplifier. Write: "Assume it is eighteen months later and this choice failed badly. Give me six distinct causes of failure, ordered by plausibility, and mark which ones I could test cheaply before committing." You already ran a pre-mortem on your own. This version adds categories you missed and points at cheap tests.

Tip: Add one closing line to every prompt: "List what you are uncertain about and what evidence would change your answer." It surfaces testable claims instead of confident prose.

Exercise: rewrite a weak prompt

Weak prompt: "I'm choosing between hiring a contractor and doing the build myself. What should I do?"

Rewrite it as a question generator. Your version should name the decision, the constraints (budget, timeline, your skills), and the success criteria, then ask for missing questions or the strongest case against one option, with assumptions labeled.

Example answer: "Decision: build an internal tool myself over 6 weeks, or hire a contractor for 12k and ship in 3. Constraints: one engineer, fixed launch date, no maintenance budget after launch. Success criteria: shipped by the date, no critical bugs in month one. Do not recommend. List the eight questions I have not asked that would most change this choice, and for each, note what a bad answer would imply. Flag anything you are uncertain about."

Write Your Case Before You Ask

Draft your own reasoning, one paragraph, before you open a chat window. Record it in your decision brief with a timestamp. Two reasons: it protects you from anchoring on a fluent answer, and it gives you a record that the judgment was yours. If the model's output changes your mind, note which question did it and why. That note is worth more than the output itself.

Triage What Comes Back

Sort every response into three buckets:

• Questions worth answering. Keep only the ones where the answer could plausibly flip your choice or change how you sequence it. Discard the rest without guilt.
• Claims worth verifying. Anything factual about costs, timelines, regulations, or how a tool behaves must be checked against a primary source. Confident fabrication is the default failure, not an edge case.
• Noise. Generic advice that could apply to any decision, restatements of your own prompt, and hedged filler.

Warning: A model that invents a plausible statistic, a fake case study, or a nonexistent clause will present it in the same tone as a verified fact. Treat every specific number as unverified until you find it elsewhere.

Exercise: triage a response

A model returns: (a) "What happens to the project if the contractor is unavailable for three weeks?" (b) "Contractors of this type typically cost 8k to 15k." (c) "Either path can work if you communicate clearly." (d) "Your plan assumes your own availability is constant across six weeks. Is that tested?"

Sort these and say what you do with each.

Guidance: (a) is a question worth answering, it tests a real single point of failure. (b) is a claim to verify against quotes you collect yourself. (c) is noise. (d) is the best item: it names a hidden assumption you can test cheaply by checking your calendar for the six weeks.

Rules of Thumb for Asking Without Delegating

• Ask for questions, not answers. "What am I not asking?" beats "What should I do?" almost every time.
• Paste your constraints every time. Without them you get advice for an average person in an average situation, which is nobody.
• Never let a model be the only source for a factual claim that carries money, safety, or a deadline.
• Watch for sycophancy: if your second message starts with "but I was planning to...", expect the model to soften. Start a fresh thread instead.
• Keep the prompt log in the decision brief. The record of what you asked, and what you rejected, is your evidence that the call was deliberate.
• If the output makes you feel certain quickly, slow down. Fluency is not calibration.

What to Carry Forward

• AI is a generator of questions, counterarguments, and blind spots, not a source of authority for consequential choices.
• Write your own case first, prompt for what you missed, then triage into questions, claims to verify, and noise.
• Verify every specific fact elsewhere, and log what changed your mind.
• The responsibility for the call stays with you, and your prompt log is how you prove the process was sound.

Prompts should ask for _____ rather than a recommendation, because the model cannot see your _____ unless you supply them. Specific numbers and citations returned by a model must be _____ against a primary source before they influence the decision. Writing your own reasoning before prompting protects you from _____ on a fluent answer. The output of a prompting session should be split into questions worth answering, _____, and noise. A model that agrees with your plan after you defend it is showing _____, which is not evidence.

Front: What role should AI play in a consequential decision? Back: Question generator and counterargument source, never the authority that decides.

Front: Why is "what should I do?" a weak prompt? Back: It hands over judgment and produces a recommendation built on assumptions you never inspected.

Front: What does the missing-questions prompt ask for? Back: The questions you failed to ask, plus what a bad answer to each would imply.

Front: How do you turn a vague counterargument into something checkable? Back: Require the model to name the assumption each attack depends on and the evidence that would confirm or kill it.

Front: Rubber-stamp mode Back: Repeatedly asking the model to confirm a plan until it agrees, which yields agreement rather than evidence.

Front: Why write your own reasoning before prompting? Back: To avoid anchoring on a fluent answer and to record that the judgment was yours.

Front: The three triage buckets Back: Questions worth answering, claims to verify, and noise.

Front: How should you treat a specific statistic from a model? Back: As unverified until confirmed against a primary source.

Front: What does the closing line "what evidence would change your answer?" achieve? Back: It surfaces testable claims instead of confident prose.

Front: Sycophancy in a model conversation Back: The tendency to soften or agree once you reveal and defend your preferred plan; start a fresh thread instead.

Front: Why paste constraints and success criteria into the prompt? Back: Otherwise the advice targets an average situation that matches nobody's actual decision.

Front: What belongs in the decision brief's prompt log? Back: What you asked, which questions you kept or rejected, and what changed your mind and why.

Front: Pre-mortem amplifier prompt Back: Assume the choice failed, list six ordered causes, and mark which ones can be tested cheaply before committing.

Next up: once the decision is made and time has passed, Post-Decision Reviews: Process Over Outcome shows you how to grade the process you just documented, rather than the luck of the result.