Falsification-First Questioning
Interrogate a claim by asking what would have to hold for it to be true, and demand the disconfirming evidence in the same breath as the confirming — rather than asking whether the claim is good, which recruits only agreement. The move is small and mechanical, which is the point: it converts a vague posture ("be skeptical") into a specific question you can actually ask a person, a pitch, or a model.
The source gives it three forms, escalating in difficulty:
- Outward, at a claim: "What needs to be true for this to be real?" — the question the Theranos investors did not ask. Not "is this a good company" (which the halo, the FOMO, and the charisma all answer for you) but "what physical facts about blood chemistry must hold for a hundred tests to run off one drop?" It is disarming because it is not hostile; it just requires a referent.
- At a model: "Show me the evidence that supports the claim. Show me all the evidence that challenges it." — the two-sided demand as a single prompt. The second half is the operative one: a model asked to support a claim will support it.
- Inward, at yourself: "What am I refusing to see because I need this story to be true?" — the source calls this the hardest, since the fifth distortion has no external adversary to interrogate, only your own wanting. There is no second opinion available on a belief you have not noticed you hold.
The shared structure is that each question makes the claim pay a cost in specifics. A word-noodle ("an all-new iPhone unlike anything we have created"), a $9B secret, and a model's confident summary all share one property: they survive an approving question and die on a falsifying one.
For LLMOps this is the individual-scale root of machinery the vault already has at system scale. Adversarial Planning Council manufactures the disconfirming case with a roster; Evidence-Gated Completion demands the artifact rather than the assertion; Negative Prompting constrains the model away from pleasing. This concept is the primitive under all three — the reason they work is that agreement from a sycophantic model (or a charismatic founder, or a confident room) is uninformative, and only a question that could return a bad answer carries signal.
Claims
- Agreement is uninformative unless the question could have returned disagreement — so ask what would falsify the claim, not whether the claim is good. principle — durable, and the reason this page sits beneath Adversarial Planning Council rather than beside it: the council is this question staffed and automated. Applies identically to a model, a pitch deck, and your own belief.
- Demand the supporting and the challenging evidence in the same request, because a model asked to support a claim will support it. best practice → context: using an LLM to evaluate a claim you have any stake in. The pairing is what makes it work — asking only for challenges just inverts the sycophancy rather than removing it, and yields a model arguing against something it would have argued for a moment earlier. Weaker than Cross-Model Independence (the challenger here shares the supporter's blind spots) and cheaper; a floor, not a substitute.
- "What needs to be true for this to be real?" forces a claim to name its own preconditions, which is what charisma, credentials, and consensus are all supplying answers instead of. (principle — as asserted by the source) — durable in the source's framing; the Theranos anatomy (an impressive board that knew no blood chemistry, FOMO, twelve years of secrecy) is offered as the case where every social signal pointed one way and the precondition question was never put.
- Repeat a marketing claim back to yourself with a question mark on the end ("up to eight times faster? Faster than what?"). best practice — context: parsing sales and marketing language, where the tells are lexical ("up to," "as low as," "starting at," "clinically proven," "recommended by experts"). Narrow by construction; recorded because the same weasel-grammar shows up verbatim in model and inference-vendor marketing, where it is worth exactly the same suspicion.
- The self-directed form has no available second opinion, which is what makes it the hardest. (principle — as asserted by the source) — you can escalate an external claim to another person or another model; you cannot escalate a belief whose motivated character you have not detected. The practical consequence is that the inward form has to be a standing ritual rather than a triggered one — there is no trigger, since the wanting suppresses it.
- A claim you want to be true is one you stop checking, without ever deciding to. (principle — as asserted by the source) — "there is one source that you will never fact-check because you trust it completely: yourself." The video's illustration is a friend's certainty that devotion was itself a reason to be loved, answered by "his love for me is not enough of a reason for me to fall in love with him" — wanting is not evidence.
Related
- Adversarial Planning Council — this question, staffed and automated: the council manufactures the disconfirming case with a roster of opposed personas instead of one operator remembering to ask.
- Evidence-Gated Completion — the artifact-level sibling: don't accept the assertion, require the proof. "What needs to be true for this to be real?" is the same demand pointed at a claim rather than a completion.
- Negative Prompting — the operator-set constraint that stops a model defaulting to agreement; the prompt-level implementation of this page's second form.
- Cross-Model Independence — the stronger version of the two-sided demand: a challenger from another lineage does not share the supporter's blind spots, where a same-model challenger does.
- Authenticity Collapse — the environment that makes this non-optional: when the channel cannot authenticate, the question is what is left.
- Cognitive Offload Cost — the stakes: this is the judgment function the source argues you cannot delegate, so the practice is what keeps it exercised.
- LLM-as-Judge — a judge asked "is this good?" is subject to exactly this failure; the falsifying form is what a calibrated rubric is trying to enforce structurally.
- Error Analysis — "look at your data" is the same instinct at the eval layer: go find the disconfirming case rather than confirming the aggregate.
- Distillate: This Skill Makes You Dangerous In The AI Era