Field Guide
Research with AI using a frontier check
External evidence (preregistered field experiment); method not yet tested by Today’s Worker; example is illustrative
Before you hand a research or synthesis task to AI, decide whether the task sits inside what AI does well or outside it. Let AI extract, tabulate and draft from your sources. Keep the judgment calls that depend on context the files do not hold.
The method comes from one large preregistered experiment. Today’s Worker has not tested it ourselves yet. The example below is invented.
What you get, and where not to use it
Outcome: a sourced synthesis with each claim tied to a quote and page, plus a short list of what the sources do not settle.
Do not use it for: a novel judgment that turns on subtle evidence, claims you cannot verify, or files your agreements keep confidential.
How strong is the evidence?
| Question | Answer |
|---|---|
| Evidence label | External evidence. Not tested by Today’s Worker. |
| Best source | Dell’Acqua et al., “Navigating the Jagged Technological Frontier.” First released as a working paper on September 18, 2023; published in Organization Science. 758 Boston Consulting Group consultants in a preregistered experiment using GPT-4. |
| Inside the frontier | On 18 product-development tasks, consultants with AI completed 12.2% more tasks and finished 25.1% more quickly on average, with significantly higher quality. |
| Outside the frontier | On one business-case task built so AI would mislead, the working paper says consultants using AI were “19 percentage points less likely to produce correct solutions.” The journal abstract words the same result as “19% less likely to produce correct solutions.” The journal’s discussion section uses “19 percentage points.” Control participants were correct about 84.5% of the time; the two AI groups scored 60% and 70.6%. |
| What it does not show | One outside-the-frontier task, one firm, early-career consultants, one 2023 model. Where the frontier sits for your task and your model today is something you have to test. |
Why the wording matters: a drop from about 84% to about 65% is a fall of about 19 points. As a relative change it is larger than 19%. We quote both versions as written rather than pick one.
The study also found that AI-assisted answers to the hard task were rated more coherent and persuasive even when wrong. A polished answer is not evidence that it is right.
What to bring
- The question, in one sentence.
- The source documents.
- The decision the synthesis feeds.
Decisions that stay with you
- Whether each part of the task is inside or outside the frontier.
- Which claims to verify by hand.
- The final recommendation.
Tools and setup
A chat assistant that accepts file uploads, or a research tool that cites passages. About twenty minutes the first time to set up the claim table and the spot-check habit.
Worked example (illustration, not a measured result)
EXAMPLE: A small-agency owner compares three vendor proposals. AI pulls prices, terms and renewal dates into a table with page references. She spot-checks three rows against the PDFs and finds one wrong renewal date. Choosing a vendor depends on how this client handles change, which no proposal mentions, so she makes that call herself.
Acceptance checks
- Every claim carries a quote and a page or section.
- You opened the source and confirmed at least three claims.
- The “not settled” list is present and specific.
- The recommendation is yours, written after the table.
Failure cases
- You accepted a confident answer on a judgment task. That is the outside-the-frontier failure the study measured.
- A claim cites a page that does not say it.
- The table quietly includes facts from outside your sources.
For a longer checking routine, see how to check AI research.
Copy this prompt
Question: [one sentence] Use only the attached sources. Do not use outside knowledge. 1. Build a claim table: claim | exact quote | source | page or section. 2. List what these sources do not settle, one line each. 3. Flag any part of the question that needs a judgment call the sources cannot make. Do not make a recommendation. I will decide. If a quote cannot be found, write NOT FOUND instead of paraphrasing.
Version and test record
Method version 0.1. Sources checked 2026-10-15. Today’s Worker test: not yet run. When we run one, we will record the task, tool, date, acceptance-check results and any failure here, and change the evidence label to internal test.
This page is one of five graded method pages: draft, then edit and sign, research with a frontier check, learn with a hint-only tutor, visit notes to scope wording and lead reply and quote follow-up drafts. Each one says how strong the evidence is, not only what to do.