How to check its work

The first guide in this track made the point. The tone is identical whether it’s right or wrong. There’s no wobble, no hedge, no tell.

That’s worth sitting with, because every instinct you have for spotting a bluffing human is useless here. People who are unsure sound unsure. This doesn’t.

So checking can’t be a feeling. It has to be a procedure. Here are three, cheapest first.

---

The question that does most of the work

Before the three methods, one habit worth more than any of them:

Which part of what I gave you says that?

Ask it about any specific claim. A number, a name, a date, a deadline, a quote.

If the claim came from your source, you get a pointer to the line. If it didn’t, you get one of two things: a retraction, or a confident restatement with no actual location in it. Both tell you what you needed.

This works because it changes the task. “Summarise this” invites it to produce something summary-shaped. “Show me where this came from” can only be answered by the document or not at all.

---

Check 1: trace every specific back to a source

The fastest sort you can do on any answer is to split it in two.

Came from you	Came from it
Facts in the document you pasted	Anything supplied from memory
Numbers you provided	Numbers it calculated or recalled
Names, dates, quotes in your source	Names, dates, quotes it produced
Your constraints, restated	Context it added for you

Everything in the right column is unverified. Not wrong. Unverified. That’s a different claim and a more useful one.

In practice this takes about a minute. Read the answer and underline every specific. Then ask, for each one, whether you supplied it. What’s left underlined is your checking list.

Most of the time that list is short, which is the point. You’re not redoing the work. You’re checking three things instead of twenty.

---

Check 2: ask again, cold

Invented details are unstable. Real ones aren’t.

Open a new conversation. This part matters. Ask again in the same thread and it will agree with itself, because your earlier exchange is sitting right there shaping the answer. Agreement inside one conversation tells you nothing at all.

In the fresh window, ask the same question in different words. Then compare the specifics.

• The specifics match. Good sign. Not proof, but the answer is at least stable.
• The specifics drift. A number moves, a name changes, a date shifts by a year. That detail was generated rather than recalled, and you’ve found it.

This is most useful on the things you’d struggle to check directly. Which release added a feature, whether a rule has an exception, what a limit actually is.

It costs about ninety seconds and it catches the confident inventions that reading alone won’t.

---

Check 3: spot-check what’s cheapest to verify

You can’t verify everything. So verify the parts where being wrong is cheapest to detect, and treat the result as a signal about the rest.

Good spot-checks, in order of speed:

• Search for a quote. If it quoted your document, use ctrl-F on the actual words. A quote that isn’t findable is not a quote.
• Add up the numbers. If it produced a table with a total, check the total. Arithmetic is where carelessness shows first.
• Count the things. You asked for five and it says “the five main points”. Count them.
• Follow one link. If it gave you sources, open one. Not all of them. One.

Here’s the useful part. A spot-check isn’t only about the thing you checked. If the quote is fabricated or the total is wrong, that tells you the whole answer was produced without care, and the parts you didn’t check deserve the same suspicion.

One failed spot-check should send you back to the top, not only to that line.

---

How much checking is enough

Not everything needs all three. Match the effort to what happens if it’s wrong.

If it’s wrong, then	Do this
You notice immediately and fix it	Read it. That’s the check
Someone else sees it before you do	Trace the specifics, check 1
It goes into a document that gets approved	Checks 1 and 3
A decision gets made on it	All three, and cite your sources yourself
It touches money, people, legal, or medical	Check it the way you would if a stranger had written it, because one did

The last row is the honest position on all of this. The output arrived in your voice, in your document, in a format you recognise. None of that means it came from anywhere.

---

When it breaks

It insists, confidently, that it’s right. Confidence is not evidence, and pushing back harder tends to produce a more confident version rather than a more accurate one. Go to check 2 instead. A fresh conversation settles it faster than an argument does.

It apologises and changes the answer immediately. Equally uninformative. It will often fold the moment you express doubt, whether or not it was wrong. A retraction under pressure is not a correction, so verify the new answer the way you’d have verified the old one.

It gives you a source that doesn’t exist. Fabricated citations look completely ordinary: real-sounding author, plausible title, well-formed link. This is why check 3 says open one. If a source doesn’t resolve, treat every other source in that answer as unchecked.

Both conversations agree and both are wrong. It happens, mostly on things widely repeated and widely wrong. Consistency means stable, not true. For anything that matters, one check has to be against the world rather than against the model.

It’s right about the facts and wrong about what they mean. The hardest one to catch, and no procedure here finds it. That’s your job, and it’s the part of the work that stays yours.

---

What to take from this

You don’t need to check everything. You need to know which parts you haven’t checked.

The failure mode isn’t trusting it. It’s losing track of which sentences came from your document and which arrived from nowhere. Once those are separated, the rest is ordinary work.