Last week I watched You Can See Everything, the new Elizabeth Holmes documentary from Nathan Fielder and Lance Oppenheim. Fielder was the perfect person to direct and star in it. I think it wins Best Documentary, and it might even have a shot at Best Picture. Very few documentaries leave you thinking and not sure what’s real and what’s fake. It’s a must-watch!
A few days later I was talking with Matthew Nelson who films my videos. A few people texted me that his lighting is so good it almost looks fake. He was annoyed: “Great, so now doing too good of a job is seen as AI.” I told him I don’t think that’s true. People can still tell great work from AI. But we’re all going to need quick ways to verify, and for us that starts with checking what AI tells us.
Looking good isn't the same as being right, and that goes for AI answers too. An AI answer can sound perfect and still be wrong. This week's tip is simple: make AI show its work, then check it yourself.
Give it a try
I use this method all the time to check AI results, especially with long regulatory documents. When I want to understand something, I give the AI a detailed prompt and ask for details and examples from the document. Then I use Ctrl+F to confirm each one, so I know the information is accurate. Here’s how I do it, with a real example below.
You can use this approach in ChatGPT, Claude, Gemini, Grok, or Copilot. Just make sure to use a public document like a publicly available regulation.
Open your AI tool and start a new chat.
Upload the document, or copy and paste its text into the chat.
Paste this prompt, then add your question at the end:
Answer using only the document I gave you. For each part of your answer, quote the exact sentence you used and tell me which section it's in. If the document doesn't say, reply "Not in the document." Don't guess.
My question: [type your question here]
Check the quotes. Open the original document, press Ctrl+F (Command+F on a Mac), and search for a few words from each quote.
If you can't find the quote, don't use the answer. Ask, "Where exactly does it say that?" or check it yourself.
Now try it
Let's use a big regulatory document: OSHA's 2024 update to the Hazard Communication Standard (the final rule, 89 FR 44144). It's about 318 pages and it's public: https://www.govinfo.gov/content/pkg/FR-2024-05-20/pdf/2024-08568.pdf
Here's my question: "What are desensitized explosives?"
Paste this prompt, then the question:
Using only the document I gave you, answer my question with one exact sentence, copied word for word from the document. Give me the page number. If the document doesn't say, reply "Not in the document."
My question: What are desensitized explosives?
The AI answered with this sentence, which I then checked:
"Desensitized explosives are solid or liquid explosive chemicals which are phlegmatized to suppress their explosive properties in such a manner that they do not mass explode and do not burn too rapidly and therefore may be exempted from the hazard class "Explosives"
I left out two footnote markers: 10 after “phlegmatized” and 11 at the end. Claude said the sentence was on PDF page 251 (Federal Register page 44394). Hold that thought.
Now check it. Open the PDF, press Ctrl+F, and search for a short, exact phrase from the quote. I picked phlegmatized to suppress their explosive. It matches once, on PDF page 193 (Federal Register page 44336). So Claude got the page wrong, right?
Nope, I did. If you read the full sentence on page 193, it's actually a different one: “by definition, desensitized explosives are phlegmatized to suppress their explosive properties, and therefore the ‘Keep wetted with . . .’ statement is appropriate.” That’s OSHA explaining a label statement, not giving the definition. The real sentence is on page 251, just like Claude said. My search missed it because footnote marker 10 is between “phlegmatized” and “to,” so the PDF didn’t find it as a match. The AI was right and I was wrong, which is a good reminder to double-check. A better way to search is to use the start of the quote: Desensitized explosives are solid or liquid. That only comes up once, on page 251. Choose words that don’t have footnote markers in them, and make sure to read the whole sentence where you land. Also, avoid broad terms: searching for “desensitized explosives” alone gives about 80 results.
If the file is too large for your AI tool, paste just the section you need. You can still use Ctrl+F on the full PDF to check the quote.
One more tip: always check if your source has been updated. For example, the compliance dates for this 2024 rule were extended by four months in a January 2026 OSHA notice (https://www.osha.gov/hazcom/rulemaking/extension). The quote is in the document, but the document may not be the final version.
When you ask about something online
If you ask a question without giving it a document, add one line to the end of your question:
Give me a link to the source for each fact.
After that, click the links. Check that each page is real and actually says what the AI claims.
A few quick rules
- A quote you find in the original document is better than an answer that just sounds correct.
- "Not in the document" is a good answer. It shows the AI isn’t inventing information.
- Never put confidential formulas or customer information into a public AI tool.
AI news this week:
Google has made Deep Review available to everyone. On October 14, they released a tool that reads technical documents and checks for logic errors or unsupported claims. It started with research papers but now works with legal and financial documents too. Everyone gets two free reviews per week, and it accepts PDFs and plain text. Google says uploaded documents aren’t used to train its models. Think of it as a second pair of eyes, not a final answer. So keep using Ctrl+F to check.
https://research.google/blog/analyzing-technical-documents-with-deep-review/
Google’s new Gemini 4 Argon is now available. It launched on October 13 and is in the Gemini app for Google AI Ultra and Pro subscribers. Google says it’s designed for long, multi-step tasks like legal and finance work, and that it closely follows the material you provide. These are Google’s claims, so try it out on a public document before relying on it.
https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-argon/
The FTC has sent investigation demands to OpenAI and Anthropic. On October 14, the agency sent civil investigative demands and confirmed this to CBS News. According to Axios, they’re asking for records about safety incidents, privacy policies, and safeguards. This is just an investigation, not proof that either company did anything wrong.
https://www.cbsnews.com/news/ftc-subpoenas-anthropic-openai-consumer-protection/
See you next Thursday.
Brett