The scary AI story is not the robot uprising. It is a perfectly formatted quote that goes to a client with a number in it that nobody checked. It is a proposal citing a regulation that does not exist. None of these require anything dramatic. They just require one person to trust a confident paragraph.
We are not anti AI. Harrison says two things constantly around here, and the first is that AI is the dumbest it will ever be today. These tools are genuinely useful and improving fast. The second is what makes them safe to use: treat AI like a junior staff member. That framing is the whole article.
Treat It Like a Junior Staff Member
Picture the sharpest new hire you have ever had. Reads fast, writes cleanly, works at two in the morning, and does not know what they do not know. You would not let that person send a pricing quote to your biggest client without a second set of eyes. You also would not refuse to give them work. You would give them real work, check it, and learn which tasks they get right alone.
The specific failure mode is well documented. In its generative AI guidance published in 2024, the National Institute of Standards and Technology uses the word confabulation, defining it as a phenomenon in which these systems generate and confidently present erroneous or false content in response to prompts. NIST notes the risk shows up when users believe false content, often because of the confident nature of the response, and then act on it or pass it along.
NIST goes further. It warns that outputs can include confabulated logic or citations that appear to justify the answer, which can further mislead people into trusting the system inappropriately. The fake answer often arrives with a fake receipt attached. Scanning for something that looks wrong is not enough. Wrong output does not look wrong.
What Always Gets Checked
Verifying everything is unrealistic and wastes the productivity you were after. Verifying nothing is how you end up in an uncomfortable phone call. Draw the line by category, not by feeling.
- Numbers. Prices, quantities, hours, percentages, totals, dimensions, deadlines. Any figure that came out of the model rather than out of your system gets traced back to a source you control.
- Names. People, companies, job titles, product names. Getting a contact’s name or title wrong in the first line undoes everything the rest of the email was trying to do.
- Dates. Meeting dates, deadlines, effective dates, renewal dates, anything historical. Models are notably casual about calendars.
- Quotes and citations. If the output quotes a person, a study, a law, or a standard, either open the actual source or delete the quote. Highest risk category, most often skipped.
- Legal, medical, financial, and safety claims. Anything a customer could rely on to their detriment. These get reviewed by whoever is actually responsible for that subject.
- Anything leaving the building. Client emails, proposals, invoices, contracts, social posts, website copy. Internal drafts can be rough. External anything carries your name on it.
For a cautionary tale that is public record, read the New York federal court decision in Mata v. Avianca from June 2023. Attorneys filed a brief containing what the court described as non existent judicial opinions with fake quotes and citations created by an AI tool. The court identified at least six fabricated decisions and imposed a five thousand dollar penalty. The judge did not condemn the technology. The court wrote that technological advances are commonplace and there is nothing inherently improper about using a reliable artificial intelligence tool for assistance, but that existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings. Swap the word attorneys for your own job title. The gatekeeping role does not transfer to the tool.
The Two Minute Verification Habit
Long checklists die. Short habits survive. Here is the version that takes about two minutes.
- Underline every fact. Read the draft once and mark each number, name, date, and claim. There are usually fewer than you expect, which is the point of doing it explicitly.
- Ask where each one came from. Did you paste it in, or did the model produce it? Anything the model produced is unverified by definition, no matter how ordinary it looks.
- Check the unverified ones against a real source. Your CRM, your accounting system, the signed contract, the actual web page. Asking the same AI whether it is sure does not count.
- Delete what you cannot confirm. This is the step that saves people. If you cannot verify a statistic or citation in under a minute, cut it. The sentence is almost always fine without it.
- Read it out loud before it goes. Tone problems, awkward confidence, and claims you would never make in person all surface when you hear them.
One more rule worth adopting as policy: never let a tool send anything on its own. Draft is a feature. Send is a decision. Keeping a human between the two is the cheapest control available.
Making It a Norm Rather Than a Nag
Any rule enforced by one person reminding everyone else has a short shelf life. Verification has to be built in, not policed.
- Make it about the work, not the tool. Nobody checks facts because a policy says to. They check because sending a wrong number to a client is embarrassing. Frame it as professional standards that happen to apply to AI output too.
- Put it in the template. Add a checked before sending line to your proposal and quote templates. A field beats a memo every time.
- Normalize saying where it came from. If people can say I drafted this with AI without a reaction, they will ask for a review. If it feels like a confession, they hide it and nobody reviews anything.
- Show the catches, not just the misses. When someone spots a fabricated citation before it went out, say so in a team meeting. That does more for the habit than any training slide.
- Give the junior staff member better sources. Much of the pain disappears when people paste in real numbers and real documents instead of asking the model to remember facts.
This is the same territory we covered in how to actually prepare your team for AI without the hype. Habits are cheaper to build early than to retrofit after an incident.
Where This Fits in the Bigger Picture
If you eventually need something more formal, NIST released its AI Risk Management Framework in January 2023 as a voluntary framework built around four functions: govern, map, measure, and manage. You do not need to adopt the whole thing to benefit from the sequence. Decide who is accountable, know where AI is being used, have a way to tell whether it is working, then act on the gaps.
The same technology is used against you. Attackers write cleaner phishing messages with it, which we covered in our piece on AI powered cyberattacks. A team trained to pause and verify is better defended in both directions.
The Bottom Line
AI output is a confident first draft from a capable junior employee who has never met your clients. Treat it that way and it makes your team faster. Treat it as a finished answer and it will eventually cost you a relationship. Check the numbers, names, dates, quotes, regulated claims, and anything leaving the building. Two minutes, every time.
If you want help setting up sensible AI habits, tooling, and guardrails for your team without slowing anyone down, that is work we do every week for businesses in Denton County and across North Texas. Contact us today.
Sources:
Comments are closed