Check Your Work: The Growing Problem of AI Hallucinations in the Workplace

This week in our newsletter:

• Private Assets Inch Closer to 401(k)s– But Risks Are Mounting
• Are ‘Tiny Teams’ the Future of Work Teams?
• A Landmark Win Against Social Media Giants

Read the full newsletter here.

According to a recent Stanford University study, 83% of professionals accept AI-generated content in the workplace without verifying it as fact, even when it contains obvious errors.

Before you repeat that stat to anyone, you should know: we made it (and study) up.

Welcome to the problem that’s creeping into every professional discipline. AI hallucinations are one of the most consequential issues facing knowledge workers today, with consequences already showing up in courtrooms, boardrooms, and government reports.

Just last week, Gizmodo reported that an Oregon court fined an attorney $10,000 for a legal brief that contained 15 fake case citations and nine invented quotes. When challenged, the attorney blamed his paralegal. He also argued that his staff had tried to verify the citations by asking Google.

Turns out, using AI to fact-check AI doesn’t actually ‘AI-proof’ your work.

Unfortunately, this story is just one of many impacting the legal profession.

Lawyer and data scientist Damien Charlotin maintains an ongoing database tracking legal decisions where generative AI produced hallucinated content. As of this writing, that count is at 1,218 cases worldwide, with 807 from the US alone.

And the legal profession is far from the only industry dealing with AI hallucinations.

Deloitte, one of the world’s largest consultancy firms, was caught up in two high profiles cases of AI hallucinations in 2025– a $1.6 million CAD healthcare report to the Canadian government and a $290,000 report for the Australian government. Both were found to contain fictional citations to research papers that don’t exist, with the Australian report containing over 20 instances

In response to the fallout, Deloitte Australia issued a partial refund while maintaining that the substance of its recommendations was unchanged. But critics argue that the scandal raises a more fundamental question: who, exactly, is doing the work being paid for?

Australian senator Deborah O’Neill, for example, said: “Perhaps instead of a big consulting firm, procurers would be better off signing up for a ChatGPT subscription.”

When clients perceive that AI is doing the bulk of the work– and poorly at that– the result is a dilution of both the value and perceived integrity of professional services.

Why Do AI Hallucinations Happen So Often?

In a nutshell, large language models function like highly sophisticated autocomplete tools. They are designed to predict the next word or phrase, not to verify whether the output is true.

These models are trained on vast amounts of internet text, which includes both accurate and inaccurate information. They learn patterns rather than ground truth, and they can’t distinguish between the two. Even if trained only on accurate data, they could still combine patterns in ways that produce new, false claims.

When faced with knowledge gaps, ambiguous prompts, or conflicting information, the model generates a plausible answer based on the learned patterns.

These systems are also optimized to produce fluent, confident-sounding output. Hallucinated information can sound just as polished and authoritative as the real thing, making it hard to tell the difference at first glance.

At this point, you may be thinking that AI hallucinations are surely an early-era problem that newer, smarter models will eventually solve. But the opposite is actually closer to the truth– AI hallucinations are becoming more commonplace as we rely on AI more.

The New York Times reported on this less than a year ago, citing research that found OpenAI’s reasoning systems hallucinated more frequently with each new model.

On one benchmark that tested factual knowledge about public figures, the o3 model hallucinated 33% of the time– more than twice the rate of the o1 model it replaced. The latest o4-mini system fared even worse, hallucinating 48% of the time on the same test.

Testing on a broader factual benchmark, the o3 was found to hallucinate 51% of the time, while the o4-mini hallucinated 79% of the time.

Zooming out, AI hallucinations are also occurring across all different AI systems, not just ChatGPT.

A more recent study published by AI Multiple in January 2026 tested AI hallucination rates by asking 37 different LLMs 60 questions. They found that even the latest generation of models still hallucinate at least 15% of the time under normal conditions.

Source: AI Multiple.

The ubiquity of AI hallucinations is quickly becoming a primary concern for users. In fact, for all the fear that AI will cause job losses, there is now ample evidence that AI hallucinations actually worry users more than job loss.

When Anthropic surveyed more than 81,000 people about their concerns with AI, they found that unreliability was the biggest worry the respondents had over AI, clearly outweighing concerns over jobs and the economy.

Source: Anthropic

So, What Does Responsible AI Use Look Like?

None of this is to say that you should stop using AI altogether in professional work. That would be like suggesting in the 1990s that companies resist computers and stick with typewriters.

Instead, there are practical strategies that you can implement to reduce the risk of hallucinations.

Write structured, specific prompts

Vague instructions yield vague, error-prone answers. Defining the task, format, and constraints clearly will improve the output quality.

For complex tasks, ask the AI model to explain its reasoning step-by-step (often called chain-of-thought prompting). This helps guide how the model processes a problem, reducing the likelihood of it jumping to incorrect conclusions.

In more structured workflows, provide the AI model with examples of a good answer (known as few-shot prompting). This can improve consistency by showing the model exactly how to approach a task.

Maintain human oversight and critically evaluate every output

Treat AI-generated content the way you would treat an unverified tip from an enthusiastic but unreliable colleague: useful as a starting point, not as a conclusion.

AI can assist with research, drafting, and summarization, but it shouldn’t be the final reviewer. You must read and take professional responsibility for every document before it is submitted or delivered.

Always verify against independent sources

Cross-reference with primary sources, authoritative databases, or subject-matter experts. If a citation can’t be independently confirmed, then it shouldn’t appear in a professional document.

For attorneys, no AI-generated case citation should appear in a filing, brief, or client communication without independent confirmation in an authoritative legal database.

Establish firm-level AI protocols

Individual good intentions aren’t enough. Firms and legal departments need written policies governing when AI can be used, for what purposes, and what verification steps are required before AI-assisted work is considered finalized.

Document your process

If a court or client questions how a document was prepared, the ability to demonstrate a responsible AI-use protocol, including the verification steps taken, should provide meaningful protection.

The Stakes Are Only Getting Higher

The reality is that AI hallucinations are not going away with new updates to reasoning systems.

Meanwhile, the scope of AI use will continue to expand. Earlier this month, Bloomberg reported that Americans are now turning to AI tools like Claude and ChatGPT to assist with their tax returns. Whatever you think of that development, it’s a clear signal that AI is migrating from productivity tool to core professional infrastructure, in contexts where errors can carry legal, financial, and personal consequences.

The professionals who will succeed aren’t those who refuse to use AI completely. Rather, they are those who treat AI output the way good lawyers treat witness testimony: useful evidence that’s always subject to scrutiny and never taken on faith.

Always check your work. That has always been the professional standard. AI, if anything, has only raised the bar.