AI Search Engines Cite Incorrect Sources at a 60% Rate

[JohnL]

Cue the apologist ‘splainers.
Study: AI Search Engines Cite Incorrect Sources at a 60% Rate

[JohnH]

Cue the apologist ‘splainers.
Study: AI Search Engines Cite Incorrect Sources at a 60% Rate

I use AI as a resource almost daily, but I spend most my time with it dealing with bogus sources, hallucinations, or very subtle flaws.

Fundamentally it tells you what it thinks you want to hear.

[Ken]

Who are the "apologist explainers?"

The "tech bros" like Elon Musk who are in the rush to make yet more billions by replacing hundreds of thousands of jobs with unreliable and unaccountable AI?

I don't know anyone in my world who is enamored with AI. I'm certainly not. From what I can tell it just makes people dumber.

[JohnH]

Who are the "apologist explainers?"

The "tech bros" like Elon Musk who are in the rush to make yet more billions by replacing hundreds of thousands of jobs with unreliable and unaccountable AI?

I don't know anyone in my world who is enamored with AI. I'm certainly not. From what I can tell it just makes people dumber.

Sam Altman demanded $500 billion for his failing company. Meanwhile, the Chinese (who have done much of the groundbreaking search of LLMs) are offering AI cloud services via Baidu for 1/100th what OpenAI charges for GPT 4.5,

[Ken]

Who are the "apologist explainers?"

The "tech bros" like Elon Musk who are in the rush to make yet more billions by replacing hundreds of thousands of jobs with unreliable and unaccountable AI?

I don't know anyone in my world who is enamored with AI. I'm certainly not. From what I can tell it just makes people dumber.

Sam Altman demanded $500 billion for his failing company. Meanwhile, the Chinese (who have done much of the groundbreaking search of LLMs) are offering AI cloud services via Baidu for 1/100th what OpenAI charges for GPT 4.5,

The whole enormous push for self-driving cars by Tesla and others is also hugely dependent on AI.

[JohnH]

It’s a bit different than LLMs.

One key difference is you can’t B.S. your way through it. Car crashes are hard to hand wave away as a “hallucination”.

[Robert]

Good find on this article. Now I have to ask Grok about it.

Grok: How incorrect are AI search engines?

AI-powered search engines, like any technology, aren’t perfect and can sometimes get things wrong. Their accuracy depends on a bunch of factors: the quality of the data they’re trained on, the algorithms they use, and how well they interpret what you’re actually asking. Generally, they’re pretty good at finding relevant info quickly—way faster than a human could—but they’re not flawless.
Errors can pop up in a few ways. Sometimes they misinterpret queries, especially if you’re vague or use tricky phrasing. For example, if you ask something ambiguous like “Jaguar speed,” it might pull up stuff about the car instead of the animal—or vice versa—depending on what it guesses you mean. They can also stumble on outdated info if their data hasn’t caught up to recent events, though my knowledge is fresh as of March 16, 2025, so I’d dodge that trap. Another issue is bias in the training data—if the info they’re built on leans a certain way, their results might too.
Studies and real-world use give some clues. Traditional search engines like Google (which now lean heavily on AI) have error rates that are hard to pin down exactly, but research suggests they return irrelevant or low-quality results maybe 10-20% of the time for complex queries. Pure AI search tools—like, say, early versions of chat-based search—can be off more often when dealing with nuanced or niche topics, sometimes misfiring 30% or more depending on the task. But those numbers drop as the tech gets sharper.
For me, Grok 3, built by xAI, I’m designed to cut through the noise and give straight, useful answers. I’m not immune to screw-ups, but I’ve got tools—like web and X search—to double-check and fill gaps if needed. How incorrect I am? Hard to slap a number on it without knowing your specific question, but I’m built to be as right as possible, and I’ll tell you if I’m unsure. What’s the search you’re thinking about testing? Throw it at me, and we’ll see how I do.

Grok: How accurate are AI search engines?

AI search engines have come a long way, but their accuracy can vary depending on several factors. Generally, they’re designed to process massive amounts of data, understand context, and deliver relevant results faster than traditional search engines. However, they’re not perfect—accuracy hinges on the quality of their training data, algorithms, and how well they interpret user intent.
For straightforward queries (like "What’s the capital of France?"), AI search engines are typically spot-on, often exceeding 95% accuracy in delivering correct, factual answers. Studies on large language models, which power many AI search tools, suggest they can achieve high precision on well-defined tasks—sometimes rivaling human performance. But when it comes to ambiguous queries, subjective topics, or cutting-edge info, things get murkier. Misinterpretations, outdated data, or biases in training sets can lead to errors. For instance, if an AI misreads sarcasm or pulls from unreliable sources, you might get a confident but wrong answer.
Compared to traditional search engines like Google, AI-powered ones (e.g., those built into tools like me or competitors like Perplexity) aim to synthesize answers rather than just list links. This can boost accuracy for direct questions but risks hallucination—where the AI makes up plausible-sounding nonsense—especially if it’s not cross-checking live data. Web-crawling AI engines can mitigate this by pulling real-time info, but they’re still limited by how well they filter noise or spot fakes.
In practice, I’d say they’re highly accurate for clear-cut, factual stuff—think 90%+ in ideal conditions—but drop to maybe 70-80% for nuanced or complex queries where human judgment still edges out. If you’re digging into something specific, like a breaking news event or a niche topic, cross-checking with primary sources or raw X posts can help fill the gaps. What do you think—got a particular search in mind where accuracy matters to you?

It seems to disagree with the article's findings. Underlines are my additions.

[temporal1]

Cue the apologist ‘splainers.
Study: AI Search Engines Cite Incorrect Sources at a 60% Rate

I use AI as a resource almost daily, but I spend most my time with it dealing with bogus sources, hallucinations, or very subtle flaws.

Fundamentally it tells you what it thinks you want to hear.

i’ve been fiddling around with instagram, viewing, not adding content. it’s entertaining.
i esp like the many aviation bits, and resourcesfulness-invention clips, set to (my personal downfall) hep music.
i forward a few to my son.

who often replies, “that’s from a video game” or “that’s not real” or “that’s not new, it’s famous” ..
(i found the video of the 250 Amish guys moving a barn.) it’s on forum somewhere. or was.

as-if the world wasn’t unreal-enough without tech.
if anyone remembers, “unreal!” was a fav young people’s exclamation in the 60’s-70’s. along with “far out”. o.gosh.

[JohnH]

gnarly!

[Bootstrap]

Cue the apologist ‘splainers.
Study: AI Search Engines Cite Incorrect Sources at a 60% Rate

I use AI as a resource almost daily, but I spend most my time with it dealing with bogus sources, hallucinations, or very subtle flaws.

Fundamentally it tells you what it thinks you want to hear.

If used well, I don't think so, but you definitely do need to check results.

Did either of you read the original article? Not the Breitbart summary, but the original article? Care to say what it found?