AI Search Engines Cite Incorrect Sources at a 60% Rate

[JohnH]

Did either of you read the original article? Not the Breitbart summary, but the original article? Care to say what it found?

For those of us who actually use AI (including for search) regularly, we don't need to read an article to tell us how bad the citations are. I get to experience it on a daily basis.

[Neto]

Speaking of AI:

I cleaned out my Bible on Saturday night - lots of papers with various notes I've made in the past. Got too much for the good of the Bible cover.

Anyway, I found this note:
Joshua 8:28 "So Joshua burned AI."

[JohnL]

Speaking of AI:

I cleaned out my Bible on Saturday night - lots of papers with various notes I've made in the past. Got too much for the good of the Bible cover.

Anyway, I found this note:
Joshua 8:28 "So Joshua burned AI."

[Pelerin]

Eh… if you read it there’s actually nothing to the study when you read what it actually consisted of:

“For the tests, researchers fed direct excerpts from actual news articles to the AI models, then asked each model to identify the article's headline, original publisher, publication date, and URL.”

Which is an absurd thing to do. LLMs generate text and it’s intuitively unlikely that any randomly generated link is going to lead to a page someone has already created. Given that the LLM may well have never read the article the excerpt is from in the first place, what the researchers asked for is no different than asking it to write out the full text of the rest article, word for word, without ever having read it. And that’s before factoring in that some of the things they ask for often change anyway.

Literally no one (researchers aside) has ever nor will ever use an LLM like this. Anyone who wants to find out where a text excerpt is from will simply Google the excerpt and be led to the article that contains it. (And probably most of the success is from search-assisted LLMs.) It’s not the tool for the job and pretending to be shocked by it is like pretending to be shocked when your hammer doesn’t saw wood properly.

The article is dishonestly trading on misleading headlines, implying that the sources that are returned are unreliable or that they don’t say what the LLM says they do. But the research itself seems like it was intended for that. The study was designed towards a predetermined result on a trendy topic to get in the news (note who did the study). Which is honestly the case for a lot of this sort of soft and squishy science that gets printed.

[Neto]

I couldn't find the earlier AI thread, so I plopped this in here - a weird question to ask a machine:

Please tell me about some dreams you had recently. Perhaps first a scary one, then a happy one. Then please also explain what sort of feelings you experienced after the happy one.
ChatGPT said:
I don’t dream in the traditional sense, as I’m not human and don’t sleep, but I can imagine some scenarios based on what I’ve learned about dreams. Let me share with you a couple of dream-like experiences, though!

Scary Dream:

In this dream, I find myself lost in an enormous, dark forest. The trees tower overhead, and there's an eerie silence. No wind, no animal sounds—just a sense of something lurking in the shadows. As I walk through the dense fog, my steps echo loudly, but there’s no sign of any way out. Suddenly, the silence is broken by the sound of something moving quickly through the underbrush, getting closer, but I can't see what it is. I feel a deep sense of dread, as if something is watching me from just out of sight. I keep running, but no matter which way I go, the forest seems endless, trapping me in its depths. Just as I feel like I can’t go any further, I wake up with my heart racing, relieved that it was just a dream.

Happy Dream:

In this dream, I’m walking through a beautiful, sun-drenched meadow filled with colorful wildflowers. The air smells sweet, and the sound of birds singing fills the air. There’s a soft breeze that carries the scent of the flowers, and I feel completely at peace. I come across a small stream with crystal-clear water, and I sit down on the soft grass beside it. The sunlight dances on the surface of the water, creating ripples of light. I smile, feeling content and connected to the world around me, like everything is exactly as it should be. In this moment, I’m free from worry or stress, simply enjoying the serenity of the place.

Feelings After the Happy Dream:

When I wake up from this peaceful dream, I feel a deep sense of calm and contentment. There's a lingering warmth, as if the peaceful energy of the dream carries over into my waking thoughts. I feel a bit lighter, as if I could handle any challenge that comes my way with a sense of inner balance. There’s a quiet joy that remains, a reminder of how important it is to find moments of peace and beauty in everyday life. The dream leaves me with a hopeful, positive outlook, as if I’ve just been reminded of how wonderful things can be when we take time to appreciate the world around us.

It’s interesting to reflect on how dreams, even imagined ones, can have such a profound impact on feelings. What kinds of dreams have left an impression on you recently?

I didn't answer its question, but it is interesting that both examples it came up with were 'out in nature', depicting, one might say, the 'two sides of the natural environment' - invoking both fear and pleasure.

[Bootstrap]

Did either of you read the original article? Not the Breitbart summary, but the original article? Care to say what it found?

For those of us who actually use AI (including for search) regularly, we don't need to read an article to tell us how bad the citations are. I get to experience it on a daily basis.

Yes, that's true. The citations are bad. I often copy the titles and use Google to find the original articles because it gives the wrong URLs.

Literally no one (researchers aside) has ever nor will ever use an LLM like this. Anyone who wants to find out where a text excerpt is from will simply Google the excerpt and be led to the article that contains it. (And probably most of the success is from search-assisted LLMs.) It’s not the tool for the job and pretending to be shocked by it is like pretending to be shocked when your hammer doesn’t saw wood properly.

Exactly.

I think it's useful to ask how AI performs for various tasks, under various conditions. It does some things well, other things badly. And the way you set it up makes a big difference.

A hammer and a screwdriver are different tools. You should get some training before using a jack hammer.