When AI Gets Creative (A Little Too Creative): The Gamma vs. ChatGPT Experiment

I just had one of those "this is why we can't have nice things" moments with AI.

The Setup

I decided to create a slide deck about some well-known, high-profile books. The kinds of books with tons of data floating around the internet. Should be easy, right? I wanted book covers, author photos, author names, and some solid quotes from reputable reviewers. Basic stuff.

Enter Gamma (with ChatGPT-5 beta agent)

Here's where it gets wild. I used Gamma's AI Agent (beta) feature, which supposedly runs on ChatGPT5. And it fully hallucinated almost everything.

I'm not talking about small errors or slightly off information. I mean:

Made-up book covers Wrong author names Fabricated author photos Completely invented book quotes Completely invented a book that was not even listed in the prompt

Like, not even close. It was confident though! Gave me everything I asked for with the audacity of someone who absolutely knows what they're talking about. Except none of it was real.

Article content Example of Gamma hallucination with annotation Plot Twist: ChatGPT Does It Right (Mostly)

So I took the exact same prompts and ran them through ChatGPT directly (paid version). And boom, it delivered everything I needed. Real book covers with image links. Actual author photos. Proper citations. It was like night and day.

But Wait, There's More

Here's the funniest part: ChatGPT mixed up the quotes. I asked for quotes from reputable book reviewers, and instead it gave me:

A quote about Karen Hao's book... from Andrew Wu A quote about Andrew Wu's book... from Karen Hao And for The Age of Surveillance Capitalism? It gave me a quote from Shoshana Zuboff... the actual author of the book, praising her own work

The authors were reviewing each other's books, and one was apparently reviewing her own? Not exactly what I meant by "reputable book reviewer source," but at least these people actually exist!

What Does This Tell Us?

This experiment was a masterclass in why the tool matters, even when the underlying model is supposedly the same. Gamma's implementation of what should be powerful AI completely failed at basic fact retrieval, while ChatGPT's interface, with its ability to search and verify information, got me 90% of the way there.

It's a good reminder that we're still in the "trust, but verify" era of AI. And maybe keep your screenshots of the hallucinations for a good laugh later.

Have you had any wild AI hallucination experiences lately? I'd love to hear your stories!