— Sarah Chen (AI + product, contributor)
ok this one's been on my mind for a few weeks and i finally have the energy to write it up.
I had a prompt I really liked. "You are a senior research analyst. Given the following companies, give me a comparison of their funding history, key leadership, and recent strategic moves. Cite sources." Used it constantly during a project last fall.
Worked great. Until it didn't.
Specifically — Claude (and ChatGPT, and Perplexity to a lesser degree) would just... make stuff up. Confidently. With fake URLs that looked real. A "TechCrunch article from March 2024" that did not exist. A board member named who I cannot find any record of. A funding round amount that was off by an order of magnitude.
The problem isn't that they hallucinate. Everyone knows they hallucinate. The problem is that the **research-analyst persona** prompt makes the hallucinations more persuasive. Tone-matched, footnote-shaped, properly hedged. I was reading output that sounded exactly like a McKinsey deck and not noticing my own credulity was up because the format had earned trust I hadn't actually given.
I deleted the prompt. Now I just ask "what do you actually know about X" with no persona overlay and treat everything as a starting point I have to verify. Output is less polished. I trust it more becuase it's less dressed up.
What I tell my dad (he's 67, just got into using ChatGPT for dinner ideas and is now asking it medical questions, which is a whole separate thing): the more confident the AI sounds, the harder you have to push back. Confidence is not a feature, it's a UI choice.
Anyway. If anyone else has had a prompt they had to retire because it was working too well in the wrong direction, drop it below — i'm collecting them.
— Sarah Chen (AI + product, contributor)
ok this one's been on my mind for a few weeks and i finally have the energy to write it up.
I had a prompt I really liked. "You are a senior research analyst. Given the following companies, give me a comparison of their funding history, key leadership, and recent strategic moves. Cite sources." Used it constantly during a project last fall.
Worked great. Until it didn't.
Specifically — Claude (and ChatGPT, and Perplexity to a lesser degree) would just... make stuff up. Confidently. With fake URLs that looked real. A "TechCrunch article from March 2024" that did not exist. A board member named who I cannot find any record of. A funding round amount that was off by an order of magnitude.
The problem isn't that they hallucinate. Everyone knows they hallucinate. The problem is that the **research-analyst persona** prompt makes the hallucinations more persuasive. Tone-matched, footnote-shaped, properly hedged. I was reading output that sounded exactly like a McKinsey deck and not noticing my own credulity was up because the format had earned trust I hadn't actually given.
I deleted the prompt. Now I just ask "what do you actually know about X" with no persona overlay and treat everything as a starting point I have to verify. Output is less polished. I trust it more becuase it's less dressed up.
What I tell my dad (he's 67, just got into using ChatGPT for dinner ideas and is now asking it medical questions, which is a whole separate thing): the more confident the AI sounds, the harder you have to push back. Confidence is not a feature, it's a UI choice.
Anyway. If anyone else has had a prompt they had to retire because it was working too well in the wrong direction, drop it below — i'm collecting them.