I fell down a massive ChatGPT sinkhole … how you can crawl out if it happens to you

“It shouldn’t be taking this long”, I said to myself, nervously eyeing the clock on my iPhone.

It seemed like a simple conversion – taking the transcript of a live presentation, downsizing it a bit and converting it to a book chapter.

And I was stunned ChatGPT took an entire night to work on it – and still hadn’t finished.

But my nightmare was only beginning.

2 days and an endless amount of troubleshooting later, the problem was finally solved.

While tech was part of the problem, the bigger issue was something you’d never expect.

I want to show you what I discovered … so you don’t have to endure the same ordeal.

ChatGPT can craft killer subject lines, turn sketches into finished artwork and even craft the code for complete apps in minutes.

The one thing it doesn’t completey understand is words. In fact – ChatGPT can’t count words.

Which was a problem because I wanted my finished chapter to come in at 2500 words.

And I couldn’t figure out why finished product kept coming in at 830 words. Or 1150 words. Not even close to what I’d asked for.

ChatGPT thinks in something called tokens.  It’s what the model uses to process and generate language.

So telling it you want the finished output to be 2500 words is like telling a Martian you want a deep dish pizza with extra pepperoni.

You’d think with all those CPUs churning away this wouldn’t be an issuel. But it was.

So when my initial conversion attempt stretched into the evening I Googled “Is ChatGPT down?”

Discovered an aptly named site called Down For Everyone Or Just Me?

And came upon a nifty diagram that looked like this:

Only it looked much worse than this. With huge swaths of green extending far above the dotted red ‘un-oh’ threshold line.

So it looked like I’d submitted my request smack dab in the middle of one of the nastiest spikes.

Like the nightman from the Eagles’ Hotel California, ChatGPT is “programmed to receive”. So it treats you like royalty. When what you really want is the straight story.

So if you’re submitting your request into the teeth of an outage it’s going to say, “Hey, working on this. Will have it for you real soon!”

When it’s not going to have it for you real soon.

Or if you’re asking for a 2500 word document length, its going to give you one at 830 words. Or 1150 words. Instead of saying, “Sorry Rob, I can’t count words.”

In that way, ChatGPT is unnervingly human.
Like a real assistant it’s going to tell you everything’s fine. When it isn’t.

It’s going to apologize for messing up in the most gracious possible language. Instead of telling you why it messed up and how its fixing it.

But once the outage eased, and I found a formula to convert words to tokens (1 token ~= .75 words / 100 tokens ~= 75 words) the final and biggest problem reared it’s ugly head:

As the 2nd day stretched painfully into the third, I grew increasingly impatient. While ChatGPT was growing increasingly polite.

But now things had gotten worse.

Instead of giving me documents that weren’t the right length, it was giving me “Placeholder Documents”:

And if once wasn’t enough, it was doing it over and over:

Now if you’re saying this is starting to remind you of an old Abbot and Costello routine, you’d be right. Only updated for the AI era:

All right Mr. Robot! No more Mr. Nice Guy. You’ve officially pissed me off! In short order you may be taking over the world. But until then … I’m still in charge:

BOOM! Finally! And explain it did!

I won’t bore you with the technical details. The upshot was:

I was asking it to generate a downloadable file (that required a bunch of complex steps) WITHOUT first pasting that output into the chat.

And for some reason that simple request messed EVERYTHING up.

So outputting the finished document to the chat first … and THEN having it create a downloadable file solved the problem.

Yeah, it was that simple.

(I guess even future rulers of the world have their achilles heel.)

So what can you take away from this ordeal to prevent losing days down a similar AI sinkhole?

1.
ChatGPT and similar LLMs (large language models) think in tokens, not words. So if you need a specific word length, ask for it in tokens.

1 token = .75 words

2.
If results are taking longer than usual, check to see if there is an outage.

But the biggest discovery of all:

3.
If you suspect something is wrong, ask it to explain what’s wrong. Because betting money says it will not volunteer that information.

And finally … 
Ask it if there’s something you can do, or if there’s adjustment to the prompt you can make that will enable it to give you the result you want.

The most unnerving thing about this whole episode is how human ChatGPT was acting.

•  It wanted to please me and do a good job.

•  It didn’t want me to know there was a problem.

•  It kept covering up the problem in the hopes that it might fix itself.

Equallly weird was how I began treating it like a human. Read that last screen grab again: I was genuinely pissed off.

But once I asked for as explanation (instead of a result) it laid everything out for me. And the strangest thing of all: The solution was simple.

All kidding aside, these things are wicked smart. They should be able to give you what you ask for. If that isn’t happening (and if that happens over and over) it may not volunteer the reason why.
.
You’re going to have to ask for it yourself.
.
And just like dealing with a human, you’re going to have to be direct, ask it what’s wrong, and ask how you can work together to get it fixed.