ChatGPT isn't a Trekkie (AI research)

My original title for this was "ChatGPT sucks at Research" but that seemed unkind.

This observation is based a number of informal experiments I've done lately. The basic format is to ask the AI a question to which I already know the answer.

The best example is "how many starships were built by starfleet in the original star trek? What were their names?" because everyone knows the answer to that.

[What? You don't know the answer? It's TWELVE. Kirk says it explicitly in one episode. (About HALF of which were described as lost or visibly destroyed onscreen. It wasn't just the redshirts that regretted joining StarFleet.)]

Trying this identical question on Google produces the correct answer, if you use your human brain to skip through the first few responses.

My OpenAI transcript originally produced an answer that was incomplete and inaccurate. It produced the correct response only after I outright pointed it at an authoritative source.

You could quibble that my original question was open to interpretation. But - in an ideal world - the AI would appreciate that and respond with clarifying questions like:

• "Do you mean starships like the Enterprise (Constitution class) or any Starfleet vessel capable of warp?"
• "Do you mean starships that were named in the TOS episodes explicitly? "

Here are my takeaways.

• The AI didn't ask those clarifying questions but a human assistant would.
• The original answer produced was quick, clear and convincing, but incomplete and inaccurate.
• This was research for information that is widely available on the Web, and predates Sept 2021 (last date at which OpenAI trained ChatGPT). What could I expect for information that DIDN'T meet those conditions?

Sooo, don't ask ChatGPT a question .. to which you don't already know the answer.

However, if I wanted the (known) answer to be presented at some length and with background material and a clear, fluid presentation, I'd use ChatGPT and save myself the time of preparing this material.??