ChatGPT + Bing Shenanigans

Last night I was curious to know why my new cellphone + laptop charger is so small (after all my laptop needs >65W to charge!). I researched and read papers for almost an hour, then I asked ChatGPT, then I asked The New Bing. Here's what I found about LLMs, citations and sources. Enlightening.

Story time. I likely spent more time than I should've researching about semiconductors' physical properties, about how gallium-nitride (GaN) has a higher electron mobility than the good ol' silicon-carbide (SiC). Then I decided to ask ChatGPT and compare notes.

As expected, it gave me an accurate answer within seconds (and more articulate than I could've ever done), but the only reason I knew that the information was true it's because I had already done the research myself. I asked ChatGPT to give me citations, it gave me a list of links that seemed legit but linked to nothing (404-NotFound), or to different unrelated articles. Yikes.

Of course I asked the model why should I trust the information it just gave me without proper sources? And it gave me a simple answer: "I'm just a LANGUAGE MODEL, and rely on my training data to generate responses".

It made total sense, and I was disappointed. I can't trust this thing first hand, it knows how to write and summarize pretty well, but the facts... well. Who knows.

Today, I tested ChatGPT + Bing, The New Bing (or Bing Chat, currently on public preview). And it was absolutely amazing, Bing added just what the whole thing was missing: the references!

I guess the next step would be to validate the referenced pages and get some sort of weight or ranking on each reference, but this is a great start!