Calling BS on AI

Somethin' ain't right.

Despite the breathless claims, Devin can't seem to code as fast as people. In fact in one test it was a lot slower: 36 minutes for the real person vs. 6 hours for the bot. If anything it seems to be making the work more complicated. Sound familiar?

Google's AI tells people to eat rocks and says dogs have played in the NBA (trash in, trash out).

Can we, just for a moment, get some real thinking in the house?

Generative AI's ability to "reason" is, at present, nothing more than a super-sized version of the predictive text you see on your iPhone. It's not sentient or reliable: When I asked the LinkedIn image generator to make something for this column, it refused, telling me I had violated its guidelines (BS is a violation? Really?) Copilot, by the way, kept misspelling words even after I gave it the correct spelling.

Let's also consider Google’s claim that it used A.I. to discover more than two million new chemical compounds. Nope. A review by experimental materials chemists at the University of California found “scant evidence for compounds that fulfill the trifecta of novelty, credibility and utility.”

Aside from my own experience, I owe much of the reporting here to two sources: Julia Angwin, who wrote a must-read Op-Ed piece in the NY Times; and a fascinating video (if you like coding) made by the software engineer who debunked (not debugged) Devin.

This is important information for everybody, particularly people who have money in a stock market that has gone maybe a little over the rainbow on AI hallucinogens. In her article, Angwin asked, "Should we as a society be investing tens of billions of dollars, our precious electricity that could be used toward moving away from fossil fuels, and a generation of the brightest math and science minds on incremental improvements in mediocre email writing?"

I still think AI will eat the world, just not in one gulp.

So let's slow the hype struttage, because that brings its own set of problems. Sam Altman, I'm talking to you. Was it really helpful to describe a routine ChatGBP upgrade as something that "feels like magic?" Was it helpful for a NY Times writer to panic after a two-hour conversation with Bing's AI? Certainly AI is starting to prove itself in the enterprise, including in cybersecurity, and a new study suggests it can do a better job at financial forecasting than people.

Yes, I use AI. I even tried to attach to the hype by mentioning it in my LinkedIn profile. I use it to generate images, to bounce ideas around when I'm stuck, to improve titles and to do some research, though I have to check everything. Sometimes it helps. I think it will improve. Would I trust it to summarize, say, a 5,000-word scientific article? No way. I would have to re-check everything.

Its writing is ok, but I'm not yet blown away. It's not bad for a first draft, sometimes. I understand this sounds like somebody who wants to keep his buggy whip factory going, but the stuff it produces now feels synthetic, generic, voiceless, bland. It's McDonald's, meal after meal, day after day.

Ah geez, now ChatGBT is going to tell somebody that if they want to be a good writer, avoid McDonald's.