Learning about Generative AI

I've been playing with Generative AI and Large Language Models (LLMs) lately and thought I would share my experience and learnings. This isn't anything official, just some interesting notes from my learning journey.

I mostly played around with the Bing chat interface, focused on image generation, and did creative variations on similar themes to try and test its limits.

It's been fascinating to see what this technology is good at, what it isn't good at, and what is new and different than I expected.

A couple of my early queries resulted in some very impressive results

There were a few little artifacts around some of the visuals (whiskers blended with eyelines, etc.) but overall very realistic and genuinely impressive.

When I started to push the envelope in various creative ways, it started to break down and get a little weird in places like this one.

It seems to do well when the topics are simple/straightforward or there is a clear surface connection between those things like this pasta version of the dog:

When you start driving into truly novel or creative areas without much precedent or example, it seems to really show its limits. Sometimes it comes up with something really broken (it is terrible at fingers and sometimes bad at faces - I will spare you that ugliness), and sometimes it comes up with really interesting things like this "orchid but as a cat"

I really liked the ability to go "Back and forth" with it and quickly try new ideas. This 'dialog' created a very nice creative dynamic that would otherwise require two or more people to achieve. The above picture inspired me to try a new query of an evil owl with a mushroom hat that resulted in some strong results.

As I stepped back to consider my experience, I noticed that this technology is very strong at things that people presumably have already created a lot with many good examples. I noticed it had particular weaknesses around understanding the physical world beyond what is depicted or represented already (e.g. failing at fingers, faces, etc.) it doesn’t seem to 'understand' that these things it's drawing are meant to be a three-dimensional object, it just 'understands' the patterns and lines of the pictures.

My understanding is that the text models are similar in that they only learn language patterns, they are not expressing an actual thought/idea/emotion (which is what humans use language for).

I later found this great quote that does an excellent job describing what the models don't do which is very consistent with my experience. From 4 tips for spotting deepfakes and other AI-generated images : Life Kit : NPR

"They don't have models of the world. They don't reason. They don't know what facts are. They're not built for that," he [Gary Marcus] says. "They're basically autocomplete on steroids. They predict what words would be plausible in some context, and plausible is not the same as true."

I also found this fascinating article that describes some of the second and third order effects of the increased usage of these models. ?

https://venturebeat.com/ai/the-ai-feedback-loop-researchers-warn-of-model-collapse-as-ai-trains-on-ai-generated-content/

?Hopefully this was helpful for y'all.

I welcome any comments from experts correcting on anything I got wrong or that I missed.