Learning about Generative AI
Bing Chat / DALL·E 2 Image of Pomeranian George Washington Riding a Pomeranian Horse

Learning about Generative AI

I've been playing with Generative AI and Large Language Models (LLMs) lately and thought I would share my experience and learnings. This isn't anything official, just some interesting notes from my learning journey.

I mostly played around with the Bing chat interface, focused on image generation, and did creative variations on similar themes to try and test its limits.

It's been fascinating to see what this technology is good at, what it isn't good at, and what is new and different than I expected.

A couple of my early queries resulted in some very impressive results

No alt text provided for this image
Bing Chat / DALL·E 2 Image of a hacker pomeranian in a hoodie

There were a few little artifacts around some of the visuals (whiskers blended with eyelines, etc.) but overall very realistic and genuinely impressive.

When I started to push the envelope in various creative ways, it started to break down and get a little weird in places like this one.

No alt text provided for this image
Bing Chat / DALL·E 2 Image of a Pomeranian fighting(?) with a ??????

It seems to do well when the topics are simple/straightforward or there is a clear surface connection between those things like this pasta version of the dog:

No alt text provided for this image
Bing Chat / DALL·E 2 Image of a Pomeranian made of pasta

When you start driving into truly novel or creative areas without much precedent or example, it seems to really show its limits. Sometimes it comes up with something really broken (it is terrible at fingers and sometimes bad at faces - I will spare you that ugliness), and sometimes it comes up with really interesting things like this "orchid but as a cat"

No alt text provided for this image

I really liked the ability to go "Back and forth" with it and quickly try new ideas. This 'dialog' created a very nice creative dynamic that would otherwise require two or more people to achieve. The above picture inspired me to try a new query of an evil owl with a mushroom hat that resulted in some strong results.

No alt text provided for this image

As I stepped back to consider my experience, I noticed that this technology is very strong at things that people presumably have already created a lot with many good examples. I noticed it had particular weaknesses around understanding the physical world beyond what is depicted or represented already (e.g. failing at fingers, faces, etc.) it doesn’t seem to 'understand' that these things it's drawing are meant to be a three-dimensional object, it just 'understands' the patterns and lines of the pictures.

My understanding is that the text models are similar in that they only learn language patterns, they are not expressing an actual thought/idea/emotion (which is what humans use language for).

I later found this great quote that does an excellent job describing what the models don't do which is very consistent with my experience. From 4 tips for spotting deepfakes and other AI-generated images : Life Kit : NPR

"They don't have models of the world. They don't reason. They don't know what facts are. They're not built for that," he [Gary Marcus] says. "They're basically autocomplete on steroids. They predict what words would be plausible in some context, and plausible is not the same as true."

I also found this fascinating article that describes some of the second and third order effects of the increased usage of these models. ?

https://venturebeat.com/ai/the-ai-feedback-loop-researchers-warn-of-model-collapse-as-ai-trains-on-ai-generated-content/

?Hopefully this was helpful for y'all.

I welcome any comments from experts correcting on anything I got wrong or that I missed.

That Evil Owl looks awesome!

Tony Carrato

Consulting Architect at Independent (Semi-Retired), Board Member, Standards Author, Investor

1 年

Good analysis & article, Mark. Too many people, who’ve either never tried out one of the available AIs or at least tried very little are offering “expert” opinions. You’re views are very useful and clearly informed by actually doing things.

要查看或添加评论,请登录

Mark Simos的更多文章

  • Clarity Matters: Identity and Access Capabilities

    Clarity Matters: Identity and Access Capabilities

    I am working on a proposed revision to the Zero Trust Reference Model from The Open Group and wanted to get your…

    5 条评论
  • Words Matter #3 - Incident, Compromise, and Breach

    Words Matter #3 - Incident, Compromise, and Breach

    The Open Group is working on updated definitions for various Security and Zero Trust terms for an upcoming security…

    18 条评论
  • Security Roles and Responsibilities

    Security Roles and Responsibilities

    Security is a team sport across the organization If you think that "security is the security team's job", you will have…

    11 条评论
  • Words Matter #2 - Security Policy and Security Policy exception

    Words Matter #2 - Security Policy and Security Policy exception

    The Open Group is working on updated definitions for various Security and Zero Trust terms for an upcoming security…

    9 条评论
  • Words Matter: Trust and Trustworthiness

    Words Matter: Trust and Trustworthiness

    What is Trust? Do we really need Zero of it? What about Trusting AI? The Open Group is working on updated definitions…

    26 条评论
  • Security Roles

    Security Roles

    Nikhil Kumar and I found that we had to create a list of roles impacted by cybersecurity for the Zero Trust Playbook…

    1 条评论
  • Zero Trust Playbook - Cutting through the complexity

    Zero Trust Playbook - Cutting through the complexity

    We recently announced the first book in the Zero Trust Playbook Series (https://zerotrustplaybook.com) that is aimed at…

    7 条评论
  • Microsoft Secure Future Initiative (my thoughts)

    Microsoft Secure Future Initiative (my thoughts)

    I highly encourage everyone to read this blog from Microsoft on the Secure Future Initiative (SFI). I have been with…

    14 条评论
  • SecOps Tools Strategy in 2023, Part 1

    SecOps Tools Strategy in 2023, Part 1

    I really enjoyed reading Anton Chuvakin's recent posts on SOC tools (https://www.linkedin.

    17 条评论
  • Worship neither tradition nor technology's sparkle

    Worship neither tradition nor technology's sparkle

    Worship neither tradition nor technology's sparkle. Both are valuable beyond measure, but each alone will blind you and…

    2 条评论

社区洞察

其他会员也浏览了