Edition 4: AI generated music, art, dragons and girlfriends

Welcome to the Machine Learning Memo, where we’re back to usual programming this week:

• What I cooked up in Google’s AI Test Kitchen
• Why education is key to developing generative AI in a safe way
• What the most prompted artist thinks about Stable Diffusion
• Why I’m terrified of virtual girlfriends

MusicLM

Good news everyone!?I finally got access to Google’s AI Test Kitchen.?Those of you who know me know how much of a massive music nerd I am (I play piano, cello and drums), so the first thing that I tested was MusicLM.?It generates tracks from text prompts.

For my first prompt, I tried to be prescriptive as possible and honestly, it didn’t work.?Instead of a waltz I got a piece of music in 4/4.?Cellos, as ever, got the boring job of a plodding baseline.?And there wasn’t any tremolo.

So I decided to make it less prescriptive:

I was hit with a wave of dissonance. It was still not a waltz, but the cello was the star of the show this time at least.?I continued to try, mixing up my prompts, but it wasn't responding to "technical" music language. I would also be willing to bet that the majority of training data for this model was 4/4 music, or time signatures weren't considered a feature.

Overall I feel like I’m being picky for saying that a piece of AI that can generate classical music in about 3 seconds fell short of my expectations, but honestly it just didn't respond to my prompts.?The quality of the audio is impressive though - the strings actually sounded like strings! I decided to move on and mix up genres to see if it could do better.

This time I focused more on emotion rather than the technical side of music. When I asked it to create a piece of elevator music with jazz drums and relaxing saxophone, it absolutely nailed it. I was genuinely blown away. When I moved back to being more prescriptive (including terms like 90s house music, house piano) it started falling flat again.

Overall, I want to have faith in this technology and I would say that this feels like a “user error” in a way.?In order to get the most out of this tool it feels like you really need to learn how to talk to it, as the results generated by Google in demos have been impressive.

If you want to hear any of the samples feel free to drop me a DM and I can share them with you. I'll update this article when LinkedIn allows us to attach audio clips :)

Why education is key to developing generative AI in a safe way

The ever impressive Anne-Marie was on Sky News this week, and she did a fantastic job at taking some of the heat out of the debate surrounding generative AI at the moment.?It does feel a bit doom-and-gloom out there at the moment, so it was refreshing to hear a balanced take on the risks and opportunities associated with Generative AI.

I particularly agreed with her point around making sure that everyone is included in understanding the developments that are coming our way in order to build generative AI in a balanced way.?It’s imperative to ensure that the data scientists, engineering teams, designers, product managers, content creators, writers, marketeers, customers… basically everyone who is a stakeholder in delivering an AI system… are a) aware of bias and the impact of bias on generative AI systems, and b) are a diverse group themselves.?We need to be considering the impact of generative AI from all angles, and the most authentic way of achieving this is to make sure that the systems themselves are being designed by teams that reflect the diversity of society itself.

What the most prompted artist thinks about Stable Diffusion

“dragons fighting in a ruined London scenescape, style of Greg Rutkowski”

It’s super cool, and I’m the first to admit that all my Christmas cards last year were made by Stable Diffusion.?I had great fun thinking of prompts that would be fun and personalised for my friends, yielding some hilarious artwork.?(You can read more about it here).

But how does it feel for the actual artists??The most “prompted” artist, Greg Rutkowski, has spoken out about how generative AI has had a negative impact on his mental health.?

“I felt, and I still feel, really bad about it, like, bad.?I’m shocked and anxious actually.”

He has spent a lifetime honing his artist talents, with his paintings taking an average of 20-40 hours to create, yet it only took me a few seconds to create those examples above using generative AI.?You can read more about how Greg feels about Stable Diffusion here.

It’s got me thinking:

• Should we be excluding artists’ IP from training datasets, or letting them actively opt-in?
• If they opt-in, should they receive royalties?
• Is opting out even possible at this point??For instance, there is now a vast body AI generated derivative artworks from Greg Rutkowski like the ones I made above.?It’s technically not "his" work as in he didn’t paint it, so would he have the right to exclude this from a training dataset?

These kinds of questions apply everywhere, from writing to art to code to music.?What do you think?

Why I’m terrified of virtual girlfriends

Deep-fakes and generative AI as a whole are making the previously sci-fi idea of virtual girlfriends become a reality.

A Snapchat influencer, Caryn Marjorie, partnered with Forever Voices to create a virtual girlfriend version of herself.?She offers advice, she invites people on virtual dinner dates, and she costs a dollar a minute.?She allegedly made $72k in her first week.

I should practise what I preach and be unbiased, but honestly this creeps me out.?However, there was an argument raised by Caryn around how these kinds of services could help with the problem of loneliness.

What do you think?

Like it or not, there are more and more companies springing up in this space.?GirlfriendGPT has recently been announced, by a sole developer looking to open-source the ability for people to create their own girlfriends.?Your virtual girlfriend can send you selfies and have a chat with you via Telegram voicenotes.

Tencent has also entered the custom deepfake space (more here).?For $145 a pop, you can give them 3 minutes of audio (covering 100 different sentences) in either English or Chinese, and they’ll create you a virtual avatar.?As someone with a load of video content of me on YouTube giving presentations over the years, I find it utterly bizarre that someone could effectively create an AI clone of me for just over 100 quid. ?I’m not saying anyone would want to, but this must play on the minds of more prominent figures than me!

As an aside, after writing this article I dread to think what my targeted ads will look like.?Onwards and upwards eh.

I'd love to hear your thoughts on this week's newletter. Also feedback is always welcome. I'm becoming more opinionated here - what do you think?

---

In case you don’t know me, my name’s Jenny.?I work at Oracle as VP of Data Science, but all views expressed here are my own.?I try to cite others’ work whether possible, so please follow links in my writing to dive deeper into topic areas, written by others.?I’m not responsible for the contents of those links.