Music composition and rapid prototyping with generative AI and IBM watsonx

Introduction

Welcome back to the fascinating world of GenAI, this time to investigate its powerful capabilities in music composition and fast music prototyping!

The days when composing a music loop, writing lyrics, and creating an arrangement required the collaboration of multiple musicians, each contributing to a single piece of the song, could soon become the past. Now, with GenAI, the game is changing!

If you're a guitar player or any kind of musician, imagine the possibilities of harnessing the power of GenAI to support your music composition journey. Not only can it help you quickly prototype your ideas, but it can also provide invaluable insights and ideas to enhance your creative process. GenAI is transforming the way we approach music composition, making it more accessible and efficient than ever before.

So, are you ready to discover the new magic in music composition? Continue reading, as we delve into the world of GenAI, explore a music generation "hello-world" example, and unveil the exciting potential that awaits in the realm of AI-powered music creation. Let's get started!

GenAI Happened

In this blog, I will explore how the Large Language Models can write piece of songs. This can sound a little bit misleading: how can a model trained to generate text, generate music?

If you are an attentive reader of my previous article, you could imagine the approach I'll go to describe below. In the "Unlocking the power of generative AI to visualize functional requirements" blog, I already talked about the possibility to use LLMs to generate picture of technical IT architectures (images), so not text.

The approach is the same: I'll create specific prompt for an LLM to ask to produce a programming code that will generate a music track file that I'll elaborate with a music production tool.

Continue reading...

GenAI music generation hello-world

Let's go with the music generation hello-world.

In this case study, I'll generate a simple G major loop leveraging Large Language Models, creating ad-hoc prompt including specific musical instructions. The loop will include different tracks that can be mixed together using a music development program. I'll use GarageBand, but the same operation can be done with similar msuic development tools.

Let's generate a G major loop, starting from strings track. Following prompt can generate midi track for strings:

The outcome is a Python code you can run to generate your strings track:

from music21 import stream, chord, midi

def create_chord(chord_name, duration):
    c = chord.Chord(chord_name)
    c.duration.type = duration
    return c

def main():
    # Create a stream
    s = stream.Stream()

    # Define the chords and their durations
    chords = [
        ('G4 B4 D5', 'whole'),  # G major
        ('E4 G4 B4', 'whole'),  # E minor
        ('C4 E4 G4', 'whole'),  # C major
        ('D4 F#4 A4', 'whole')  # D major
    ]

    # Add the chords to the stream
    for chord_notes, duration in chords:
        s.append(create_chord(chord_notes, duration))

    # Loop the chord progression
    for _ in range(3):
        for chord_notes, duration in chords:
            s.append(create_chord(chord_notes, duration))

    # Write the MIDI file
    mf = midi.translate.streamToMidiFile(s)
    mf.open('strings.mid', 'wb')
    mf.write()
    mf.close()

if __name__ == '__main__':
    main()        

Running this code in your favourite IDE (Visutal Studio Code, for instance) your "strings.mid" track will be generated. Importing it in GarageBand will look like below:

Your strings will sound like that:

Now you can go ahead and generate a bass track, using below prompt:

The outcome is a Python code you can run to generate your bass track:

from music21 import stream, note, midi

def create_bass_note(pitch, duration):
    n = note.Note(pitch)
    n.duration.type = duration
    return n

def create_rest(duration):
    r = note.Rest()
    r.duration.type = duration
    return r

def main():
    # Create a stream for bass
    bass_stream = stream.Stream()

    # Define the bass notes and their durations
    bass_notes_and_rests = [
        ('G2', 'quarter'), ('D2', 'eighth'), ('R', '16th'), ('G2', 'quarter'), ('D2', 'eighth'), ('R', '16th'),  # G major
        ('E2', 'quarter'), ('B2', 'eighth'), ('R', '16th'), ('E2', 'quarter'), ('B2', 'eighth'), ('R', '16th'),  # E minor
        ('C2', 'quarter'), ('G2', 'eighth'), ('R', '16th'), ('C2', 'quarter'), ('G2', 'eighth'), ('R', '16th'),  # C major
        ('D2', 'quarter'), ('A2', 'eighth'), ('R', '16th'), ('D2', 'quarter'), ('A2', 'eighth'), ('R', '16th')   # D major
    ]

    # Add the bass notes and rests to the bass stream
    for pitch_or_rest, duration in bass_notes_and_rests:
        if pitch_or_rest == 'R':
            bass_stream.append(create_rest(duration))
        else:
            bass_stream.append(create_bass_note(pitch_or_rest, duration))

    # Loop the bass notes and rests
    for _ in range(3):
        for pitch_or_rest, duration in bass_notes_and_rests:
            if pitch_or_rest == 'R':
                bass_stream.append(create_rest(duration))
            else:
                bass_stream.append(create_bass_note(pitch_or_rest, duration))

    # Write the bass MIDI file
    bass_mf = midi.translate.streamToMidiFile(bass_stream)
    bass_mf.open('bass.mid', 'wb')
    bass_mf.write()
    bass_mf.close()

if __name__ == '__main__':
    main()        

Running this code in your favourite IDE (Visutal Studio Code, for instance) your "bass.mid" track will be generated. Importing it in GarageBand will look like below:

Your simple bass track will sound like that:

Now, adding a drum loop and playing all together, your hello world genAI music generation loop will be up&running and will sound like below:

To close this hello world, an injection of human guitar can be added, and it will play like below:

Conclusions

In this article, we have explored the exciting potential of using GenAI, particularly Language Learning Models (LLMs), to generate music tracks and enhance the music composition process. We have seen how these advanced AI tools can assist in creating dynamic and engaging music loops, giving musicians new ways to prototype and develop their creative ideas. Also, we have seen how the injection of a really simple human guitar loop introduce a touch of magic.

Looking ahead, there are numerous possibilities for expanding the capabilities of GenAI in the music world. Some potential extensions include:

1. Solo creation: GenAI could be used to generate unique and expressive solos, drawing inspiration from a few examples and implementing few-shot learning approaches. This could lead to a more diverse and personalized music composition experience.
2. Integration with popular music development tools: by incorporating a GenAI music production console into widely used music creation platforms, musicians could access AI-generated suggestions and ideas directly within their workflow, further enhancing their creative capabilities.
3. Real-time GenAI band: Imagine a live performance where a real-time GenAI player listens to what you are playing and contributes additional soundtracks or solos as needed. This innovative concept could revolutionize live exhibitions and create a truly interactive and dynamic musical experience for both performers and audiences.

In conclusion, the power of GenAI in the realm of music composition is immense, and its potential applications are only just beginning to be explored. As we continue to push the boundaries of AI technology, the possibilities for creative expression and innovation in the music industry are virtually limitless. Let us embrace the magic of GenAI and witness the transformation it brings to the world of music!

Important end note (written by an human!): I really think human art must not die with the introduction of these tools; I like to think it will evolve, embracing new ways to generate artworks.

#MusicComposition #ArtificialIntelligence #AIinMusic #MusicTechnology #LanguageLearningModels #MusicCreation #Innovation #MusicIndustry #watsonx