A Harmonious Confluence: Jordan Rudess and the Future of AI in Music

A Concert Like No Other

In September, a diverse crowd gathered at the MIT Media Lab for a concert that would defy conventional musical experiences. The event featured award-winning keyboardist Jordan Rudess, violinist and vocalist Camilla B?ckman, and an innovative artificial intelligence model called jam_bot. Developed over several months in collaboration with an MIT team, this AI made its public debut, introducing a new dimension to live music.

Exploring New Musical Dialogue?

Throughout the performance, Rudess and B?ckman exchanged signals typical of seasoned musicians. However, interactions with the jam_bot revealed a different kind of musical dialogue. During a duet inspired by Bach, Rudess alternated with the AI, each contributing to the baroque style. The audience witnessed Rudess’ expressions of bemusement and concentration, reflecting the novelty of such collaboration. He admitted, “That is a combination of a lot of fun and challenging.”

The Visionary keyboardist

?Jordan Rudess, renowned as one of the greatest keyboardists by Music Radar, is celebrated for his work with Dream Theater and his solo projects. His latest album, "Permission to Fly," showcases his passion for innovation. As a visiting artist with MIT’s Center for Art, Science, and Technology, Rudess embarked on a new journey to blend music and technology, supported by the MIT Media Lab’s Responsive Environments group.

The Power of Collaboration

The project was led by Professor Joseph Paradiso, with vital contributions from graduate students Lancelot Blanchard and Perry Naseck. Blanchard, a classical pianist, developed the AI's musical capabilities, while Naseck focused on creating interactive visualizations. Together, they aimed to achieve “symbiotic virtuosity,” allowing humans and AI to perform in real time.

Visualizing AI’s Musical Contributions

A kinetic sculpture, designed by Naseck, added a visual element to the concert, responding dynamically to the AI’s music. This installation, enhanced by machine learning, transformed the stage with its movements, distinguishing the AI’s contributions and intensifying the performance’s emotional impact. The AI culminated the concert with an independent exploration, captivating the audience.

Pioneering Musical Futures?

Blanchard used a music transformer model to enable the AI to predict and generate music in real time, akin to language models predicting text. This adaptability allowed Rudess to engage in a dynamic musical conversation with the AI, highlighting the potential of such technology in live performances.

Expanding the Horizons of AI in Music

Rudess and his collaborators envision vast potential for this technology, beyond performance. They foresee applications in music education, where AI assists in teaching, and anticipate the development of AI tools that emulate the styles of favorite musicians. Despite some resistance from traditionalists, Rudess remains optimistic about AI’s role in enhancing the music industry.

A New Era of Musical Exploration

This collaboration at MIT signifies a groundbreaking integration of AI into creative processes, setting the stage for future innovations. As Professor Paradiso observes, "Jordan is ahead of the pack. Once it’s established with him, people will follow." The partnership between Rudess and MIT is a testament to the transformative power of AI in music, heralding a new era where technology and human creativity harmoniously coexist.

Jamming with MIT: A Journey of Discovery

Rudess’ journey with MIT began with his interest in the Knitted Keyboard, leading to fruitful engagements with students and faculty. This collaboration has been mutually enriching, as Rudess shared his expertise and learned from MIT’s pioneering work in music technology. “I feel the sense that, wow, all of my musical ideas and inspiration and interests have come together in this cool way,” he reflects. Through this innovative venture, Rudess and the MIT team have not only expanded the frontiers of music but have also laid the groundwork for future explorations where AI and musicians create in tandem, exploring uncharted musical landscapes.