What is Adaptive Music?

We may not be aware of it, but most music that we’re used to listen to is linear. Linear music has a beginning, a development and an ending that sound exactly the same every time we listen to it. The soundtrack of a movie, the songs of Bob Dylan and a Mozart’s symphony are all examples of linear music. Linear music works great if it’s used for concert music or as musical background for fixed media. Every time we watch Star Wars, for instance, the sequence of events occurring on screen are always the same. No matter how much we would like it to be different, (40-year-old spoiler alert) Obi-Wan Kenobi is going to be killed by Darth Vader! The fixed structure of a movie is great for the composer who has to write the soundtrack, because he or she can create musical cues that are specifically tailored to the on-screen images on a moment-by-moment basis. When the cannons of the Millennium Falcon hit the ships of the Empire, for example, the explosions can be underlined by the music and the overall excitement of the moment can be captured and enhanced by the soundtrack.

In the case of interactive media like video games the story is quite different though. In a game, the player has a direct influence on the sequence of events that occur. This is a challenging situation for the composer, whose role – just like in movies – is to create a musical environment that may enhance the player’s experience. The issue is that the composer has no previous knowledge about what’s going to happen next in a game, in that every playthrough may bedifferent. A player may decide to engage in a fight with an enemy that is part of a level, whereas another may just ignore it and move to the next scene. Same game, same level, two different sequences of events.

How does a composer create music that can fit a game even if every single player has a unique game experience? That’s where adaptive music comes in. An adaptive soundtrack constantly changes dynamically by reacting to some type of control input coming from the game. Be aware that there isn’t a uniform naming for adaptive music. Depending on the source, you can also find adaptive music named as interactive, nonlinear or dynamic music. We decided to use the term adaptive as it alludes back to evolution. In nature, organisms adapt and evolve according to several environmental factors.

With an adaptive score, a piece may sound different every time it is played backbased on changes to some control inputs. Players make decisions while playing a game which may have a direct effect on the plot. These decisions, in turn, may influence the shape of the music and how it evolves over time. The aim of these changes is to tailor the music to the unique experience of the player and to enhance his or her emotional experience. Several game parameters can be used to affect the music, such as the time of the day, the health state of the character, the number of enemies nearby, the location and the presence of specific characters.

Changes to the game parameters can be mapped onto different musical aspects.For example, in Uncharted 2: Among Thieves when the character controlled by the player encounters enemies, the music changes from an explorative cue to a suspenseful one. If the player starts fighting with the enemies, the music switches to a battle cue. The battle cue can reflect the intensity of the combat situation. When the battle is finished, the music goes back to the explorative cue. If the character perishes during the battle, a death cue is triggered instead. This type of adaptive technique is called horizontal resequencing, which consists of creating a sort of dynamic playlist in real time based on the game states.

Other musical changes can involve tempo manipulation, musical modulations, and adding or removing instruments from a cue. Tempo may be sped up to provide an audio cue to the player. For example, in Super Mario Bros tempo is increased when the player is playing well and he or she has collected enough power-ups. By adding and removing instruments it is possible to increase or lower the tension of a musical cue. Imagine a battle situation. If there are less than three enemies, the strings may be the only section playing. When the number of enemies is between 5 and 10, the wind instruments may kick in. When the boss finally shows up, percussion may be introduced into the ensemble to bring the music to maximum tension. This technique, that is often used to create adaptive soundtracks, is called vertical remixing.

There are endless combinations in the musical parameters and control inputs a composer may use to create successful adaptive scores. Regardless of the techniques used, the final aim of an adaptive soundtrack is always the same: to provide the player with a customised musical experience that may enhance their game experience. There is still a lot of work to do for composers to perfect the techniques employed in adaptive music. At Melodrive we believe that the ultimate solution for adaptive music is real time machine-composed music. This can guarantee continuously original music that dynamically responds to in-game events at a very granular level. However, we should say that there has been a lot of progress since the music that accompanied the arcade games of the 1980s, where the soundtracks consisted of very simple/short musical loops played back ad nauseam on quite rudimentary audio supports.

What is Adaptive Music? https://melodrive.com/blog/what-is-adaptive-music/