Skip to content

Adaptive Audio

October 26, 2010

“Adaptive audio is the latest concept to see industry-wide support and increased usage in games. In an effort to better follow the unpredictable and ever changing on-screen action, music and sound which can “adapt” to what the player is experiencing is being integrated into gaming and quickly establishing itself as a new standard.” – Aaron Marks, the Complete Guide to Game Audio.

Adaptive audio is one of the reasons composing for video games is so appealing for me.  While I mean no slight to film composers or what they do (having scored a small scene for a film myself, I know just how hard it is and it definitely requires it’s own set of specialized skills) I find scoring to a linear narrative makes the music much more subservient to the film than what music is to video games.

Essentially, film music compliments the story the film maker is telling, while adaptive music attempts to ‘score’ the unpredictable and unique decisions players make in-game. In a way, creating an experience that is unique and personal to them.

Adaptive audio and music is pretty unique to video games and can have a huge bearing on how the composer approaches the score. But the use of adaptive audio, when done right, can create a compelling game play experience that is hard to match using conventional music implementation methods.

There are many examples of adaptive audio being used in games today. I’ll look at Oblivion, scored by Jeremy Soule, as our example. The context of the game is a massive, free roaming fantasy game, set in a medieval style era.

Digging around in the game’s audio directory reveals twenty eight musical files. Seven are labeled ‘atmosphere’, eight under ‘battle’, five ‘dungeon’, five tracks labeled ‘town’, there is also a ‘success’ and ‘death’ theme along with the title music.

Here are some of these tracks and what they are labeled with in the game directory:

-The Glory of Cyrodiil – which is labeled as ‘atmosphere’ in the game folder.

-Here is one of the pieces that is labeled as battle music:

The labels are pretty self explanatory when it comes to their use in-game. Walking around in the wilderness will give us a random ‘atmosphere’ track to be played. But if, while walking around in said wilderness, we were to be attacked by a bear, tranquil ‘atmosphere’ is replaced by an uptempo ‘battle’ track. The track will stop playing when the battle is over to be  gradually replaced by  some ‘atmosphere’ music once again.  Then, if we were to enter a town, one of the ‘town’ themes will begin playing and vice-versa for a dungeon. Obviously the ‘death’ theme plays when your character dies, and you get a ‘success’ fanfare if you accomplish some particularly heroic deed.

This is a pretty simplistic approach of explaining how adaptive audio works. But the possibilities of it are both exciting and challenging for a composer. Not least is the challenge of creating a soundtrack where all the pieces work when changing from one to another, in potentially any combination.

Aaron Marks gives some advice for writing adaptive music:

“Obviously the first key to interactive music is to use the same sound bank and same instrumentation. This provides an inherent similarity working in our favor. Another recommendation is to base all of the music around the same key. This provides a solid foundation to make key changes within the same scale and enables the music to return to a familiar root. These changes will have greater impact, while blending perfectly with the rest of the sound track.”

Of course, all of this has quite a large impact contextually, on how the composer approaches writing a score. Knowing that the score you are going to be writing is going to be adaptive, will force the composer to think about some of the things Aaron Marks suggests, such as writing all the music in one key and keeping to the same instrumentation. Something the composer may not of done if he wasn’t to be scoring an adaptive audio  game.

I also found some more advice on how to approach writing adaptive audio from Game Career Guide:

  1. Music comes first. Remember that no matter how closely your music follows the game play and how interactive it is, if it doesn’t gel as a musical composition, you’re better off writing a linear score. Always explore all possibilities of transitions from one game state to the next, and see if the music reacts the way you meant it to react. Make sure that you write transition sequences and that the engine is intelligent enough not to change game states midmeasure or midphrase.
  2. Decouple segments horizontally and vertically. Compose your music so that different segments may be combined end-to-end (horizontally), as well as on top of each other (vertically). This way, you can combine different melody lines with bass lines, use different ornamentation, and so on.
  3. Don’t give away too much information. Sometimes a musical cue might say too much, when it was meant just to highlight the game state change. For example, in a certain game, an upward chord progression always signifies to a player that a starship is on his tail. When working on game state changes, make sure your event-driven music isn’t used as an early warning system for the game.
  4. Define a series of translation tables to track game state changes. For example, in Multiplayer Battletech, a game state change from “winning” to “advantage” implies a losing trend. The music reacts to this change by selecting a different set of segments than it would if the change occurred from “advantage” to “winning.” By composing in a nonlinear fashion, and by having the music react to the player’s actions directly and indirectly, we introduce a new level of interactivity. Emotionally, the soundtrack carries the person seamlessly along with the action in much the same way as the static, linear media of film. In this fashion, music becomes the gateway to the player’s emotional response to the game.

While this is all sound advice, I think there is plenty of room for more methods of creating adaptive music and it is only because adaptive audio is a fairly young development in the world of video games that we may of not seen some other methods in approaching creating an adaptive score.

While there was no real way for me to get practical experience writing adaptive music on my CEP (adaptive audio, by it’s very nature requires a specific kind of game and audio engine to run on) I think looking at how it has been used in games has open my eayes to the possibilities of it in creating a unique musical and gameplay experience for the player. And this is something I would definitely like to explore in the future with my work.


From → Uncategorized

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: