This article will cover how the interactive audio system was implemented for a turn-based RPG Game, in the likes of Baldur's Gate, Divinity: Original Sin, Pillars of Eternity, etc. Unity is being used as the game engine, with FMOD Studio acting as the audio engine.
______________________________________________________________
ENVIRONMENT AUDIO
In order to create the most varied and un-repetitive soundscape possible, each level has a Main Environment Event which contains usually a 2D room tone and 3D elements which are played through scattered sounds. To enhance the player’s immersion, these are also played from different locations on the 3D environment, through automation. Wind gusts are also simulated through EQ automation on windy loops. Other techniques such as automating the Stereo Axis and Width of windy stereo loops are also used to give some movement to the audio environment.
In addition to this Main Environment Event, a Background Music Event also plays with musical pieces played randomly, intercalated with periods of silence, to add some overall tension.
______________________________________________________________
DYNAMIC MUSIC SYSTEM
The Music System varies between an Horizontal approach for some levels, a Vertical Layered approach for other levels and also a mix of both for other levels. The music dynamics are controlled by the ChangeMusicDynamics() method, which is called by certain events of the game. This uses FMOD’s setParameterValue method to attribute a certain parameter value to the BackgroundMusic Event Instance:
public void ChangeMusicDynamics(float value)
{
BackgroundMusic.setParameterValue("dynamics", value); //Sets the received value to the “dynamics” parameter
Debug.Log("Change Music Dynamics to " + value);
}
{
BackgroundMusic.setParameterValue("dynamics", value); //Sets the received value to the “dynamics” parameter
Debug.Log("Change Music Dynamics to " + value);
}
Horizontal + Vertical approach:
For the first dungeon, there are two puzzles that must be solved in order to progress through the level. The Music system changes as these puzzles are completed. This is achieved by the ActivatePuzzle() method, which sends out the correspondent MusicDynamics value to the AudioManager script, where the ChangeMusicDynamics() method is called.
public void ActivatePuzzle()
{
activated = true; //Puzzle is completed
if (AudioManager.thisInstance != null)
{
AudioManager.thisInstance.ChangeMusicDynamics (MusicDynamics); //Calls out the ChangeMusicDynamics() method from the AudioManager script, with the MusicDynamics value
}
}
As we enter the level, different tension-filled ambience loops are randomised with periods of silence between them. As soon as the first door is unlocked, the music event receives the dynamics parameter value of 0.5, which triggers a new music loop that accentuates the gameplay progression.
After a while, the system goes back to play the randomised loops, until a final lever is activated which triggers a more intense composite of layered loops, by setting the dynamics parameter to 1.0
Vertical approach:
On this level, the player must activate a set of levers in a specific order, in order to open the main door to the dungeon main boss.
As the player advances or retracts in the correct order, the dynamics parameter value is gradually increased or decreased. This changes the volume of the various layers. Transitions are used for smooth fades of the loops.
When the puzzle is complete the value 1.0 is sent to the event which plays the final stinger, as the door opens. The event finishes in a trapped loop with silence, to accentuate the tension as the player approaches the main dungeon boss.
Vertical approach:
On this level, the player must activate a set of levers in a specific order, in order to open the main door to the dungeon main boss.
As the player advances or retracts in the correct order, the dynamics parameter value is gradually increased or decreased. This changes the volume of the various layers. Transitions are used for smooth fades of the loops.
When the puzzle is complete the value 1.0 is sent to the event which plays the final stinger, as the door opens. The event finishes in a trapped loop with silence, to accentuate the tension as the player approaches the main dungeon boss.
______________________________________________________________
SNAPSHOT-BASED MIX
The Dynamic Mix of the game is based on several snapshots that are called out during the game. These snapshots change the volume level of the busses, reverb settings and also control ducking effects (such as huge sounds muting smaller less-important ambience events, as to not clutter the mix).
On the following example, we can see how the “Combat” snapshot changes the volume of several busses, when the player’s party enters a combat. The exploration gets muted out and most Environment sounds go lower in the mix. The background combat music slowly raises to set the overall tension. When the player wins the combat, the "win" parameter is set to 1, which gradually transitions from the "Combat" snapshot to the level's Main Environment snapshot.
______________________________________________________________
DISTANCE CROSSFADE EVENTS WITH SUSTAIN POINTS
The following example shows an Event which plays an audio loop of a barrier that must be destroyed, in order for the player to progress through the level.
To enhance the player’s proximity and immersion, a low-end layer only plays back near proximity. This is achieved by the built-in distance parameter.
When the barrier is destroyed, the OnDisable() method is called, which triggers the Sustain Point (Cue Instance) that fades-out the loop.
public FMODAsset A_EventSound;
EventInstance evt;
CueInstance cue;
//Starts the Loop has soon as the barrier is enabled
void OnEnable () {
if (evt != null)
{
evt = FMOD_StudioSystem.instance.GetEvent (A_EventSound);
evt.start ();
}
}
//When the barrier is disabled, it triggers the Sustain Point (FMOD uses the “KeyOff” string as the default name for a sustain point)
void OnDisable () {
if (evt != null)
{
evt.getCue ("KeyOFF", out cue);
if ( cue!=null ) {
cue.trigger ();
}
}
}
//Sets the Event’s 3D attributes for each frame
void Update()
{
if (evt != null && evt.isValid ())
{
var attributes = UnityUtil.to3DAttributes(transform.position);
evt.set3DAttributes(attributes);
}
else
{
evt = null;
}
}
______________________________________________________________
CHARACTER’S HIT EFFORTS
For the character’s hit efforts during combat (when they get hit by an opponent’s attack), the type of effort varies according to the amount of damage received. For this to work, the PlayCharacterSounds() method receives the int value “damage”, which is then divided by 10, to correspond to a value between 0 and 1.
public void PlayCharactersSounds(VoiceType _what, ECharacterRace _charRace, Vector3 _pos, string name = "", int damage = 0)
{
List<FMODAsset> sounds = new List<FMODAsset>();
string path = _charRace.ToString(); //Attributes the Character’s Race to the correspondant path
Debug.Log("Call a " + _what.ToString() + " @ " + path);
sounds.AddRange(getAudioInPath("Characters/" + path)); //Attributes the Fmod Assets inside the Character Race’s folder to the “sounds” List
//Plays the “hit” sound, according to the damage value
if (damage > 0)
{
if (sounds.Exists(a => a.path.Contains(_what.ToString())))
{
EventInstance autch = FMOD_StudioSystem.instance.GetEvent(sounds.Find(a => a.path.Contains(_what.ToString())));
autch.setParameterValueByIndex(0, (float)damage / 10f);
var attributes = FMOD.Studio.UnityUtil.to3DAttributes(_pos);
autch.set3DAttributes(attributes);
autch.start();
}
}
//Plays the “dodge” sound, when the damage is not 0
else if (sounds.Exists(a => a.path.Contains(_what.ToString())))
{
FMOD_StudioSystem.instance.PlayOneShot(sounds.Find(a => a.path.Contains(_what.ToString())), _pos);
}
}
The FMOD Event consists of different multi sounds with different intensity hit efforts. These are spread out through the Damage parameter track, according to their intensity. Pitch and Volume are also automated, to further enhance the hit’s intensity. If the damage value isn’t higher than 0, that means the attacker has missed his/her target, which then plays back the multi-sound on the “dodge” track.
______________________________________________________________
EVENTS TRIGGERED BY COLLIDERS
The mixer has a Main Reverb Bus, where some of the 3D audio is sent to, such as footsteps, spells, dialog, etc.
To change the reverb settings, according to the area where the player is, colliders are placed on smaller areas. These contain an FMOD event that triggers a higher-priority overriding snapshot with the correspondent reverb settings. Since this snapshot is of the overriding type, it overrides the reverb settings that were present on the main level’s snapshot (this main evel snapshot is used for the general reverb settings, usually the “outside” reverb, in case of a level with an outdoors area with smaller rooms spread throughout the level).
//Player enters smaller area
void OnTriggerEnter(Collider other)
{
if (other.GetComponent<PawnSelector> () && other.GetComponent<PawnSelector> ().myPawn.selectedPlayer)
{
TriggerEvent.start ();
Debug.Log ("Trigger Sound Event " + other.name);
}
}
//Player exits smaller area to the main level's area
void OnTriggerExit(Collider other)
{
if (other.GetComponent<PawnSelector> () && other.GetComponent<PawnSelector> ().myPawn.selectedPlayer)
{
TriggerEvent.getCue ("KeyOFF", out cue);
if ( cue!=null ) {
cue.trigger ();
}
Debug.Log ("Stop Sound Event");
}
}
This Event is triggered when the player's party leader steps in a railbridge area with different reverb settings than the shorter mining caves around the rest of the level. The snapshot is gradually brought in, where it is locked on a sustain point, which is triggered when the player goes back inside the mining caves (main level), making the snapshot go away smoothly.