History of Audio in Unity

Transcription

1 Audio Showcase

2 About Me (Wayne) Core / Audio team at Unity Work in the Copenhagen office Originally worked for FMOD (Audio Engine) Work on Audio and other new tech in Unity Interested in most phases of data processing

3 History of Audio in Unity Limited development of in-editor Audio features. Aspects of the Audio system are powerful, yet obscure. Most audio features require scripting. Advanced audio solutions required in house development or use Asset Store solutions. Some functionality impossible, such as audio sub-mixing. AudioSources always mixed together at the AudioListener

4 A new direction Renewed push of Audio in Unity. Re-write of underlying audio asset pipeline. Re-write of audio channel framework, with big optimizations. A push for more UI centric tools that empower Sound Designers. Address gaps in audio functionality within Unity.

5 The 5.0 Release Audio System re-write The Audio Mixer

6 The Asset Pipeline Re-Write Audio Import as a separate process Unity forks a specially developed tool to import the asset. Reduces memory footprint of Unity. Arbitrary sized audio clips can be imported. Eliminates the chances of corrupted audio files crashing Unity on import. Unity loads and processes sanitized data only. Optimized codec support Removed MP3 compression target (mobile) New optimized Vorbis Tremor decoder. Additional ADPCM compression option.

7 The Asset Pipeline Re-Write PCM No compression No decode costs Suitable for short sounds (footsteps or gunshots) ADPCM Constant compression ratio (3.5:1) Cheap decode costs Not suitable for all types of sounds. (Noisy, low frequency sounds are best)

8 The Asset Pipeline Re-Write Compressed (Vorbis) Maximum compression ratio Heaviest decode costs Suitable to most sound types Benefits of new compression type More performant than MP3 / straight OGG Vorbis etc Low memory footprint No looping artifacts (loops seamlessly)

9 Example Sweep tone

10 OGG Vorbis File (OGG Size 74166) Vorbis Compression (FSB Size 75392) MP3 Compression (FSB Size ) Compression Stats Sound Memory Stream Compressed in Memory Decompress Stream Compressed in Memory Decompress Stream Compressed in Memory Decompress OGG Vorbis File (OGG Size 74166) Vorbis Compression (FSB Size 75392) MP3 Compression (FSB Size ) Compressed in Compressed in Compressed in Decompress Stream Decompress Stream Decompress Stream Memory Memory Memory Sound Memory

11 AudioClip Improvements New Load Type: DynamicLoading Audio data loading is delayed, not loaded when the scene loads Data can be loaded in any way (Load into memory, stream, etc) AudioClips become lightweight structures and loading strategies can be employed by developers Loading can be blocking or non-blocking

12 AudioClip Improvements Better channel management Playback resources are now freed immediately on completion of a sound. Improved streaming support Including proper streaming from AssetBundles Correct streaming inside editor, replicates behaviour of built game Better handling of large audio files AudioClip information can be queried without data loaded Improved audio profiling information Multi-edit AudioClip assets!!!

13 Audio Mixer

14 What is an Audio Mixer An Audio Mixer allows complex mixing, routing and mastering of your entire games soundscape Add DSP effects anywhere in the audio signal Group categories of sounds and apply DSP, attenuation and other effects in the editor Master the games mix during Playmode A new asset type you can create in Unity Extensive editor authoring tools (modeled on existing DAW tools) Create and chain together multiple Audio Mixers Example!

15 Audio Mixer Features: Routing Create complex routing hierarchies to categorize sounds and apply effects and volume adjustments as one. The group hierarchy is mixed together until it hits the Master Group. The Master Group is then routed into the AudioListener or another AudioGroup of another AudioMixer

16 Audio Mixer Features: Routing

17 Audio Mixer Features: Routing AudioSources can be routed to any group of any AudioMixer in the Project 3D attenuation and other properties are applied before the signal enters the AudioMixer from the AudioSource Example!

18 Audio Mixer Features: Effects Add DSP Effects anywhere within the Audio Mixer Hierarchy Modify effect parameters in the inspector Allow wet mixing to tune the amount an affect should be audible. Stack effects in each AudioGroup A selection of built in effects to choose from

19 Audio Group

20 Audio Mixer Features: Native Plugin Effects Developers can implement custom native DSP effects that will be accessible within the effect menu Define parameters to the native effects that can be tweaked within the inspector Define custom UI for effect plugins that will be displayed in the inspector for that plugin Developers or third parties can create complex custom DSP processing effects Release on Asset Store Example!

21 Audio Mixer Features: Snapshots Snapshots capture the state of all parameters within an Audio Mixer Effect Parameters (including plugin effects) Volume & Pitch settings Ducking settings These states can be transitioned between, during gameplay Allows Audio Mixers to change the theme and mood of the entire soundscape with ease Entire sections of the soundscape can be brought in and out of the mix, or individual effects removed or added Example!

22 Audio Mixer Features: Exposed Parameters Expose individual effect parameters to script via names Modify these parameters directly via game logic Fine grain tweaking of the mix

23 Audio Mixer Features: Sends & Returns Send portions of the signal from anywhere within an Audio Group to anywhere else in the Audio Mixer Example: Split the signal and apply a series of effects and mix the two signals together again Very powerful tool to combine with Snapshots to create complex mood transitions Example!

24 Audio Mixer Features: Ducking Lower the volume of one part of the mix based on the signal in another part Example: Compress loud gunshot sounds when an NPC talks so that he/she is always audible Also known as Side Chain Compression Unity provides this as part of any Audio Group Example!!

25 Thank You! Questions