Intelligibility and Representation of Timbre in Primary Auditory Cortex

Transcription

1 Intelligibility and Representation of Timbre in Primary Auditory Cortex Powen Ru Taishih Chi Didier Depireux David Klein Nina Kowalski & Acoustic Research Electrical and Computer Engineering, College Park

2 Early Auditory Processing Stages Reconstruction log f 2 log f log f log u u log f log f Time eardrum cochlea basilar membrane filters hair cell stages lateral inhibitory network Auditory Spectrogram y1(t;s) t y 1 (t;s) g( t y 1 (t;s)) y 2 (t;s) y 3 (t;s) y(t;s) x(t) t h(t;s) / t g(.) t w(t) / s s v(s) max(, ) T y5 cochlear filters hair cell stages lateral inhibitory network

3 Normal Compress Time Down-Shift Time Dilate Time Time

4 Violin (vibrato) Musical Spectrograms Piano Time Time Time

5 Spectral Dynamics Violin (vibrato) Piano Time Time 2 Violin (vibrato) 1 Piano Time

6 Decomposing a Spectrogram into Dynamic Ripples Time Σ t 25 Scale (cyc/octaves) (octaves) Rate (Hz) 12

7 Experimental Methods to Measure the STRF Ripple Transfer Function Method w x = log f Frequency (khz) t ripple velocity (Hz) 212/11a Ω Inverse Transform Fourier Transform STRF 25 Time ripple frequency (cycles/octave) t Reverse-Correlation Method Acoustic Waveform Fast Spectro-Temporal Envelope Slow Spectro-Temporal Envelope Spikes to trigger average f f t t t t

8 Examples of AI STRF Shapes f (khz) f (khz) f (khz) f (khz) f (khz) f (khz)

9 Spectro-Temporal Response Field Decomposition

10 Model of Scale-Rate Decomposition Reconstruction

11 Spectro-Temporal MTFs of the Auditory Model

12 Combined Spectro-Temporal Modulation Transfer Function

13 Evaluating Intelligibility MTF of auditory model Stationary White Noise Reverberant Environment Noise & Reverberation cyc/oct cyc/oct cyc/oct S/N = 3 db rev =.1 sec S/N= -3 db, rev =.5 sec Hz S/N = -3 db rev =.2 sec S/N= -3 db, rev =.1 sec S/N = -9 db db rev =.5 sec Distance with Reverberation Hz S/N= -3 db, rev =.2 sec Distance Distance Distance with White Noise rev(sec) Distance with White Noise (-3 db) Distance Hz Rev (sec)

14 8 Violin (vibrato) Piano _ + _ Scale (cyc/oct) Clarinet Oboe Rate (Hz)

15 Manipulating Temporal and Spectral Modulations Normal Spectrally smeared Time 7 Temporally smeared Time Temporally sharpened Time Time

16 2 1 5 Female Time Oboe Time Female Oboe Time

17 Spectro-temporal modulation transfer functions and speech intelligibility Chi et al., JASA, Nov <