Search Papers | Poster Sessions | All Posters
Poster A85 in Poster Session A - Tuesday, August 6, 2024, 4:15 – 6:15 pm, Johnson Ice Rink
Reconstructing sound from auditory spiking trains
Po-Ting Liu1; 1Academia Sinica
Auditory periphery encodes sound stimuli into spike trains and interacts with the higher auditory system; however, decoding activities in the auditory periphery remains under-explored in computational neuroscience. In this study, we aim to reconstruct the acoustic stimuli from the spike trains they elicit. To this end, the decoding models must handle the stochastic responses of auditory nerve fibers (ANFs) and compensate for the adaptations by the highly non-linear and bidirectional interactions in the pathways. We proposed a deep artificial neural network (DANN)-based speech synthesis models to decode the spike trains of ANFs’ responses. Our model achieved averaged PESQ and SSIM scores of 4.0969 and 0.9225, respectively. Furthermore, in the generalization test, our model performs well on unseen datasets, including VCTK, MCDC8, and excerpts of single musical instruments. In conclusion, our model reconstructs speech with high fidelity from neuronal spiking activities in human peripheral auditory pathways, and the model effectively compensates for any nonlinear and dynamical Acoustic Reflex (AR) and Medial OlivoCochlear Reflex (MOCR) effects.
Keywords: Auditory Reconstruction Decoding Spiking activity