Speech distortion tracing : A noise classification approach for robust speech processing

Woongjae Lee; Souhwan Jung

doi:10.7776/ASK.2025.44.5.533

All Issue

2025 Vol.44, Issue 5 Preview Page Next Page

Research Article

Speech distortion tracing : A noise classification approach for robust speech processing 음성 변형 추적 : 강건한 음성 처리를 위한 노이즈 분류 접근법

30 September 2025. pp. 533-539

PDF XML

Abstract

Understanding and classifying various types of noise is crucial for improving the robustness of speech processing models. While previous studies have primarily focused on environmental noise such as white noise and background noise, the effects of speech manipulation including pitch shift and time stretch have been relatively less explored. This study proposes a noise classification model that can effectively distinguish various types of noise including speech manipulation to overcome these limitations. We constructed a noise dataset of 538,000 samples using LibriSpeech, Voice Cloning ToolKit (VCTK), DSD-Corpus, and TIMIT datasets, and designed a Multi-Feature Fusion model that integrates spectrogram, MFCC, and F0 features. The proposed model achieved 95 % accuracy with 7.2 M parameters, demonstrating suitable lightweight characteristics for real-time preprocessing. This suggests that noise classification-based adaptive preprocessing can effectively contribute to enhancing the robustness of speech processing models in various noise environments.

Keywords

Noise classification

Speech manipulation

Speech processing

Audio deepfake detection

Automatic speaker verification

다양한 노이즈 유형을 이해하고 분류하는 것은 음성 처리 모델의 강건성을 향상시키는 데 있어 매우 중요하다. 기존 연구들은 주로 백색 잡음이나 배경 소음과 같은 환경 노이즈에 초점을 맞추었으나, 피치 시프트, 타임 스트레치와 같은 음성 변조의 영향은 상대적으로 덜 탐구되어 왔다. 본 연구는 이러한 한계를 극복하기 위해 음성 변형을 포함한 다양한 노이즈 유형을 효과적으로 구분할 수 있는 노이즈 분류 모델을 제안한다. LibriSpeech, Voice Cloning ToolKit (VCTK), DSD-Corpus, TIMIT 데이터셋을 활용하여 538,000개의 노이즈 데이터셋을 구축하였으며, Spectrogram, MFCC, F0를 융합한 Multi-Feature Fusion 모델을 설계하였다. 제안된 모델은 95 %의 정확도를 달성하였으며, 7.2 M 파라미터로 실시간 전처리에 적합한 경량성을 확보하였다. 이는 노이즈 분류 기반 적응형 전처리가 다양한 노이즈 환경에서의 음성 처리 모델 강건성 향상에 기여할 수 있음을 시사한다.

키워드

노이즈 분류

음성 변조

음성 처리

오디오 딥페이크 탐지

자동 화자 검증

References

S. Katkov, A. Liotta, and A. Vietti, “Evaluating the robustness of ASR systems in adverse acoustic conditions,” Proc. 5th IDSTA, 76-80 (2024).

10.1109/IDSTA62194.2024.10746999

V. K. Singh, K. Sharma, and S. N. Sur, “A survey on preprocessing and classification techniques for acoustic scene,” Expert Syst. Appl. 229, 120520 (2023).

10.1016/j.eswa.2023.120520

T. Ko, V. Peddinti, D. Povey, and S. Khudanpur, “Audio augmentation for speech recognition,” Proc. Interspeech, 3586-3590 (2015).

10.21437/Interspeech.2015-711

L. Wijayasingha and J. A. Stankovic, “Robustness to noise for speech emotion classification using CNNs and attention mechanisms,” Smart Health, 19, 100165 (2021).

10.1016/j.smhl.2020.100165

S. E. Bou-Ghazale and K. Assaleh, “A robust endpoint detection of speech for noisy environments with application to automatic speech recognition,” Proc. IEEE ICASSP, IV-3808 (2002).

10.1109/ICASSP.2002.5745486

M. N. Ali, A. Brutti, and D. Falavigna, “Enhancing embeddings for speech classification in noisy conditions,” Proc. Interspeech, 2933-2937 (2022).

10.21437/Interspeech.2022-10707

V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “LibriSpeech: an ASR corpus based on public domain audio books,” Proc. IEEE ICASSP, 5206- 5210 (2015).

10.1109/ICASSP.2015.7178964

C. Veaux, J. Yamagishi, and K. MacDonald, “CSTR VCTK corpus: English multi-speaker corpus for CSTR voice cloning toolkit,” The Centre for Speech Technology Research (CSTR), University of Edinburgh, Tech. Rep., 2017.

T.-P. Doan, H. Dinh-Xuan, T. Ryu, I. Kim, W. Lee, K. Hong, and S. Jung, “Trident of poseidon: A generalized approach for detecting deepfake voices,” Proc. CCS, 2222-2235 (2024).

10.1145/3658644.3690311

J. S. Garofolo, L. F. Lamel, W. M. Fisher, D. S. Pallett, N. L. Dahlgren, V. Zue, and J. G. Fiscus, “TIMIT acoustic-phonetic continuous speech corpus,” National Institute of Standards and Technology (NIST), Tech. Rep., 1993.

D. Snyder, G. Chen, and D. Povey, “MUSAN: A music, speech, and noise corpus,” arXiv preprint arXiv:1510.08484 (2015).

K. J. Piczak, “ESC: Dataset for environmental sound classification,” Proc. 23rd ACM Int. Conf. Multimedia, 1015-1018 (2015).

10.1145/2733373.2806390

Simple Auto-Tune in Python, https://github.com/JanWilczek/python-auto-tune, (Last viewed October 10, 2024).

T. Ko, V. Peddinti, D. Povey, M. L. Seltzer, and S. Khudanpur, “A study on data augmentation of reverberant speech for robust speech recognition,” Proc. IEEE ICASSP, 5220-5224 (2017).

10.1109/ICASSP.2017.7953152

J. B. Awotunde, R. O. Ogundokun, F. E. Ayo, and O. E. Matiluko, “Speech segregation in background noise based on deep learning,” IEEE Access 8, 169568- 169575 (2020).

10.1109/ACCESS.2020.3024077

S. Rosenzweig, S. Schwär, J. Driedger, and M. Müller, “Adaptive pitch-shifting with applications to intonation adjustment in a cappella recordings,” Proc. 24th DAFx, 121-128 (2021).

10.23919/DAFx51585.2021.9768268

X. Wang, J. Yamagishi, M. Todisco, H. Delgado, A. Nautsch, N. Evans, and Z. H. Ling, “ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech,” Comput. Speech Lang. 64, 101114 (2020).

10.1016/j.csl.2020.101114

B. Desplanques, J. Thienpondt, and K. D. Ecapa- tdnn, “Emphasized channel attention, propagation and aggregation in tdnn based speaker verification,” arXiv: 2005.07143 (2020).

10.21437/Interspeech.2020-2650

H. J. Heo, U. H. Shin, R. Lee, Y. Cheon, and H. M. Park, “NeXt-TDNN: Modernizing multi-scale temporal convolution backbone for speaker verification,” Proc. IEEE ICASSP, 11186-11190 (2024).

10.1109/ICASSP48485.2024.10447037

J. Thienpondt and K. Demuynck, “Ecapa2: A hybrid neural network architecture and training strategy for robust speaker embeddings,” Proc. IEEE ASRU, 1-8 (2023).

10.1109/ASRU57964.2023.10389750

J. W. Jung, H. S. Heo, H. Tak, H. J. Shim, J. S. Chung, B. J. Lee, and N. Evans, “Aasist: Audio anti- spoofing using integrated spectro-temporal graph attention networks,” Proc. IEEE ICASSP, 6367-6371 (2022).

10.1109/ICASSP43922.2022.9747766

H. Tak, M. Todisco, X. Wang, J. W. Jung, J. Yamagishi, and N. Evans, “Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation,” arXiv:2202. 12233 (2022).

10.21437/Odyssey.2022-16

D. T. Truong, R. Tao, T. Nguyen, H. T. Luong, K. A. Lee, and E. S. Chng, “Temporal-channel modeling in multi-head self-attention for synthetic speech detection,” arXiv:2406.17376 (2024).

10.21437/Interspeech.2024-659

A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Proc. NeurIPS, 12449-12460 (2020).

Y. Gong, C. I. Lai, Y. A. Chung, and J. Glass, “Ssast: Self-supervised audio spectrogram transformer,” Proc. AAAI Conf. Artif. Intell. 10699-10709 (2022).

10.1609/aaai.v36i10.21315

R. Akter, M. R. Islam, S. K. Debnath, P. K. Sarker, and M. K. Uddin, “A hybrid CNN-LSTM model for environmental sound classification: Leveraging feature engineering and transfer learning,” Digit. Signal Process. 163, 105234 (2025).

10.1016/j.dsp.2025.105234

Information

Publisher :The Acoustical Society of Korea
Publisher(Ko) :한국음향학회
Journal Title :The Journal of the Acoustical Society of Korea
Journal Title(Ko) :한국음향학회지
Volume : 44
No :5
Pages :533-539
Received Date : 2025-08-01
Revised Date : 2025-08-28
Accepted Date : 2025-09-01
DOI :https://doi.org/10.7776/ASK.2025.44.5.533

The Journal of the Acoustical Society of KoreaISSN:1225-4428(Print) 2287-3775(Online)한국음향학회

All Issue