Real time single channel speech enhancement for quadruped robots under real world ego noise conditions

Hyun Jae Shin; Juhyun Pyo; Hyeongseok Song; Chang-Min Ahn; Murim Kim; Seon Man Kim

doi:10.7776/ASK.2026.45.5.572

All Issue

2026 Vol.45, Issue 5 Preview Page Next Page

Real time single channel speech enhancement for quadruped robots under real world ego noise conditions 4족보행 로봇의 실측 자가소음 조건에서의 단일채널 실시간 음성 향상

30 September 2026. pp. 572-581

PDF XML

Abstract

This study evaluates a single channel real time speech enhancement method for auditory situational awareness of quadruped robots. An ego noise database of 170 samples was constructed from a Unitree Go2 robot under 10 site and ground conditions, 6 motion conditions, and 4 microphone positions. Time frequency analysis showed that the ego noise is neither a pure tone nor white noise, but a combination of multiple narrowband components originating from the internal drivetrain and impulsive components arising from foot ground contact, whose proportions vary with the motion condition. A DeepFilterNet3 based enhancement model was then trained with data including the measured ego noise together with general environmental noise, with an algorithmic latency of 40 ms. Under measured ego noise conditions, the model trained with ego noise improved the short time objective intelligibility from 0.87 to 0.90 and the perceptual evaluation of speech quality from 2.31 to 2.77 over a baseline trained only on environmental noise, with consistent gains in both English and Korean conditions.

Keywords

Quadruped robot

Ego noise

Single channel speech enhancement

Real time processing

Robustness

본 연구는 4족보행 로봇의 청각 기반 상황 인지를 위해 자가소음 조건에서 단일채널 실시간 음성 향상 방법의 성능을 검증한다. 이를 위해 Unitree Go2 로봇을 대상으로 10종 수집 장소·지면 조건, 6종 동작 조건, 4개 수음 위치에서 170개 자가소음 데이터베이스를 구축하였다. 시간에 따른 주파수 분석 결과, 4족보행 로봇 자가소음은 단순 순음이나 백색소음이 아니라, 내부 구동계에 기인하는 다수의 협대역 성분과 발과 지면 접촉에 기인하는 충격 성분이 동작 조건에 따라 서로 다른 비율로 결합된 구조를 보였다. 이러한 분석에 기반하여 실측 자가소음과 일반 환경소음을 포함한 학습 데이터를 구성하고, DeepFilterNet3 기반 단일채널 음성 향상 모델을 학습하였으며, 이때 모델의 알고리즘 지연시간은 40 ms이다. 정량 평가 결과, 실측 자가소음 조건에서 자가소음 학습 모델은 자가소음을 학습하지 않고 일반 환경소음만으로 학습한 기준 모델 대비 음성 명료도 평가를 0.87에서 0.90으로, 지각 음질 평가를 2.31에서 2.77로 향상시켰으며, 영어와 한국어 조건 모두에서 일관된 개선을 확인하였다.

키워드

4족보행 로봇

자가소음

단일채널 음성 향상

실시간 처리

강건성

References

P. Biswal and P. K. Mohanty, “Development of quadruped walking robots: A review,” Ain Shams Eng. J. 12, 2017-2031 (2021).

10.1016/j.asej.2020.11.005

Z. Cao, B. Nie, Y. Zhang, and Y. Gao, “Minimizing acoustic noise: Enhancing quiet locomotion for quadruped robots in indoor applications,” Proc. IROS, 17972-17979 (2025).

10.1109/IROS60139.2025.11246563

G. Ince, K. Nakadai, T. Rodemann, H. Tsujino, and J.-I. Imura, “Ego noise cancellation of a robot using missing feature masks,” Appl. Intell. 34, 360-371 (2011).

10.1007/s10489-011-0285-0

K. Furukawa, K. Okutani, K. Nagira, T. Otsuka, K. Itoyama, K. Nakadai, and H. G. Okuno, “Noise correlation matrix estimation for improving sound source localization by multirotor UAV,” Proc. IROS, 3943-3948 (2013).

10.1109/IROS.2013.6696920

A. Briegleb, A. Schmidt, and W. Kellermann, “Deep clustering for single-channel ego-noise suppression,” Proc. ICA, 2813-2820 (2019).

D. Mukhutdinov, A. Alex, A. Cavallaro, and L. Wang, “Deep learning models for single-channel speech enhancement on drones,” IEEE Access, 11, 22993-23007 (2023).

10.1109/ACCESS.2023.3253719

H. Schröter, A. N. Escalante-B., T. Rosenkranz, and A. Maier, “DeepFilterNet: A low complexity speech enhancement framework for full-band audio based on deep filtering,” Proc. ICASSP, 7407-7411 (2022).

10.1109/ICASSP43922.2022.9747055

H. Schröter, A. N. Escalante-B., T. Rosenkranz, and A. Maier, “DeepFilterNet2: Towards real-time speech enhancement on embedded devices for full-band audio,” Proc. IWAENC, 1-5 (2022).

10.1109/IWAENC53105.2022.9914782

H. Schröter, A. N. Escalante-B., T. Rosenkranz, and A. Maier, “DeepFilterNet: Perceptually motivated real-time speech enhancement,” Proc. Interspeech, 2008-2009 (2023).

C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “An algorithm for intelligibility prediction of time- frequency weighted noisy speech,” IEEE Trans. Audio Speech Lang. Process. 19, 2125-2136 (2011).

10.1109/TASL.2011.2114881

ITU-T Rec. P.862, Perceptual Evaluation of Speech Quality (PESQ): An Objective Method for End-to-end Speech Quality Assessment of Narrow-band Telephone Networks and Speech Codecs, 2001.

J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “SDR – half-baked or well done?,” Proc. ICASSP, 626-630 (2019).

10.1109/ICASSP.2019.8683855

Acoustic Camera - SoundCam 2.0, https://www.cae-systems.de/en/products/acoustic-camera-sound-source-localization/soundcam-20.html, (Last viewed August 5, 2026).

Recording Mode,https://izyrec.zendesk.com/hc/en-us/articles/7443876632335-Recording-Mode, (Last viewed August 5, 2026).

Galaxy S24 Ultra Specifications, https://www.samsung.com/sec/smartphones/galaxy-s24-ultra/specs/, (Last viewed August 5, 2026).

Conversational Speech Dataset for Emotion Classification, https://aihub.or.kr/aihubdata/data/view.do?dataSetSn=263, (Last viewed July 9, 2026).

J.-U. Bang, S. Yun, S.-H. Kim, M.-Y. Choi, M.-K. Lee, Y.-J. Kim, D.-H. Kim, J. Park, Y.-J. Lee, and S.-H. Kim, “KsponSpeech: Korean spontaneous speech corpus for automatic speech recognition,” Appl. Sci. 10, 6936 (2020).

10.3390/app10196936

Noisy Speech Database for Training Speech Enhancement Algorithms and TTS Models, https://doi.org/10.7488/ds/2117, (Last viewed August 5, 2026).

10.7488/ds/2117

C. K. A. Reddy, V. Gopal, R. Cutler, E. Beyrami, R. Cheng, H. Dubey, S. Matusevych, R. Aichner, A. Aazami, S. Braun, P. Rana, S. Srinivasan, and J. Gehrke, “The INTERSPEECH 2020 deep noise suppression challenge: Datasets, subjective testing framework, and challenge results,” Proc. Interspeech, 2492-2496 (2020).

10.21437/Interspeech.2020-3038

R. Chappel, B. Schwerin, and K. K. Paliwal, “Phase distortion resulting in a just noticeable difference in the perceived quality of speech,” Speech Commun. 81, 138-147 (2016).

10.1016/j.specom.2016.04.005

Information

Publisher :The Acoustical Society of Korea
Publisher(Ko) :한국음향학회
Journal Title :The Journal of the Acoustical Society of Korea
Journal Title(Ko) :한국음향학회지
Volume : 45
No :5
Pages :572-581
Received Date : 2026-07-14
Revised Date : 2026-08-11
Accepted Date : 2026-08-24
DOI :https://doi.org/10.7776/ASK.2026.45.5.572

The Journal of the Acoustical Society of KoreaISSN:1225-4428(Print) 2287-3775(Online)한국음향학회

All Issue