Improving transformer-based speech recognition performance using data augmentation by local frame rate changes

Seong Su Lim; Byung Ok Kang; Oh-Wook Kwon

doi:10.7776/ASK.2022.41.2.122

All Issue

2022 Vol.41, Issue 2 Preview Page Next Page

Research Article

Improving transformer-based speech recognition performance using data augmentation by local frame rate changes 로컬 프레임 속도 변경에 의한 데이터 증강을 이용한 트랜스포머 기반 음성 인식 성능 향상

31 March 2022. pp. 122-129

PDF XML

Abstract

In this paper, we propose a method to improve the performance of Transformer-based speech recognizers using data augmentation that locally adjusts the frame rate. First, the start time and length of the part to be augmented in the original voice data are randomly selected. Then, the frame rate of the selected part is changed to a new frame rate by using linear interpolation. Experimental results using the Wall Street Journal and LibriSpeech speech databases showed that the convergence time took longer than the baseline, but the recognition accuracy was improved in most cases. In order to further improve the performance, various parameters such as the length and the speed of the selected parts were optimized. The proposed method was shown to achieve relative performance improvement of 11.8 % and 14.9 % compared with the baseline in the Wall Street Journal and LibriSpeech speech databases, respectively.

Keywords

End-to-end speech recognition

Data augmentation

Linear interpolation

Frame rate change

본 논문은 프레임 속도를 국부적으로 조절하는 데이터 증강을 이용하여 트랜스포머 기반 음성 인식기의 성능을 개선하는 방법을 제안한다. 먼저, 원래의 음성데이터에서 증강할 부분의 시작 시간과 길이를 랜덤으로 선택한다. 그 다음, 선택된 부분의 프레임 속도는 선형보간법을 이용하여 새로운 프레임 속도로 변경된다. 월스트리트 저널 및 LibriSpeech 음성데이터를 이용한 실험결과, 수렴 시간은 베이스라인보다 오래 걸리지만, 인식 정확도는 대부분의 경우에 향상됨을 보여주었다. 성능을 더욱 향상시키기 위하여 변경 부분의 길이 및 속도 등 다양한 매개변수를 최적화 하였다. 제안 방법은 월스트리트 저널 및 LibriSpeech 음성 데이터에서 베이스라인과 비교하여 각각 11.8 % 및 14.9 % 의 상대적 성능 향상을 보여주는 것으로 나타났다.

키워드

종단간 음성 인식

데이터 증강

선형보간

프레임 속도 변경

References

T. Ko, V. Peddinti, D. Povey, and S. Khudanpur, "Audio augmentation for speech recognition," Proc. Interspeech, 35863589 (2015). 10.21437/Interspeech.2015-711

D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V. Le, "SpecAugment: A simple data augmentation method for automatic speech recognition," arXiv:1904.08779 (2019). 10.21437/Interspeech.2019-2680

X. Song, Z. Wu, Y. Huang, D. Su, and H. Meng, "SpecSwap: A simple data augmentation method for endtoend speech recognition," Proc. Interspeech, 581 585 (2020). 10.21437/Interspeech.2020-2275

D. B. Paul and J. M. Baker, "The design for the wall street journalbased CSR corpus," Proc. Speech and Natural Language Workshop, 357-362 (1992). 10.3115/1075527.1075614

V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, "LibriSpeech: An ASR corpus based on public domain audio books," Proc. ICASSP. 52065210 (2015). 10.1109/ICASSP.2015.7178964

W. Chan, N. Jaitly, Q. V. Le, and O. Vinyals, "Listen, attend and spell," arXiv:1508.01211 (2018).

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, "Attention is all you need," Proc. NIPS. 59986008 (2017).

A. Graves, S. Fernandez, F. Gomez, and J. Schmidhuber, "Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks," Proc. ICML. 369376 (2006). 10.1145/1143844.1143891

S. Kim, T. Hori, and S. Watanabe, "Joint CTCattention based endtoend speech recognition using multitask learning," Proc. ICASSP. 48354839 (2017). 10.1109/ICASSP.2017.7953075

Sox, Audio Manipulation Tool, http://sox.sourceforge.net/ , (Last viewed March 25, 2015).

S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y. Unno, N. E. Y. Soplin, J. Heymann, M. Wiesner, N. Chen, A. Renduchintala, and T. Ochiai, "ESPnet: End-to-end speech processing toolkit," arXiv: 1804.00015 (2018). 10.21437/Interspeech.2018-145629730221

Information

Publisher :The Acoustical Society of Korea
Publisher(Ko) :한국음향학회
Journal Title :The Journal of the Acoustical Society of Korea
Journal Title(Ko) :한국음향학회지
Volume : 41
No :2
Pages :122-129
Received Date : 2021-09-28
Revised Date : 2021-12-20
Accepted Date : 2021-12-30
DOI :https://doi.org/10.7776/ASK.2022.41.2.122

The Journal of the Acoustical Society of Korea ISSN:1225-4428(Print) 2287-3775(Online) 한국음향학회지

All Issue