All Issue

2020 Vol.39, Issue 5 Preview Page

Research Article

Acoustic model training using self-attention for low-resource speech recognition

저자원 환경의 음성인식을 위한 자기 주의를 활용한 음향 모델 학습

Hosung Park1
,
Ji-Hwan Kim1∗
박 호성1
,
김 지환1∗
1서강대학교 컴퓨터공학과
∗Corresponding Author
30 September 2020. pp. 483-489
PDF XML Citation
Abstract

This paper proposes acoustic model training using self-attention for low-resource speech recognition. In low-resource speech recognition, it is difficult for acoustic model to distinguish certain phones. For example, plosive /d/ and /t/, plosive /g/ and /k/ and affricate /z/ and /ch/. In acoustic model training, the self-attention generates attention weights from the deep neural network model. In this study, these weights handle the similar pronunciation error for low-resource speech recognition. When the proposed method was applied to Time Delay Neural Network-Output gate Projected Gated Recurrent Unit (TNDD-OPGRU)-based acoustic model, the proposed model showed a 5.98 % word error rate. It shows absolute improvement of 0.74 % compared with TDNN-OPGRU model.

Keywords
Speech recognition
Acoustic model
Self-attention mechanism
Low-resource environment

본 논문에서는 저자원 환경의 음성인식에서 음향 모델의 성능을 높이기 위한 음향 모델 학습 방법을 제안한다. 저자원 환경이란, 음향 모델에서 100시간 미만의 학습 자료를 사용한 환경을 말한다. 저자원 환경의 음성인식에서는 음향 모델이 유사한 발음들을 잘 구분하지 못하는 문제가 발생한다. 예를 들면, 파열음 /d/와 /t/, 파열음 /g/와 /k/, 파찰음 /z/와 /ch/ 등의 발음은 저자원 환경에서 잘 구분하지 못한다. 자기 주의 메커니즘은 깊은 신경망 모델로부터 출력된 벡터에 대해 가중치를 부여하며, 이를 통해 저자원 환경에서 발생할 수 있는 유사한 발음 오류 문제를 해결한다. 음향 모델에서 좋은 성능을 보이는 Time Delay Neural Network(TDNN)과 Output gate Projected Gated Recurrent Unit (OPGRU)의 혼합 모델에 자기 주의 기반 학습 방법을 적용했을 때, 51.6 h 분량의 학습 자료를 사용한 한국어 음향 모델에 대하여 단어 오류율 기준 5.98 %의 성능을 보여 기존 기술 대비 0.74 %의 절대적 성능 개선을 보였다.

키워드
음성 인식
음향 모델
자기 주의 메커니즘
저자원 환경
References
1
C. Weng and D. Yu, "A comparison of lattice-free discriminative training criteria for purely sequence- trained neural network acoustic models," Proc. ICASSP. 6430-6434 (2019).
10.1109/ICASSP.2019.8683664
2
W. Michel, R. Schluter, and H. Ney, "Comparison of lattice-free and lattice-based sequence discriminative training criteria for LVCSR," arXiv:1907.01409 (2019).
10.21437/Interspeech.2019-2254
3
J. Jorge, A. Gimenez, J. Iranzo-Sanchez, J. Civera, A. Sanchis, and A. Juan, "Real-time one-pass decoder for speech recognition using LSTM language models," Proc. Interspeech, 3820-3824 (2019).
10.21437/Interspeech.2019-2798
4
J. Y. Chung, C. Gulcehre, K. H. Cho, and Y. Bengio, "Empirical evaluation of gated recurrent neural networks on sequence modeling," arXiv:1412.3555 (2014).
5
V. Peddinti, D. Povey, and S. Khudanpur, "A time delay neural network architecture for efficient modeling of long temporal contexts," Proc. Interspeech, 2-6 (2015).
6
B. Christian and T. Griffiths, Algorithms to Llive by: The Computer Science of Human Decisions Chapter 7: Overfitting (William Collins, Hampshire, 2017), pp. 149-168.
7
D. Povey, H. Hadian, P. Ghahremani, K. Li, and S. Khudanpur, "A time-restricted self-attention layer for ASR," Proc. ICASSP. 5874-5878 (2018).
10.1109/ICASSP.2018.8462497
8
G. Cheng, D. Povey, L. Huang, J. Xu, S. Khudanpur, and Y. Yan, "Output-gate projected gated recurrent unit for speech recognition," Proc. Interspeech, 1793- 1797 (2018).
10.21437/Interspeech.2018-140328916962
9
D. Bahdanau, K. H. Cho, and Y. Bengio, "Neural machine translation by jointly learning to align and translate," Proc. ICLR. 1-15 (2015).
10
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is all you need," Proc. NIPS. 5999-6009 (2017).
11
M.-T. Luong, H. Pham, and C. D. Manning, "Effective approaches to attention-based neural machine translation," Proc. EMNLP. 1412-1421 (2015).
10.18653/v1/D15-1166
12
D. Povey, H. Hadian, P. Ghahremani, K. Li, and S. Khudanpur, "A time-restricted self-attention layer for ASR," Proc. ICASSP. 5874-5878 (2018).
10.1109/ICASSP.2018.8462497
13
Zeroth Korean, http://openslr.org/40/, (Last viewed June 4, 2020).
14
A. Stolcke, "SRILM-an extensible language modeling toolkit," Proc. ICSLP. 901-904 (2002).
15
H. Xu, T. Chen, D. Gao, Y. Wang, K. Li, N. Goel, Y. Carmiel, D. Povey, and S. Khudanpur, "a pruned RNNLM lattice-rescoring algorithm for automatic speech recognition," Proc. ICASSP. 5929-5933 (2018).
10.1109/ICASSP.2018.8461974
16
D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V. Le, "SpecAugment: a simple data augmentation method for automatic speech recognition," Proc. Interspeech, 2613-2617 (2019).
10.21437/Interspeech.2019-2680
17
Y.-Y. Wang, A. Acero, and C. Chelba, "Is word error rate a good indicator for spoken language understanding accuracy," Proc. ASRU. 577-582 (2003).
Information
  • Publisher :The Acoustical Society of Korea
  • Publisher(Ko) :한국음향학회
  • Journal Title :The Journal of the Acoustical Society of Korea
  • Journal Title(Ko) :한국음향학회지
  • Volume : 39
  • No :5
  • Pages :483-489
  • Received Date : 2020-08-07
  • Accepted Date : 2020-09-04
  • DOI :https://doi.org/10.7776/ASK.2020.39.5.483
Journal Informaiton The Journal of the Acoustical Society of Korea The Journal of the Acoustical Society of Korea
E-Book
  • scopus_JASK
  • ESCI_JASK
  • Google scholar_JASK
  • NRF
  • KOFST
  • KISTI Current Status
  • KISTI Cited-by
  • crosscheck
  • orcid
  • ccl
Journal Informaiton Journal Informaiton - close