Speech emotion recognition using attention mechanism-based deep neural networks

Sang-Sun Ko; Hye-Seung Cho; Hyoung-Gook Kim

doi:10.7776/ASK.2017.36.6.407

All Issue

2017 Vol.36, Issue 6 Preview Page Next Page

Speech emotion recognition using attention mechanism-based deep neural networks 주목 메커니즘 기반의 심층신경망을 이용한 음성 감정인식

30 November 2017. pp. 407-412

PDF XML

Abstract

In this paper, we propose a speech emotion recognition method using a deep neural network based on the attention mechanism. The proposed method consists of a combination of CNN (Convolution Neural Networks), GRU (Gated Recurrent Unit), DNN (Deep Neural Networks) and attention mechanism. The spectrogram of the speech signal contains characteristic patterns according to the emotion. Therefore, we modeled characteristic patterns according to the emotion by applying the tuned Gabor filters as convolutional filter of typical CNN. In addition, we applied the attention mechanism with CNN and FC (Fully-Connected) layer to obtain the attention weight by considering context information of extracted features and used it for emotion recognition. To verify the proposed method, we conducted emotion recognition experiments on six emotions. The experimental results show that the proposed method achieves higher performance in speech emotion recognition than the conventional methods.

Keywords

Speech emotion recognition

Attention mechanism

Deep neural networks

Gabor filters

본 논문에서는 주목 메커니즘 기반의 심층 신경망을 사용한 음성 감정인식 방법을 제안한다. 제안하는 방식은 CNN(Convolution Neural Networks), GRU(Gated Recurrent Unit), DNN(Deep Neural Networks)의 결합으로 이루어진 심층 신경망 구조와 주목 메커니즘으로 구성된다. 음성의 스펙트로그램에는 감정에 따른 특징적인 패턴이 포함되어 있으므로 제안하는 방식에서는 일반적인 CNN에서 컨벌루션 필터를 tuned Gabor 필터로 사용하는 GCNN(Gabor CNN)을 사용하여 패턴을 효과적으로 모델링한다. 또한 CNN과 FC(Fully-Connected)레이어 기반의 주목 메커니즘을 적용하여 추출된 특징의 맥락 정보를 고려한 주목 가중치를 구해 감정인식에 사용한다. 본 논문에서 제안하는 방식의 검증을 위해 6가지 감정에 대해 인식 실험을 진행하였다. 실험 결과, 제안한 방식이 음성 감정인식에서 기존의 방식보다 더 높은 성능을 보였다.

키워드

음성 감정인식

주목 메커니즘

심층 신경망

가버 필터

References

Q. Mao, M. Dong, Z. Huang, and Y. Zhan, “Learning salient features for speech emotion recognition using convolutional neural networks,” IEEE Trans. Multime-dia, 16, 2203-2213 (2014).

T. N. Sainath, O. Vinyals, A. Senior, and H. Sak, “Convolutional, long short-term memory, fully connected deep neural networks.” in IEEE ICASSP, 4580-4584 (2015).

S. Mirsamadi, E. Barsoum, and C. Zhang, “Automatic speech emotion recognition using recurrent neural networks with local attention,” in IEEE ICASSP, 2227-2231 (2017).

S. Y. Chang and N. Morgan, “Robust CNN-based speech recognition with gabor filter kernels,” in Interspeech, 905-909 (2014).

D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” arXiv:1409.0473 (2014).

S. Haq and P. J. B. Jackson, “Speaker-dependent audio-visual emotion recognition,” in AVSP, 53-58 (2009).

Information

Publisher :The Acoustical Society of Korea
Publisher(Ko) :한국음향학회
Journal Title :The Journal of the Acoustical Society of Korea
Journal Title(Ko) :한국음향학회지
Volume : 36
No :6
Pages :407-412
Received Date : 2017-06-28
Revised Date : 2017-07-21
Accepted Date : 2017-11-29
DOI :https://doi.org/10.7776/ASK.2017.36.6.407

The Journal of the Acoustical Society of KoreaISSN:1225-4428(Print) 2287-3775(Online)한국음향학회

All Issue