Self-attentive residual temporal convolutional network-based indoor emergency awareness using contextual information from sound and human activity

Ju-Hwan Lee; Hyoung-Gook Kim

doi:10.7776/ASK.2024.43.6.637

All Issue

2024 Vol.43, Issue 6 Preview Page Next Page

Research Article

Self-attentive residual temporal convolutional network-based indoor emergency awareness using contextual information from sound and human activity 사운드와 활동 정보를 이용한 셀프 어텐션 잔차 시간적 합성곱 신경망 기반 실내 비상상황 인식

30 November 2024. pp. 637-643

PDF XML

Abstract

In this paper, we propose an indoor emergency awareness alert system that recognizes and delivers a user's emergency situation through sound and activity information collected through a mobile device. In the proposed system, the indoor spatial sound inputted through the microphone of the mobile device and the signals from the accelerometer and gyroscope installed in the mobile device are applied to the Self-Attentive Residual Temporal Convolutional Networks (SARTCN) to recognize emergency situations and quickly transmit the status of the monitored person to the guardian or family using the mobile transmission system. For training, we conducted experiments to recognize emergency situations using sound data recorded through actual mobile devices, as well as accelerometer and gyroscope signals, and our proposed SARTCN model achieved a high accuracy of 93.7 % in recognizing indoor emergency events by integrating sound event detection and activity recognition. The experimental results show that the proposed system can be effectively applied to a real-time monitoring system that automatically detects and responds to emergency situations of people who need assistance, such as the elderly and children.

Keywords

Temporal convolution network

Indoor emergency awareness

Sound event detection

Activity recognition

본 논문은 모바일 기기를 통해 수집된 소리와 활동 정보를 통해 사용자의 비상상황을 인식하여 알려주는 실내 비상인식 알람 시스템을 제안한다. 제안하는 시스템에서는 모바일 기기의 마이크로 입력되는 실내 공간 사운드, 그리고 모바일 기기 내에 장착된 가속도계와 자이로스코프의 신호를 셀프 어텐션 잔차 시간적 합성곱 신경망(Self-Attentive Residual Temporal Convolutional Networks, SARTCN)에 적용함으로써 비상상황을 인식하고 모바일 전송 시스템을 사용하여 모니터링되는 사람의 상태를 보호자나 가족에게 신속하게 전달한다. 학습을 위해 실제 모바일 기기를 통해 녹음된 사운드 데이터 및 가속도계, 자이로스코프의 신호를 활용하여 비상상황을 인식하는 실험을 수행하였으며, SARTCN은 사운드 이벤트 인식과 활동인식의 통합한 실내 비상 상황 인식에서 93.7 %의 높은 정확도를 보여주었다. 제안하는 시스템은 노인과 어린아이와 같이 도움이 필요한 사람들의 비상상황을 자동으로 감지하고 대처하는 모니터링 시스템에 효과적으로 적용될 수 있다.

키워드

상황인식

잔차 시간적 합성곱 신경망

인간 활동 인식

사운드 이벤트 검출

References

Y. Hajjaji, W. Boulila, I. R. Farah, I. Romdhani, and A. Hussain, "Big data and IoT-based applications in smart environments: A systematic review," Comput. Sci. Rev. 39, 100318 (2021).

10.1016/j.cosrev.2020.100318

A. R. Javed, F. Shahzad, S. ur Rehman, Y. B. Zikria, I. Razzak, Z. Jalil, and G. Xu, "Future smart cities: Requirements, emerging technologies, applications, challenges, and future aspects," Cities, 129, 103794 (2022).

10.1016/j.cities.2022.103794

T. Khan, "Towards an indoor gunshot detection and notification system using deep learning," Appl. Syst. Innov. 6, 94 (2023).

10.3390/asi6050094

S. Pandya and H. Ghayvat, "Ambient acoustic event assistive framework for identification, detection, and recognition of unknown acoustic events of a residence," Adv. Eng. Inform. 47, 101238 (2021).

10.1016/j.aei.2020.101238

F. S. Saeed, A. A. Bashit, V. Viswanathan, and D. Valles, "An initial machine learning-based victim's scream detection analysis for burning sites," Appl. Sci. 11, 8425 (2021).

10.3390/app11188425

R. Mathur, T. Chintala, and D. Rajeswari, "Identification of illicit activities & scream detection using computer vision & deep learning," Proc. ICICCS, IEEE, 1243-1250 (2022).

10.1109/ICICCS53718.2022.9787991

A. Munir, A. Aved, and E. Blasch, "Situational awareness: techniques, challenges, and prospects," AI, 3, 55-77 (2022).

10.3390/ai3010005

M. A. Al-Qaness, A. M. Helmi, A. Dahou, and M. A. Elaziz, "The applications of metaheuristics for human activity recognition and fall detection using wearable sensors: A comprehensive analysis," Biosensors, 12, 821 (2022).

10.3390/bios1210082136290958PMC9599938

B. Zhang and Y. Zhu, "Self-attentive residual TCN speech emotion recognition with fused acoustic features," AJCIS, 6, 42-51 (2023).

10.25236/AJCIS.2023.060306

A. Vaswani, "Attention is all you need," NIPS, 5998-6008 (2017).

Information

Publisher :The Acoustical Society of Korea
Publisher(Ko) :한국음향학회
Journal Title :The Journal of the Acoustical Society of Korea
Journal Title(Ko) :한국음향학회지
Volume : 43
No :6
Pages :637-643
Received Date : 2024-09-03
Revised Date : 2024-10-11
Accepted Date : 2024-10-21
DOI :https://doi.org/10.7776/ASK.2024.43.6.637

The Journal of the Acoustical Society of KoreaISSN:1225-4428(Print) 2287-3775(Online)한국음향학회

All Issue