A study on improving the performance of the machine-learning based automatic music transcription model by utilizing pitch number information

Daeho Lee; Seokjin Lee

doi:10.7776/ASK.2024.43.2.207

All Issue

2024 Vol.43, Issue 2 Preview Page Next Page

Research Article

A study on improving the performance of the machine-learning based automatic music transcription model by utilizing pitch number information 음고 개수 정보 활용을 통한 기계학습 기반 자동악보전사 모델의 성능 개선 연구

31 March 2024. pp. 207-213

PDF XML

Abstract

In this paper, we study how to improve the performance of a machine learning-based automatic music transcription model by adding musical information to the input data. Where, the added musical information is information on the number of pitches that occur in each time frame, and which is obtained by counting the number of notes activated in the answer sheet. The obtained information on the number of pitches was used by concatenating it to the log mel-spectrogram, which is the input of the existing model. In this study, we use the automatic music transcription model included the four types of block predicting four types of musical information, we demonstrate that a simple method of adding pitch number information corresponding to the music information to be predicted by each block to the existing input was helpful in training the model. In order to evaluate the performance improvement proceed with an experiment using MIDI Aligned Piano Sounds (MAPS) data, as a result, when using all pitch number information, performance improvement was confirmed by 9.7 % in frame-based F1 score and 21.8 % in note-based F1 score including offset.

Keywords

Automatic music transcription

Machine learning

Pitch number information

Polyphonic traanscription

본 논문은 기계학습 기반 자동악보전사 모델의 입력에 음악적인 정보를 추가하는 방법을 통해 원하는 성능 향상을 얻는 방법을 다루었다. 여기서, 추가한 음악적인 정보는 각 시간 단위마다 발생하는 음고 개수 정보이며, 이는 정답지에서 활성화되는 음고 개수를 세는 방법으로 획득한다. 획득한 음고 개수 정보는 기존 모델의 입력인 로그 멜-스펙트로그램 아래에 연결하여 사용했다. 본 연구에서는 네 가지 음악 정보를 예측하는 네 종류의 블록이 포함된 자동악보전사 모델을 사용하였으며, 각 블록이 예측해야하는 음악 정보에 해당하는 음고 개수 정보를 기존의 입력에 추가해주는 간단한 방법이 모델의 학습에 도움이 됨을 확인했다. 성능 개선을 검증하기 위하여 MIDI Aligned Piano Sounds(MAPS) 데이터를 활용하여 실험을 진행하였으며, 그 결과 모든 음고 개수 정보를 활용할 경우 프레임 기준 F1 점수에서 9.7 %, 끝점을 포함한 노트 기준 F1 점수에서 21.8 %의 성능 향상을 확인하였다.

키워드

자동악보전사

기계학습

음고 개수 정보

다성전사

References

P. H. Peeling, A. T. Cemgil, and S. J. Godsill, "Generative spectrogram factorization models for polyphonic piano transcription," IEEE Trans. on Audio, Speech, and Lang. Process. 18, 519-527 (2009). 10.1109/TASL.2009.2029769

L. Su and Y.-H. Yang, "Combining spectral and temporal representations for multipitch estimation of polyphonic music," IEEE/ACM Trans. on Audio, Speech, and Lang. Process. 23, 1600-1612 (2015). 10.1109/TASLP.2015.2442411

E. Vincent, N. Bertin, and R. Badeau, "Adaptive harmonic spectral decomposition for multiple pitch estimation," IEEE Trans. on Audio, Speech, and Lang. Process. 18, 528-537 (2009). 10.1109/TASL.2009.2034186

S. Böck and M. Schedl, "Polyphonic piano note transcription with recurrent neural networks," Proc. IEEE ICASSP, 121-124 (2012). 10.1109/ICASSP.2012.6287832

S. Sigtia, E. Benetos, and S. Dixon, "An end-to-end neural network for polyphonic piano music transcription," IEEE/ACM Trans. on Audio, Speech, and Lang. Process. 24, 927-939 (2016). 10.1109/TASLP.2016.2533858

C. Hawthorne, E. Elsen, J. Song, A. Roberts, I. Simon, C. Raffel, J. Engel, S. Oore, and D. Eck, "Onsets and frames: Dual-objective piano transcription," arXiv preprint arXiv:1710.11153 (2017).

P. Smaragdis and J. C. Brown, "Non-negative matrix factorization for polyphonic music transcription," Proc. IEEE WASPAA, No. 03TH8684 (2003).

S. Lee, "Estimating the rank of a nonnegative matrix factorization model for automatic music transcription based on stein's unbiased risk estimator," Appl. Sci. 10, 2911 (2020). 10.3390/app10082911

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, "Pytorch: An imperative style, high-performance deep learning library," Proc. NeurIPS, 1-12 (2019).

D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980 (2014).

V. Emiya, N. Bertin, B. David, and R. Badeau, "MAPS-A piano database for multipitch estimation and automatic transcription of music," INRIA, Research Rep., 2010.

Information

Publisher :The Acoustical Society of Korea
Publisher(Ko) :한국음향학회
Journal Title :The Journal of the Acoustical Society of Korea
Journal Title(Ko) :한국음향학회지
Volume : 43
No :2
Pages :207-213
Received Date : 2024-01-23
Accepted Date : 2024-02-16
DOI :https://doi.org/10.7776/ASK.2024.43.2.207

The Journal of the Acoustical Society of Korea ISSN:1225-4428(Print) 2287-3775(Online) 한국음향학회지

All Issue