MRNet: A multi-route convolutional neural network for robust music representation learning

Jungwoo Heo; Hyun-seo Shin; Chan-yeong Lim; Kyo-won Koo; Seung-bin Kim; Jisoo Son; Ha-Jin Yu

doi:10.7776/ASK.2025.44.5.516

All Issue

2025 Vol.44, Issue 5 Preview Page Next Page

Research Article

MRNet: A multi-route convolutional neural network for robust music representation learning 음악 표현 학습을 위한 다중 경로 합성곱 신경망

30 September 2025. pp. 516-523

PDF XML

Abstract

Music Information Retrieval (MIR) focuses on extracting semantic information embedded in audio signals, such as genre, artist identity, and tempo. These musical cues cover a wide range of temporal characteristics, from short-term features like pitch and timbre to long-term patterns such as melody and mood, and they require processing at multiple levels of abstraction. In this paper, we propose a Multi-Route Neural Network (MRNet) designed to capture musical representations that reflect both short-term and long-term characteristics, as well as different levels of abstraction. To achieve this, MRNet stacks several convolutional layers with different dilation rates, allowing the model to analyze audio patterns over multiple time scales. Additionally, it introduces a specialized module called the multi-route Res2Block, which separates the processing path into multiple branches. Each branch processes the input to a different depth, enabling the network to extract low-level, mid-level, and high-level features simultaneously. MRNet achieves classification accuracies of 94.5 %, 56.6 %, 63.2 %, and 71.3 % on the GTZAN, FMA Small, FMA Large, and Melon datasets, respectively, outperforming previous Convolution Neural Network(CNN)-based approaches. These results demonstrate the effectiveness of MRNet in learning robust and hierarchical music representations for MIR tasks.

Keywords

Deep learning

Music information retrieval

Convolution Neural Network (CNN)

Music representation

음악 정보 검색(Music Information Retrieval, MIR)은 오디오 신호에 내재된 장르, 아티스트 정체성, 템포와 같은 의미적 정보를 추출하는 데 중점을 둔 연구 분야이다. 이러한 음악적 단서들은 피치나 음색과 같은 단기적 특성부터 멜로디나 분위기와 같은 장기적 패턴에 이르기까지 다양한 시간적 특성을 포함하며, 여러 수준의 추상화된 처리를 요구한다. 본 논문에서는 음악의 단기적 특성과 장기적 특성은 물론, 다양한 추상화 수준을 모두 반영할 수 있는 음악 표현을 학습하기 위해 설계된 다중 경로 신경망(Multi-Route Neural Network, MRNet)을 제안한다. 이를 위해 MRNet은 수용 영역의 크기가 서로 다른 여러 개의 확장 합성곱 계층을 적층하여, 다양한 시간 범위에 걸친 오디오 패턴을 효과적으로 분석할 수 있도록 한다. 또한, 입력 신호를 여러 경로로 나누어 처리하는 특수 구조인 multi-route Res2Block을 도입하여, 각 경로에서 서로 다른 깊이로 특징을 추출할 수 있게 설계하였다. 이 구조를 통해 네트워크는 저차, 중차, 고차 수준의 특성을 동시에 학습할 수 있다. MRNet은 GTZAN, FMA Small, FMA Large, Melon 데이터셋에서 각각 94.5 %, 56.6 %, 63.2 %, 71.3 %의 분류 정확도를 기록하며 기존 합성곱 신경망(Convolution Neural Network, CNN) 기반 접근법들을 능가하는 성능을 보였다. 이러한 결과는 MIR 과제에서 강건하고 계층적인 음악 표현 학습을 위한 MRNet의 효과성을 입증한다.

키워드

딥러닝

음악 정보 검색

합성곱 신경망 (Convolution Neural Network, CNN)

음악 표현

References

G. Tzanetakis and P. Cook, “Musical genre classification of audio signals,” IEEE Trans on speech and audio process. 10, 293-302 (2002).

10.1109/TSA.2002.800560

A. Casey, R. Veltkamp, M. Goto, M. Leman, C. Rhodes, and M. Slaney, “Content-based music information retrieval: Current directions and future challenges,” Proc. IEEE, 96, 668-696 (2008).

10.1109/JPROC.2008.916370

D. Bogdanov, N. Wack, E. Gómez Gutiérrez, S. Gulati, P. H. Boyer, O. Mayor, G. R. Trepat, J. Salamon, J. R. Z. González, and X. Serra, “Essentia: An audio analysis library for music information retrieval,” Proc. 14th ISMIR, 493-498 (2013).

A. Van Den Oord, S. Dieleman, and B. Schrauwen, “Deep content-based music recommendation,” Proc. NIPS, 26 (2013).

K. Choi, G. Fazekas, M. Sandler, and K. Cho, “Convolutional recurrent neural networks for music classification,” Proc. ICASSP, 2392-2396 (2017).

10.1109/ICASSP.2017.7952585

Y. Xu and W. Zhou, “A deep music genres classification model based on CNN with Squeeze & Excitation Block,” Proc. IEEE APSIPA ASC. 332- 338 (2020).

A. Ferraro, D. Bogdanov, X. S. Jay, H. Jeon, and J. Yoon, “How low can you go? Reducing frequency and time resolution in current CNN architectures for music auto-tagging,” Proc. 28th EUSIPCO, 131-135 (2021).

10.23919/Eusipco47968.2020.9287769

D. Kim, T. T. Sung, Y. S. Cho, G. Lee, and B. C. Sohn, “A single predominant instrument recognition of polyphonic music using CNN-based timbre analysis,” Int. J. Eng. Technol. 7, 590-595 (2018).

10.14419/ijet.v7i3.34.19388

S. Joshi, T. Jain, and N. Nair, “Emotion based music recommendation system using LSTM-CNN architecture,” Proc. IEEE ICCCNT. 01-06 (2021).

10.1109/ICCCNT51525.2021.9579813

C. Liu, L. Feng, G. Liu, H. Wang, and S. Liu, “Bottom-up broadcast neural network for music genre classification,” Multimed. Tools Appl. 80, 7313-7331 (2021).

10.1007/s11042-020-09643-6

J. Lee and J. Nam, “Multi-level and multi-scale feature aggregation using pretrained convolutional neural networks for music auto-tagging,” IEEE Signal Processing Letters, 24, 1208-1212 (2017).

10.1109/LSP.2017.2713830

S.-H. Gao, M.-M. Cheng, K. Zhao, X.-Y. Zhang, M.-H. Yang, and P. Torr, “Res2Net: A new multi- scale backbone architecture,” IEEE Trans. Pattern Anal. Mach. Intell. 43, 652-662 (2019).

10.1109/TPAMI.2019.2938758

I. Ikhsan, L. Novamizanti, and I. N. A. Ramatryana, “Automatic musical genre classification of audio using Hidden Markov Model,” Proc. 2^nd ICoICT, 397-402 (2014).

10.1109/ICoICT.2014.6914095

FMA: A Dataset for Music Analysis, https://arxiv.org/abs/1612.01840, (Last viewed September 16, 2025).

A. Ferraro, Y. Kim, S. Lee, B. Kim, N. Jo, S. Lim, S. Lim, J. Jang, S. Kim, and X. Serra, “Melon playlist dataset: A public dataset for audio-based playlist generation and music tagging,” Proc. IEEE ICASSP, 536-540 (2021).

10.1109/ICASSP39728.2021.9413552

S.-H. Cho, Y. Park, and J. Lee, “Effective music genre classification using late fusion convolutional neural network with multiple spectral features,” Proc. IEEE ICCE-Asia, 1-4 (2022).

10.1109/ICCE-Asia57006.2022.9954732

S. Chen, C. Wang, Z. Chen, Y. Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y. Qian , Y. Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “WavLM: Large-scale self-supervised pre- training for full stack speech processing,” IEEE J. Sel. Top. Signal Process. 16, 1505-1518 (2022).

10.1109/JSTSP.2022.3188113

Y. Yi, K.-Y. Chen, and H.-Y.Gu, “Mixture of CNN experts from multiple acoustic feature domain for music genre classification,” Proc. IEEE APSIPA ASC. 1250-1255 (2019).

10.1109/APSIPAASC47483.2019.9023314

Md. N. A. Siddiquee, Md. A. Hossain, and F. Wahida, “An effective machine learning approach for music genre classification with mel spectrograms and Knn,” Proc. IEEE IC3S, 1-4 (2013).

10.1109/IC3S57698.2023.10169397

Information

Publisher :The Acoustical Society of Korea
Publisher(Ko) :한국음향학회
Journal Title :The Journal of the Acoustical Society of Korea
Journal Title(Ko) :한국음향학회지
Volume : 44
No :5
Pages :516-523
Received Date : 2025-08-06
Revised Date : 2025-08-29
Accepted Date : 2025-08-30
DOI :https://doi.org/10.7776/ASK.2025.44.5.516

The Journal of the Acoustical Society of KoreaISSN:1225-4428(Print) 2287-3775(Online)한국음향학회

All Issue