Abstract
In marimba music, ‘stickings’ are the choices of mallets used to strike each note. Stickings significantly influence both the physical facility and expressive quality of the music performance. Choosing ‘good’ stickings and evaluating one’s stickings are complex choices, often relying vaguely on trial-and-error. Machine learning (ML) approaches, particularly with advances in sequence-to-sequence techniques, have proved suited for similar complex classification problems, motivating their application in our study. We address the sticking problem by developing Long Short-Term Memory (LSTM) models to generate stickings in 4-mallet marimba music trained on exercises from Leigh Howard Stevens’ Method of Movement for Marimba. Model performance was measured under a range of metrics to account for multiple sticking possibilities, with LSTM models achieving a maximum average micro-accuracy of 97.3%. Finally, we discuss qualitative observations in sticking predictions and limitations of this study and provide direction for further development in this field.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Al Kasimi, A., Nichols, E., Raphael, C.: Automatic fingering system (afs). In: Poster presentation at ISMIR, London (2005)
Al Kasimi, A., Nichols, E., Raphael, C.: A simple algorithm for or automatic generation of polyphonic piano fingerings (2007)
Balliauw, M., Herremans, D., Palhazi Cuervo, D., Sörensen, K.: A variable neighborhood search algorithm to generate piano fingerings for polyphonic sheet music. Int. Trans. Oper. Res. 24(3), 509–535 (2017)
Barbancho, A.M., Klapuri, A., Tardon, L.J., Barbancho, I.: Automatic transcription of guitar chords and fingering from audio. IEEE Trans. Audio Speech Lang. Process. 20(3), 915–921 (2012). https://doi.org/10.1109/TASL.2011.2174227
Bretan, P.M.: Towards an embodied musical mind: generative algorithms for robotic musicians. Ph.D. thesis, Georgia Institute of Technology (2017)
Briot, J.P., Hadjeres, G., Pachet, F.: Deep Learning Techniques for Music Generation. Springer, Heidelberg (2020). https://doi.org/10.1007/978-3-319-70163-9
Briot, J.P., Pachet, F.: Deep learning for music generation: challenges and directions. Neural Comput. Appl. 32(4), 981–993 (2020)
Burlet, G., Fujinaga, I.: Robotaba guitar tablature transcription framework. In: Proceedings of the 14th International Society for Music Information Retrieval Conference, ISMIR, Curitiba, Brazil, pp. 517–522 (2013)
Choi, K., Fazekas, G., Cho, K., Sandler, M.: A tutorial on deep learning for music information retrieval. arXiv preprint arXiv:1709.04396 (2017)
Choi, K., Fazekas, G., Sandler, M., Cho, K.: Convolutional recurrent neural networks for music classification. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2392–2396. IEEE (2017)
Chollet, F., et al.: Keras (2015). https://github.com/fchollet/keras
Coca, A.E., Corrêa, D.C., Zhao, L.: Computer-aided music composition with LSTM neural network and chaotic inspiration. In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE (2013)
Corrêa, D.C., Levada, A.L., Saito, J.H., Mari, J.F.: Neural network based systems for computer-aided musical composition: supervised x unsupervised learning. In: Proceedings of the 2008 ACM Symposium on Applied Computing, pp. 1738–1742 (2008)
Cuthbert, M., Ariza, C., Hogue, B., Oberholtzer, J.W.: music21 (version 5.7.2) [python package] (2006–2021). https://web.mit.edu/music21
Johnson, D., Damian, G.T.D.: Detecting hand posture in piano playing using depth data. Comput. Music J. 43(1), 59–78 (2019). http://muse.jhu.edu/article/746693
De Prisco, R., Zaccagnino, G., Zaccagnino, R.: A differential evolution algorithm assisted by ANFIS for music fingering. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) EC/SIDE -2012. LNCS, vol. 7269, pp. 48–56. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29353-5_6
Eck, D., Lapalme, J.: Learning musical structure directly from sequences of music. University of Montreal, Department of Computer Science, CP 6128, 48 (2008)
Eck, D., Schmidhuber, J.: Finding temporal structure in music: blues improvisation with LSTM recurrent networks. In: Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing, pp. 747–756. IEEE (2002)
Eck, D., Schmidhuber, J.: A first look at music composition using LSTM recurrent neural networks. Istituto Dalle Molle Di Studi Sull Intelligenza Artificiale 103, 48 (2002)
Hart, M., Bosch, R., Tsai, E.: Finding optimal piano fingerings. UMAP J. 21(2), 167–177 (2000)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Humphrey, E.J., Bello, J.P.: From music audio to chord tablature: teaching deep convolutional networks to play guitar. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6974–6978 (2014). https://doi.org/10.1109/ICASSP.2014.6854952
Liu, I., Ramakrishnan, B., et al.: Bach in 2014: music composition with recurrent neural network. arXiv preprint arXiv:1412.3191 (2014)
Lyu, Q., Wu, Z., Zhu, J.: Polyphonic music modelling with LSTM-RTRBM. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 991–994 (2015)
Mistler, E.: Generating Guitar Tablatures with Neural Networks. University of Edinburgh, Thesis (2017)
Miura, M., Hirota, I., Hama, N., Yanagida, M.: Constructing a system for finger-position determination and tablature generation for playing melodies on guitars. Syst. Comput. Japan 35(6), 10–19 (2004). https://doi.org/10.1002/scj.10609. https://search.ebscohost.com/login.aspx?direct=true &db=iih &AN=13217635 &site=ehost-live
Musafia, J.: The art of fingering in piano playing. MCA Music (1971)
MuseScore: MuseScore (version 3.1.0.7078) [computer software] (2019). https://musescore.org
Nakamura, E., Ono, N., Sagayama, S.: Merged-output hmm for piano fingering of both hands. In: ISMIR, pp. 531–536 (2014)
Nakamura, E., Saito, Y., Yoshii, K.: Statistical learning and estimation of piano fingering. Inf. Sci. 517, 68–85 (2020). https://doi.org/10.1016/j.ins.2019.12.068. http://www.sciencedirect.com/science/article/pii/S0020025519311879
Parncutt, R., Sloboda, J.A., Clarke, E.F., Raekallio, M., Desain, P.: An ergonomic model of keyboard fingering for melodic fragments. Music Percept. 14(4), 341–382 (1997)
Python: Python (version 3.7.3) [computer software] (2001–2021). https://python.org
Ramos, J.V., Ramos, A.S., Silla, C.N., Sanches, D.S.: An evaluation of different evolutionary approaches applied in the process of automatic transcription of music scores into tablatures. In: 2016 IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI), pp. 663–669 (2016). https://doi.org/10.1109/ICTAI.2016.0106
Savery, R., Weinberg, G.: Shimon the robot film composer and deepscore. In: Proceedings of Computer Simulation of Musical Creativity, p. 5 (2018)
Sayegh, S.I.: Fingering for string instruments with the optimum path paradigm. Comput. Music Jou. 13(3), 76–84 (1989). https://doi.org/10.2307/3680014. http://www.jstor.org.ezproxy.library.uwa.edu.au/stable/3680014
Schedl, M.: Deep learning in music recommendation systems. Front. Appl. Math. Stat. 5, 44 (2019)
Sigtia, S., Benetos, E., Dixon, S.: An end-to-end neural network for polyphonic piano music transcription. IEEE/ACM Trans. Audio Speech Lang. Process. 24(5), 927–939 (2016)
Stevens, L.H.: Method of Movement for Marimba: With 590 Exercises. Marimba Productions, Inc., Neptune City (2005)
Sturm, B.L., Santos, J.F., Ben-Tal, O., Korshunova, I.: Music transcription modelling and composition using deep learning. arXiv preprint arXiv:1604.08723 (2016)
Takegawa, Y., Terada, T., Nishio, S.: Design and implementation of a real-time fingering detection system for piano performance. In: ICMC (2006)
Tuohy, D.R., Potter, W.: Guitar tablature creation with neural networks and distributed genetic search. In: Proceedings of the 19th International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, IEA-AIE06, Annecy, France (2006)
Van Den Oord, A., Dieleman, S., Schrauwen, B.: Deep content-based music recommendation. In: Neural Information Processing Systems Conference (NIPS 2013), vol. 26. Neural Information Processing Systems Foundation (NIPS) (2013)
Walter, D.W.: The Performance of Contrapuntal Music on the Marimba and Vibraphone. Ph.D. thesis, Temple University (1984)
Wang, X., Wang, Y.: Improving content-based and hybrid music recommendation using deep learning. In: Proceedings of the 22nd ACM international conference on Multimedia, pp. 627–636 (2014)
Yang, N., Savery, R., Sankaranarayanan, R., Zahray, L., Weinberg, G.: Mechatronics-driven musical expressivity for robotic percussionists. arXiv preprint arXiv:2007.14850 (2020)
Yazawa, K., Itoyama, K., Okuno, H.G.: Automatic transcription of guitar tablature from audio signals in accordance with player’s proficiency. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3122–3126 (2014). https://doi.org/10.1109/ICASSP.2014.6854175
Ycart, A., Benetos, E., et al.: A study on LSTM networks for polyphonic music sequence modelling. In: ISMIR (2017)
Yonebayashi, Y., Kameoka, H., Sagayama, S.: Automatic decision of piano fingering based on a hidden markov models. In: IJCAI, vol. 7, pp. 2915–2921 (2007)
Yu, Y., Luo, S., Liu, S., Qiao, H., Liu, Y., Feng, L.: Deep attention based music genre classification. Neurocomputing 372, 84–91 (2020)
Zeltsman, N.: Four-Mallet Marimba Playing: A Musical Approach for All Levels. H. Leonard, Milwaukee, WI (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chong, J.K., Corrêa, D. (2022). Predicting Marimba Stickings Using Long Short-Term Memory Neural Networks. In: Aziz, H., Corrêa, D., French, T. (eds) AI 2022: Advances in Artificial Intelligence. AI 2022. Lecture Notes in Computer Science(), vol 13728. Springer, Cham. https://doi.org/10.1007/978-3-031-22695-3_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-22695-3_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-22694-6
Online ISBN: 978-3-031-22695-3
eBook Packages: Computer ScienceComputer Science (R0)