Towards a device for helping deaf people to dance: estimation of “forró” bar length using artificial neural network
Keywords:
Rehabilitation engineering, music, inclusion, multilayer perceptron, Brazil, datasetAbstract
Dance has the potential to improve people's quality of life, as well as assist to decrease depression and anxiety. However, the lack of technologies capable of exploring alternative senses of hearing limits music and dance's beneficial effects on listeners. In order to find a model capable of being implemented in accessible devices, this work evaluated the use of a model based on neural networks to estimate the forró music bar length. Model variations were trained for seven datasets composed of mixes of music samples without noise, with real noise and with white noise. For each dataset, the best variation was selected and these were evaluated for the same real noise samples. The model variations that were presented to samples with real noise in the training estimated the bar duration with an average percentage error of less than 7% in the test step, being significantly better the model trained only with real sample. The evaluated model was able to estimate the length of the forró music bar length, even in real scenarios, as long as it was presented in this scenario during training. Increased database diversity and the use of data augmentation techniques can lead to improvements in the generalizability of the model. The simplicity of the evaluated model and its ability to learn when properly trained, indicate its potential to be used, in real time, on a mobile device to pass the rhythm of forró music to deaf and hard of hearing (D/HH) people.
Downloads
References
WHO, “Deafness and hearing loss,” mar 2021. [Online]. Available: https: //www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss
D. C. Baynton, Forbidden signs: American culture and the campaign against sign language. University of Chicago Press, 1996.
M. A. L. Soares, A educação do surdo no Brasil. Editora Autores Associados, 2015.
R. M. d. Quadros and L. B. Karnopp, Língua de sinais brasileira: estudos lingüísticos. Artmed, 2007.
C. A. Padden and T. Humphries, Inside deaf culture. Harvard University Press, 2009.
B. P. Tucker, “Deaf culture, cochlear implants, and elective disability,” Hastings Center Report, vol. 28, no. 4, pp. 6–14, 1998.
P. Kushalnagar, C. J. Moreland, A. Simons, and T. Holcomb, “Communication barrier in family linked to increased risks for food insecurity among deaf people who use american sign language,” Public Health Nutrition, vol. 21, no. 5, pp. 912–916, 2018.
J. Fellinger, D. Holzinger, and R. Pollard, “Mental health of deaf people,” The Lancet, vol. 379, no. 9820, pp. 1037–1044, 2012.
M. L. Fox, T. G. James, and S. L. Barnett, “Suicidal behaviors and help-seeking attitudes among deaf and hard-of-hearing college students,” Suicide and Life-Threatening Behavior, vol. 50, no. 2, pp. 387–396, 2020.
R. A. Cooper and R. Cooper, “Rehabilitation engineering: A perspective on the past 40-years and thoughts for the future,” Medical Engineering and Physics, vol. 72, pp. 3–12, 2019.
G. Raiola, “Inclusion in sport dance and self perception,” Sport Science, vol. 8, no. 1, pp. 99–102, 2015.
M. R. Zitomer, “Children’s perceptions of disability in the context of elementary school dance education,” Revue phénEPS/PHEnex Journal, vol. 8, no. 2, 2016.
S. Koch, T. Kunz, S. Lykou, and R. Cruz, “Effects of dance movement therapy and dance on health-related psychological outcomes: A meta-analysis,” The Arts in Psychotherapy, vol. 41, no. 1, pp. 46–64, 2014.
S. C. Koch, R. F. F. Riege, K. Tisborn, J. Biondo, L. Martin, and A. Beelmann, “Effects of dance movement therapy and dance on health-related psychological outcomes. a meta-analysis update,” Frontiers in Psychology, vol. 10, p. 1806, 2019.
F. H. F. Wu and J. S. R. Jang, “A supervised learning method for tempo estimation of musical audio,” in 2014 22nd Mediterranean Conference on Control and Automation, 2014, pp. 599–604.
E. Van Dyck, D. Moelants, M. Demey, A. Deweppe, P. Coussement, and M. Leman, “The impact of the bass drum on human dance movement,” Music Perception, vol. 30, no. 4, pp. 349–359, 2013.
M. Ezawa, “Rhythm perception equipment for skin vibratory stimulation,” IEEE Engineering in Medicine and Biology Magazine, vol. 7, no. 3, pp. 30–34, 1988.
Y. Dong, J. Liu, Y. Chen, and W. Y. Lee, “Salsaasst: Beat counting system empowered by mobile devices to assist salsa dancers,” in 2017 IEEE 14th International Conference on Mobile Ad Hoc and Sensor Systems, 2017, pp. 81–89.
H. Florian, A. Mocanu, C. Vlasin, J. Machado, V. Carvalho, F. Soares, A. Astilean, and C. Avram, “Deaf people feeling music rhythm by using a sensing and actuating device,” Sensors and Actuators A: Physical, vol. 267, pp. 431–442, 2017.
P. Tranchant, M. M. Shiell, M. Giordano, A. Nadeau, I. Peretz, and R. J. Zatorre, “Feeling the beat: Bouncing synchronization to vibrotactile music in hearing and early deaf people,” Frontiers in Neuroscience, vol. 11, p. 507, 2017.
A. Sharp, B. A. Bacon, and F. Champoux, “Enhanced tactile identification of musical emotion in the deaf,” Experimental Brain Research, vol. 238, no. 5, pp. 1229–1236, 2020.
E. Frid, “Accessible digital musical instruments-a review of musical interfaces in inclusive music practice,” Multimodal Technologies and Interaction, vol. 3, no. 3, p. 57, 2019.
A. C. d. Q. Junior, E. C. Fontes, R. Dias, and C. M. Volp, “Caracterização do xote e do baião dançados no interior do estado de são paulo,” Movimento, vol. 15, no. 3, pp. 233–247, 2009.
A. D. P. D. Santos, L. M. Tang, L. Loke, and R. Martinez-Maldonado, “You are off the beat! is accelerometer data enough for measuring dance rhythm?” in ACM International Conference Proceeding Series, 2018.
L. F. Paiva, H. G. Lopes, L. B. Felix, and R. V. Neves, “Estimação do compasso musical do forró utilizando rede perceptron multicamadas,” in Anais do Congresso Brasileiro de Automática, vol. 2, no. 1, 2020.
C. W. Wu, C. Dittmar, C. Southall, R. Vogl, G. Widmer, J. Hockman, M. Muller, and A. Lerch, “A review of automatic drum transcription,” IEEE/ACM Transactions on Audio Speech and Language Processing, vol. 26, no. 9, pp. 1457–1483, 2018.
A. Eronen and A. Klapuri, “Music tempo estimation with k-nn regression,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 18, no. 1, pp. 50–57, 2010.
F. H. F. Wu and J. S. R. Jang, “A supervised learning method for tempo estimation of musical audio,” in 22nd Mediterranean Conference on Control and Automation, 2014, pp. 599–604.
E. Quinton, M. Sandler, and C. Harte, “Extraction of metrical structure from music recordings,” in 18th International Conference on Digital Audio Effects, 2015, pp. 1–7.
S. Böck, F. Krebs, and G. Widmer, “Accurate tempo estimation based on recurrent neural networks and resonating comb filters,” in 16th International Society for Music Information Retrieval Conference, 2015, pp. 625–631.
H. Schreiber and M. Müller, “A post-processing procedure for improving music tempo estimates using supervised learning,” in 18th International
Society for Music Information Retrieval Conference, 2017, pp. 235–242.
S. Gulati, V. Rao, and P. Rao, “Meter detection from audio for indian music,” in Speech, Sound and Music Processing: Embracing Research in India, vol. 7172, 2012, pp. 34–43.
C. Uhle and J. Herre, “Estimation of tempo, micro time and time signature from percussive music,” in Proc. of the 6th Int. Conference on Digital Audio Effects, 2003, pp. 1–6.
A. C. d. Q. Junior and C. M. Volp, “Forró universitário: a tradução do forró nordestino no sudeste brasileiro,” Motriz, vol. 11, no. 2, pp. 127–120, 2005.
J. Roth, J. Ehlers, C. Getschmann, and F. Echtler, “Tempowatch: A wearable music control interface for dance instructors,” in Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction, 2021.
I. N. d. Silva, D. H. Spatti, and R. A. Flauzino, Redes Neurais Artificiais para Engenharia e Ciências Aplicadas, 2nd ed. São Paulo: Artliber, 2016.
