Music generation and human voice conversion based on LSTM

• Main Authors: Li Guangwei, Ding Shuxue, Li Yujie, Zhang Kangkang
• Format: Article
• Language: English
• Published: EDP Sciences 2021-01-01
• Series: MATEC Web of Conferences
• Online Access: https://www.matec-conferences.org/articles/matecconf/pdf/2021/05/matecconf_cscns20_06015.pdf

Music is closely related to human life and is an important way for people to express their feelings in life. Deep neural networks have played a significant role in the field of music processing. There are many different neural network models to implement deep learning for audio processing. For general neural networks, there are problems such as complex operation and slow computing speed. In this paper, we introduce Long Short-Term Memory (LSTM), which is a circulating neural network, to realize end-to-end training. The network structure is simple and can generate better audio sequences after the training model. After music generation, human voice conversion is important for music understanding and inserting lyrics to pure music. We propose the audio segmentation technology for segmenting the fixed length of the human voice. Different notes are classified through piano music without considering the scale and are correlated with the different human voices we get. Finally, through the transformation, we can express the generated piano music through the output of the human voice. Experimental results demonstrate that the proposed scheme can successfully obtain a human voice from pure piano Music generated by LSTM.