By Jean-Pierre Briot and François Pachet.
Neural Computing and Applications 32(4), 981–993 (2020). First published online in 2018. DOI: 10.1007/s00521-018-3813-6.
Read the preprint (PDF) · Publisher version · Related survey
About the paper
The paper examines four challenges in using deep learning to generate music: controlling what the system produces, giving the music structure, encouraging novelty, and letting musicians interact with the system as they compose. It describes approaches to each challenge and illustrates them with existing systems.
Abstract
In addition to traditional tasks such as prediction, classification and translation, deep learning is receiving growing attention as an approach for music generation, as witnessed by recent research groups such as Magenta at Google and CTRL (Creator Technology Research Lab) at Spotify. The motivation is in using the capacity of deep learning architectures and training techniques to automatically learn musical styles from arbitrary musical corpora and then to generate samples from the estimated distribution. However, a direct application of deep learning to generate content rapidly reaches limits as the generated content tends to mimic the training set without exhibiting true creativity. Moreover, deep learning architectures do not offer direct ways for controlling generation (e.g., imposing some tonality or other arbitrary constraints). Furthermore, deep learning architectures alone are autistic automata which generate music autonomously without human user interaction, far from the objective of interactively assisting musicians to compose and refine music. Issues such as control, structure, creativity and interactivity are the focus of our analysis. In this paper, we select some limitations of a direct application of deep learning to music generation and analyze why the issues are not fulfilled and how to address them by possible approaches. Various examples of recent systems are cited as examples of promising directions.
Related work
For a broader comparison of music generation systems by objective, representation, architecture, challenge, and strategy, see Deep Learning Techniques for Music Generation — A Survey.