Researchers at Sony’s Computer Science Laboratory in Paris have created a pair of songs using Flow Machines, an Artificial Intelligence system that learns music styles from a huge database of songs. Using combinations of “style transfer, optimization and interaction techniques”, Flow Machines composes novel songs in selected styles, with vocals.
The song "Daddy's Car" below is composed in the style of The Beatles.
The next song "Mister Shadow" is composed in the style of “American songwriters”, such as Irving Berlin, Duke Ellington, George Gershwin and Cole Porter.
French composer Benoît Carré arranged and produced both songs, and wrote the lyrics. The full album is set to be released in 2017.
How is it done? What did the machines do, and what did the artist?
- Researchers set up a database called LSDB. It contains about 13,000 leadsheets from a lot of different styles and composers (mainly jazz and pop about also a lot of Brazilian, Broadway and other music styles).
- The human composer (in this case Benoît Carré) selected a style and generated a leadsheet (melody + harmony) with a system called FlowComposer.
- With yet another system called Rechord, the human musician matched some audio chunks from audio recordings of other songs to the generated leadsheets.
- Then the human musician finished the production and mixing.
Those who understand music will comprehend this better than I do.
Images and videos above from www.flow-machines.com
Google Magenta
Meanwhile, Google is also busy exploring AI for music synthesis. Google Brain’s Project Magenta team used Tensorflow, an open-source machine-learning and neural network engine built by Google, to create a 90-second piano melody. The system was given 4 notes to start off and it developed a piano tune with melody, a harmony and a beat. The drums and orchestration were not generated by the software, but added manually.
A Neural Network is a machine learning method, inspired by biological neural networks, that learns from data . Neural networks been used to solve a wide variety of tasks, like computer vision and speech recognition, that are hard to solve using ordinary rule-based programming. (I do not have the space here to provide a tutorial on Neural Networks; perhaps in a future diary).
The Magenta project is ongoing, and the team will continue to present the public with the AI engine’s artistic creations as it self-learns them.
What do you think?
It is quite obvious that these examples are not music you would listen to more than once (assuming you can listen through all of it to begin with).
But these are early steps in the human exploration of learning how to teach computers about music composition. It will get better. We have better software tools in the area of machine learning. But will it ever achieve the level required to create “real” music? Even if it does not, will it serve a purpose in creating lower-quality music, suitable in some areas like small videos, elevator music or even movie background scores? Even if not, will it help create new technologies and techniques for making computers “smarter”?
Will these efforts lead to tools to that serve as composer’s assistants? Or be beneficial as educational tools?
Or is it just an academic exercise, doomed to failure?