Get best of the week

What neural networks can “sing” and perform death metal

Let's talk about intelligent tools that can generate tracks and even lyrics. It will be about the decisions of corporations and laboratories, as well as the development of enthusiasts.


Photo Joe Green / Unsplash

Neural networks write music ...


Does this synthesizer NSynth Super . It is built on the basis of the AI ​​system, which forms new sounds that do not exist in nature from pre-recorded samples. The algorithm can combine the sound of a flute and a drum . NSynth can work with 16 musical instruments - on their basis it generates more than 100 thousand sounds. He analyzes their input characteristics, and then linearly interpolates, forming a mathematical representation.

NSynth Super is an open source project. Sources and build schemes are available to everyone in the repository on GitHub .

Another example is the Dadabots artificial intelligence system , which was developed by musicians CJ Carr and Zack Zukowski. The neural network composes death metal tunes - it was trained on the work of the Canadian team Archspire . The solution based on AI generates quite harmonious, although not always pleasant for hearing compositions - they are periodically superimposed on sharp acoustic effects. Although for the chosen style it looks like a wave organically. You can listen to the work of Dadabots on YouTube, there is a round-the-clock live broadcast .

AI music systems are also being developed at Jukedeck . This startup is developing a tool for generating trackswith a given mood and pace. A year ago, it was acquired by a company that owns TikTok. Jukedeck technologies help social networks save on royalties.

... and know how to sing


At the end of April, such a tool was introduced in OpenAI, it was called Jukebox . It generates compositions with meaningful lyrics and vocals. Here is an example:


Engineers trained a neural network on a data set of 1.2 million songs (600 thousand were in English). The text and metadata for them was taken from the LyricWiki library. To generate new tracks, the AI ​​system uses the VQ-VAE (Vector Quantized Variational AutoEncoder) method - it compresses the tracks and extracts the necessary acoustic information from them. Then on its basis forms a new composition. Jukebox spends about nine hours to write one minute of a song, but so far does not know how to generate familiar songs with repeating chorus. The system also requires large computing resources - it is not yet possible to test it at home on a computer or in a studio. In the future, developers plan to fix these shortcomings.

But will musicians be replaced


The authors of intelligent instruments for generating music say that machine algorithms are not intended to replace composers, but to expand their artistic capabilities.

American singer Taryn Southern recorded an album using an AI-based solution. The neural network wrote the music for the Break Free track and generated a video clip. YACHT lead singer Claire Evans also used machine algorithms to write Chain Tripping . The computer generated new melodies based on Claire's previous works, and the performer connected the most interesting samples to each other.

Machine learning algorithms also help musicians solve technical issues. For example, developers from LANDR offer an artificial intelligence system that automatically tracks tracks. More than 2 million performers are already using it .


Additional reading:

“Machine sound”: synthesizers based on neural networks.
History of speech synthesizers: first mechanical installations.
History of speech synthesizers: computer era.


What to read at us on Habré:


More articles:


All Articles