How we created the gallery of neural network art and why we do not give pictures to copy

Today we launched a virtual gallery where all the pictures are created by a neural network. Its peculiarity is that only one person can take each picture in full size. Almost like in a real gallery.

In this post, I will talk about how this idea was born and how we implemented it using two neural networks, one of which is used in Yandex search.



Idea

We experiment a lot with GANs and tried to find the idea of ​​a beautiful and understandable project in which we could show our achievements.

I already had a ganarts project that I launched for myself and friends. It was just a page on which pictures in the form of a print on a t-shirt were endlessly generated. They were generated by the StyleGAN neural network, trained on hand-picked art styles with wikiart.

Project Gif

We took this idea as a basis. But I didn’t want to do just another site-generator of pictures. And in the process of discussion, the concept of an art gallery with several thematic rooms appeared, where each picture should have no more than one owner. According to our idea, this should connect the virtual gallery with the more familiar galleries in which each picture has a specific owner. At the same time, being able to generate millions of images, we specifically limited their number so that each user who managed to pick up a picture felt its uniqueness. And we also added a restriction - you can take no more than one picture - so it’s much more interesting to choose.

Implementation

Since the Nvidia company posted the code for training the StyleGAN neural network, it is quite difficult to surprise someone with the generated images. Its authors managed to make a fairly universal architecture, which shows good results on different data. A breakthrough was also in the fact that the model could be trained at a sufficiently high resolution (1024 × 1024) for an acceptable time and with better quality than that of competitors.

Enthusiasts "fed" to her everything that comes to hand. If you are interested in looking at such projects, here is a list of the most striking ones.

- People: thispersondoesnotexist.com
- Anime: www.thiswaifudoesnotexist.net
- Cats: thesecatsdonotexist.com
- Game of Thrones characters: nanonets.com/blog/stylegan-got
- Cars: twitter.com/SyntopiaDK/status/1094337819659644928
- Logos: twitter.com/matthewjarvisw/status/1110548997729452035
- Children's drawings: twitter.com/roberttwomey/status/1239050186120282113
- Bugs: twitter.com/karim_douieb/status/1229903297378766854
- comics Garfield: twitter.com/willynguen/status/1220382062554898433
- Fonts: twitter.com/cyrildiagne/status/1095603397179396098
- Images from the microscope: twitter.com/MichaelFriese10/status/1229453681516412928
- Pokemon: twitter.com/MichaelFriese10/status/ 1210305621121064960

But progress does not stand still, and at the end of 2019 Nvidia released the second versionStyleGAN . A detailed overview of all changes can be read on Habré . The main visible improvement is getting rid of characteristic droplet-like artifacts by changing the method of normalizing activations within the network. Having tried the new architecture on our initial dataset, we also noticed an increase in the variety of generated paintings, which could not but rejoice. Our hypothesis: this was due to an increase in the number of parameters in the second version of the architecture, which allowed us to learn more “modes” in the distribution of data.

Another nice bonus of the second version of StyleGAN was the “smoother” latent space. Simply put, this allows you to make smooth flowing between different pictures:


To create the paintings, we used the StyleGAN2 architecture. We taught the neural network on works belonging to different areas of painting: from Fauvism and Cubism to minimalism and street art. In total, there are about 40 thousand paintings in the training set, on the basis of which completely new images are generated.

Our gallery has four thematic halls: "People", "Nature", "City" and "Mood". In order to divide the paintings into halls, we used the neural network, which is used in Yandex.Pictures. She learns on user clicks from issuing pictures on a text request. Our internal training dataset is large enough so that this neural network can figure out even the generated patterns. She helped to automatically select images according to our textual requests from a huge number of random pictures. For example, for a room called “People,” paintings were selected at the request of “beauty,” “round dance,” “date,” “master,” and so on. This created a sense of thematic rooms, and the initial queries served as the names for the paintings.

Result



We made a galleryof four thousand paintings, each of which can find its owner. With this virtual gallery, we want to further reduce the distance between neural network creativity and more tangible art.

I believe that the process of selecting a picture by the user can also be called a manifestation of creativity. And I hope that in the future there will be much more opportunities for creative choice.