🥃 ⛹🏻 ✉️ The book “Generative deep learning. The creative potential of neural networks ” ✋ 🎱 🚷

Hello, habrozhiteli! Generative modeling is one of the most discussed topics in the field of artificial intelligence. Machines can be taught to draw, write and compose music. You yourself can put artificial intelligence at your desk or an easel, for this it is enough to get acquainted with the most relevant examples of generative deep learning models: variational auto-encoders, generative-competitive networks, models such as encoder-decoder and much more.

David Foster makes the architecture and methods of generative modeling understandable and accessible, his tips and tricks will make your models more creative and effective in training. You will start with the basics of in-depth training based on Keras, and then move on to the most advanced algorithms.

Understand how variation auto-encoders change emotions in photographs.
Create a GAN from scratch.
Learn how to work with generative text generation models.
Discover how generative models help agents complete reinforced learning tasks.
Explore BERT, GPT-2, ProGAN, StyleGAN, and more.

Goals and approaches

This book discusses the key methods that have dominated the landscape of generative modeling in recent years and have made impressive progress in creative tasks. In addition to acquaintance with the basic theory of generative modeling, in this book we will create working examples of some key models borrowed from the literature, and step by step we will consider the implementation of each of them.

Throughout the book, you will come across short instructive stories explaining the mechanics of some models. Perhaps one of the best ways to study a new abstract theory is to first transform it into something less abstract, such as a story, and only then plunge into a technical description. Separate sections of the theory will be more understandable in a context that includes people, actions, and emotions, and not in the context of such rather abstract concepts as, say, neural networks, backpropagation, or loss functions.

The story and description of the model is a common method of explaining the same thing from two points of view. Therefore, when studying a model, it will sometimes be useful to return to the corresponding story. If you are already familiar with a particular trick, then just have fun discovering parallels in each story with each element of the model!

The first part of the book presents key methods for constructing generative models, including an overview of deep learning, variational autocoders and generative-competitive networks. In the second part, these methods are used to solve several creative problems (drawing, composing stories and music) using models such as CycleGAN, models such as codec and decoder and MuseGAN. We will see how generative modeling can be used to optimize the winning strategy of the game (World Models), consider the most advanced generative architectures available today: StyleGAN, BigGAN, BERT, GPT-2 and MuseNet.

Naive Bayes Parametric Model

The naive Bayesian parametric model uses a simple assumption to drastically reduce the number of parameters to evaluate. We naively assume that each characteristic xj does not depend on any other characteristic

With respect to the data set obtained at Irma, this means, for example, that the choice of hair color does not affect the choice of type of clothing, and the choice of type of glasses does not affect the choice of hairstyle. More formally, for all signs

This is known as a naive Bayesian guess. To apply it, we first use the chain probability rule to write the density function as a product of conditional probabilities:

where K is the total number of signs (that is, five in the example with the planet Irm).

Now apply the naive Bayesian assumption to simplify the last line:

This is a naive Bayesian model. The task is to evaluate the parameters

for each feature individually and multiply them to determine the probability of any possible combination.

How many parameters need to be evaluated in our task? For each characteristic, you need to evaluate the parameter for each value that this characteristic can take. Therefore, in the example with the planet Irm, this model is determined by only 7 + 6 + 3 + 4 + 8 - 5 = 23 parameters.

The maximum likelihood

score is calculated as

where

is the number of times when the attribute k takes the value l in the data set, and N = 50 is the total number of observations.

In the table. 1.2 shows the calculated parameters for a dataset from the planet Irm. To find the probability with which the model will generate some observation x, it is enough to multiply the probabilities of individual features. For instance:

Please note: this combination is absent in the original data set, but our model determines a nonzero probability for it, which means it can generate it. In addition, the probability of this combination is higher than, for example, (long haircut, straight hair; redhead; round glasses; crew-neck t-shirt; blue01), because the white color of clothes appears in the observation set more often than blue.

That is, the naive Bayesian model is able to identify some data structure and use it to create new samples that are not in the original set. The model estimated the probability of meeting each attribute value independently of the others, therefore, using the naive Bayesian assumption, these probabilities can be multiplied to construct the full density function,

In fig. 1.8 shows 10 observations selected by the model.

For this simple task, the naive Bayesian assumption of independence of attributes is reasonable and therefore provides a good generative model.

Now let's see what happens if this assumption turns out to be erroneous.

Hi Irm! Continuation

You feel a certain sense of pride when you look at ten new creations created by your naive Bayesian model. Encouraged by your success, you pay attention to the other side of the task, and this time it does not look as simple.

The data set with the straightforward name Planet Pixel that was provided to you does not contain the five high-level features that you saw above (hair color, type of accessory, etc.), but only the 32 × 32 pixels that make up each image. That is, each observation now has 32 × 32 = 1024 signs and each sign can take any of 256 values (individual colors in the palette).

Images from the new dataset are shown in Fig. 1.9, and a sample of pixel values for the first ten observations is shown in Table. 1.3.

You decide to try the naive Bayesian model again, this time trained on a pixel data set. The model will evaluate the maximum likelihood parameters that determine the color distribution of each pixel in order to generate new observations based on this distribution. However, after completing the model, you understand that something went wrong. Instead of new fashion samples, the model brought out ten similar images, on which neither accessories nor clear signs of a hairstyle or clothing can be distinguished (Fig. 1.10). Why did it happen so?

about the author

David Foster is a co-founder of Applied Data Science, a data consulting company that develops customized solutions for customers. He received a master's degree in mathematics from Trinity College, Cambridge, UK, and a master's degree in operational research from the University of Warwick.

Won several international machine learning contests, including the InnoCentive Predicting Product Purchase. He was awarded the first visualization award that allowed a pharmaceutical company in the United States to optimize the choice of location for clinical trials.

An active member of online communities interested in data science and the author of several successful deep learning blog articles, including"How To Build Your Own AlphaZero AI Using Python and Keras . "

»More information about the book can be found on the publisher’s website
» Contents
» Excerpt

For Khabrozhiteley 25% discount on the coupon - Deep learning

Upon payment of the paper version of the book, an e-book is sent by e-mail.

The book “Generative deep learning. The creative potential of neural networks ”

Goals and approaches

Naive Bayes Parametric Model

Hi Irm! Continuation

about the author

More articles: