Trying to run GAN networks in OpenVINO

The OpenVINO library's Open Model Zoo model repository contains many different kinds of deep neural networks from the field of computer vision (and not only). But we have not yet met GAN models that would generate new data from noise. In this article, we will create such a model in Keras and run it in OpenVINO.

Quite a bit about GAN networks

Generatively competitive networks (GANs) with good training allow you to create new images from noise or input images that will be perceived as real rather than synthesized. GAN networks are increasingly used in various tasks:

• drawing up a picture description;
• picture generation by description;
• creating emoji from photography;
• increase image resolution;
• image style transfer;
• improving the quality of medical images;
• face generation and much, much more.

But first, let's practice on cats numbers to make sure that OpenVINO is capable of GAN networking.

GAN Image Generation

Keras, , MNIST. OpenVINO , GAN . , NVIDIA .

Keras : . GAN Keras .

, OpenVINO, OpenVINO .
5 .

GAN , , Fashion MNIST — , — . , ?

OpenVINO

OpenVINO : Caffe, Tensorflow, ONNX .. Keras, ONNX, ONNX OpenVINO. , , Keras->ONNX->OpenVINO , Keras->TensorFlow->OpenVINO. , ONNX , Tensorflow, .

Python Keras ONNX:

import numpy as np
import argparse
import onnx
import keras2onnx
from keras.models import load_model

model = load_model('gan_digits.h5')
onnx_model = keras2onnx.convert_keras(model, model.name)
onnx.save_model(onnx_model, 'gan_digits.onnx')

ONNX OpenVINO ( Windows) Model Optimizer:

python "C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer\mo.py" --input_model gan_digits.onnx --input_shape [100,100]

, , OpenVINO. :

import numpy as np
import sys
import matplotlib.pyplot as plt
from openvino.inference_engine import IENetwork, IECore

#  OpenVINO   
ie = IECore()
net = IENetwork(model='gan_digits_R2020.1.xml', weights='gan_digits_R2020.1.bin')
exec_net = ie.load_network(net, 'CPU')
input_blob = next(iter(net.inputs))
out_blob = next(iter(net.outputs))

#      
noise = np.random.normal(loc=0, scale=1, size=[100, 100])
generated_images = exec_net.infer(inputs={input_blob: noise})

#  
generated_images = generated_images['Tanh']
generated_images = generated_images.reshape(100, 28, 28)
figsize = (10, 10)
dim = (10, 10)
plt.figure(figsize=figsize)
for i in range(generated_images.shape[0]):
    plt.subplot(dim[0], dim[1], i + 1)
    plt.imshow(generated_images[i], interpolation='nearest', cmap='gray_r')
    plt.axis('off')
plt.tight_layout()
plt.show()

, . , , 100%.

OpenVINO . , , OpenVINO , .

, , styleGAN. :

styleGAN NVIDIA , . GitHub , GPU. OpenVINO.

, PyTorch ONNX ( OpenVINO). , , !

, , .
.