Hello everyone. It's no secret that almost all the articles on our blog are published to launch this or that course. The next article could be timed to the launch of the course "Neural Networks in Python", but given the simplicity of the material, I do not want to associate it with the course, but simply post it as a bonus material, as a small tutorial for the smallest. Predicting questions, I want to immediately say that this article is not related to the course and was not written by the teacher. If you have a desire to learn more about the course, then you can do this on the corresponding page .
Hello everyone! Today I want to talk about how the neural networks are arranged, but it is so simple that even beginners who have mastered only the very basics of Python can understand.What are neural networks and artificial intelligence in general?
In the philosophy of AI, there are two types of artificial intelligence - strong and weak. The theory of strong artificial intelligence suggests that computers will be able to gain the ability to think and be aware of themselves as a separate person (well, something like what happened in Detroit Become Human). The theory of weak artificial intelligence suggests that such an intellect cannot be or is very difficult to construct, and while science is able to create neural networks that only partially repeat the work of neural networks of living creatures. But philosophy just does not apply to science because nothing can be proved there, so we will not focus on this. Weak neural networks now occupy a dominant position in Data Science and are widely used: in recognition of visual images, smart purchasing of goods,even the vehicle’s OBS system - so starting to learn AI is now more than relevant.What are neural networks?
Neural networks are mathematical models of the operation of real neural networks of living things. It’s enough to simply transfer the mathematical model into a program (and because of this, Python is so widely used in programming neural networks, so convenient for programming solutions to mathematical problems. In fact, neural networks can be written in almost any programming language that supports any kind of mathematics Even on Scratch, which was originally created to teach the basics of programming for younger students. You can see here ).Basic concepts of neural networks
There are many algorithms for the operation of neural networks (and now the mathematical side of this issue is being actively developed). The classic solution for beginners is the backpropagation method, a gradient calculation method used to update the weights of a multilayer perceptron. In the form in which it is usually studied by beginners (with a sigmoid activation function), the neural network is quite slow, but relatively accurate.The program we are going to write is called a huge stretch of the neural network.Before proceeding to its description, let's discuss what neural networks do in general.What are neural networks doing, if simplified?
If we simplify the concept of neural networks a little, then a neural network that is taught according to the principle of learning with a teacher, after training on the principle of “stimulus - reaction”, indicating the correct answers, can work with strangersdata. In other words, if you proposed a certain set of words to the input neural network (for example, a set of reviews on film search, positive and negative, in any format, at least txt, at least json, the question is only in the program for processing this data). To successfully create a neural network, you will need two data sets: a test set with which you can evaluate the performance of the created neural network and a training set in which the data is labeled positive / negative for it (and here the problem of classifying big data arises because it is a long and dreary occupation). After training the neural network (which can take a lot of time and computer resources, depending on the dimension of the data, its processing, and most often the most important, the algorithms used), it will be able to tryto predict with some accuracy a positive or negative review came to her input. But neural networks (as well as a real person) have a certain percentage of error. The task of optimization is to make it minimal, but the question of assessing the quality of neural networks is likely to never go anywhere. For example, they give you a photo of a hot dog and you say for sure that it is a hot dog. But what if the photo is blurry? Black and white? Filmed in poor visibility? Here you can already affirm with only a certain degree of probability, even if you have prepared or eaten a lot of hot dogs in your life.
Getting started programming our toy neural network
OK, let's go. There will be no test and training data in our toy neural network, our toy neural network will try to find the ratio of the ratio between any data.Stop. And what's the point? The same is found in one simple mathematical expression.Without doubt. However, I have now taken this expression to show the process of learning a neural network. Suppose we are faced with the task of finding out what is the conversion factor between inches and centimeters. How many centimeters does one inch take? For a person who knows mathematics at least fifth grade (or even earlier), it is not difficult to remember what translation coefficient is 2.54.But for now, we will forget about this for a while, and imagine that we need to create a simple algorithm that will universally calculate this parameter. However, the catch is also that neural networks are not some constants with ready-made coefficient values, otherwise they would not have “live” training.Total, we are in the position of a child who just sat down in front of a set of cubes and is going to take them into his own hands for the first time and build the first tower in his life. He only roughly knows how the physics of objects works; he, just like we, knows that a certain specific ratio exists (in his case, it is gravity). So what will the child do?He will take and at random put some dice. In the same way, we can only guess at random what coefficient we will have (and real adult neural networks do this too, only usually guided by the generation of numbers in a normal distribution).Just at random, suppose that the coupling coefficient of inches and centimeters (let's start to call it weight , as in adult neural networks) will be, for example, 2.4. Then we get the most complicated mathematical expression:1 * 2.4 = 2.4
Great, we almost guessed, and we have some result. But it is incorrect, and what is normal for the learning process, we have some error. As in the nervous system with feedback, we need to somehow respond to the error. But first you need to understand its size. As I have already said, in training neural networks with a teacher, the data is first run on marked-up data and only then sent to the classification for similar, but not marked-up data. We also know what we should get the correct result, accordingly, I can calculate the error:t_target = 2.54
t_now = 2.40
e = t_target - t_now
// , 0.14
Now we know how wrong. But what to do? Naturally, our toy neural network should read data as low as possible. And this is another underwater feature of neural networks that manifests itself - most often they have some loss factor during training, the part of Data Science, called optimization , is engaged in minimizing the error .But now is not about that. Without a doubt, we need to start changing the coupling coefficient by some step, but by what? Obviously, you need not be too big, otherwise our coupling coefficient will turn out to be inaccurate and not too small, otherwise we will have to train the neural network for a rather long time. There is no 100% correct option for finding this step, most often these parameters in a real neural network are selected more intuitively than on the basis of some formula or algorithm (although this is also possible). Now we can randomly select the value of our step (in the language of neural networks, our step is called learning_rate), for example, the value 0.05 will work optimally enough. In addition, you need to agree on how many times the indentation at the learning rate will occur.. The number of these indents will be called epochs, as in adult neural networks. Now, armed with all this knowledge, you can try to write a small Python program that will run our toy neural network program.import random
inches = 40
centimetre = 101.6
def kid_neuro(epoch, lr, accur):
W_coef = random.uniform(0, 2)
print(" : ", W_coef)
for i in range(epoch):
Error = centimetre - (inches * W_coef)
print(" ", Error)
if Error > 0:
W_coef += lr
if Error < 0:
W_coef -= lr
if Error < accur:
print(" ", W_coef)
return
epoch = int(input("epoch: "))
lr = float(input("enter learning rate: "))
accur = float(input("enter accurancy: "))
kid_neuro(epoch, lr, accur)
I leave the reader to try to run this child’s neural network with various parameters on its own. It turns out pretty well at epoch = 100-, learning rate = 0.01, accur = 0.1.Despite the seeming uselessness of this program, we have discussed with you the work and basic concepts of neural networks, which are also used in the construction of real large neural networks, for example, in the backprogation algorithm.Briefly, these basic concepts:- W — . - , . — , ,
- lr — learning rate, .
- epoch ,
As a practice, you can try to write your own children's neural network, which will translate, for example, kilometers into miles. And armed with the knowledge gained in this article, you can easily come here , for example , and already try to start the neural network more meaningfully.Some useful links that you can go to if you want to continue learning about neural networks: