Cybercalmar neuroevolution

Evolving neural network

Artificial neural networks mimic real biological nervous systems. They contain neurons and connections between them, providing the conversion of incoming signals into a significant output result. In the field of machine learning, these networks are often initialized with random connections between neurons, after which the network learns until it starts behaving in the right way. This approach is quite applicable, however, animals have many simple nervous systems that work out of the box: no one teaches fish to swim or butterflies to fly, despite the fact that their behavior is created by networks of neurons. Their nervous systems are not the result of random initialization and subsequent training, but evolution. After many generations, nature has created a pattern of cells and connections that provides complex and successful behavior.

To create neural networks that provide behavior without learning, you can use neuroevolution . Evolutionary algorithms (such as the one I used to perform plant evolution ) subject the genetic code to evolution over a long period of time. The genetic code (the model for DNA) and the organism it represents are initially very simple, but for generations small mutations increase the favorable complexity and add functions that stimulate the further spread of these properties.

Digital squid

To demonstrate the effect of neuroevolution, I want to expose the evolution of digital squid. Squids have the following properties:


Figure 1: floating squid.

• They can have any number of tentacles of different lengths.
• Each hand is controlled by one output neuron, pumping it in one direction with a low output signal and in the other with a high output signal.
• Squids have a head ; the size of the head determines the maximum possible number of neurons.
• Squid mass is determined by the size of the head and the number of tentacle segments.
• Squids float in a simulated fluid filled with dots denoting food. Touching these points, the squid “eats” them, and the squid score is calculated as the number of points eaten, divided by its mass.

These properties, thanks to evolution, should create squids that effectively swim in the environment, eating as much food as possible. Since they also have mass, squid bodies must be effective: heavy bodies and large tentacles require more food in order to have a reason for their evolutionary creation.

Since squid can have varying properties (for example, head size and tentacle configuration), these properties also evolve. Squid DNA contains not only a “drawing” of his brain, but also a plan of the body.

Figure 1 shows a simulated squid with two hands. Oscillating movement of the hands is ensured by the addition of spring force to the arm segments; if the muscles stop moving the arms, the segments will gradually align until the arms turn into straight lines. To calculate the magnitude of the acceleration, all lateral movement is summed. All the lateral movement shown in the figure is added to this number. Swinging tentacles behind the body, the squid provides forward movement.

Spike neural networks

Choosing the right neural network for our project is not an easy task. There are many different types of neural networks; in this article contains an easy overview of some of them. The task of squid management in certain aspects is unlike the work of well-known networks:

• «-». , . - , .
• . «-», .
• , . , .

These requirements are met by a spike (pulse) neural network . Such networks work in real time, they were designed to more accurately simulate wildlife. Like real neurons, cells in a spike neural network accumulate potential, gradually collected from all input signals, and in the absence of input signals gradually return to their “neutral” state. Spike neural networks are not required to adhere to a strict wiring diagram. They consist of a layer of input neurons, a layer of output neurons (in our case, controlling tentacles) and several neurons between them, called a hidden layer. The neurons in the hidden layer can be connected to the input and output neurons, but the neurons can also be disconnected. In our task, hidden layer neurons can be connected to each other.


Figure 2: logistic function.

All neurons of the nervous system have an activation function . This function determines the output value of a neuron based on the sum of its input signals. Neurons are connected by axons connecting the source and target neurons. The axon adds to the activation of the target neuron the output value of the original neuron multiplied by the weight of the axon (which can be a positive or negative number). Then the activation function determines the output value of the neuron based on its activation. For this simulation, we selected the logistic function as the activation function :

$$

$$\frac{1}{1+e^{-a}}$$

In this equation $$$a$Is the activation of a neuron. Figure 2 shows a graph of the function. At$$$a=0$ output value is $$$0.5$; in our simulation, this is useful because the network must be able to provide behavior even in the absence of an input value. If the default output value is not equal to zero, then some signals will constantly flow through the system. Theoretically meaning$$$a$ may be very small or very large, but the asymptotes of the logistic function ensure that the output value will always be in the range $$$[0, 1]$. Because of this, extreme system output values ​​are not propagated.

Simulation of evolution

The simulation environment consists of the following components:

• Any number of squids with different body structures and spike neural networks to control their limbs.
• Food scattered around the environment.

To begin to simulate evolution, a fixed lifetime of each generation is simulated in the system. In my simulations, I chose segments from 20 to 30 seconds. After this time, the most productive squid is selected, which is duplicated several times to create a new generation of squid. Before performing the next simulation, all squids undergo a slight mutation. The following properties can mutate:

• The radius of the body, and with it the maximum allowable number of neurons in the squid brain.
• Number and location of tentacles.
• The length of the tentacles.
• The number of neurons in the brain.
• Axon connections between neurons (connections can appear or disappear, the weights of bonds can also change).

At the moment, the squid brain does not contain input neurons. The number of output neurons is always equal to the number of tentacles, and one output neuron is assigned to each tentacle. If the tentacle disappears during the mutation, then the output neuron corresponding to it is also removed. When a new tentacle appears during mutation, it receives a new output neuron with random connections.

The source code for the simulation is uploaded to GitHub , and the simulation is done in the browser .

results

Figure 3: The nervous system creating endless impulses.

When performing a simulation, moving squids are usually obtained in several hundred generations. When a working swimming strategy arises evolutionarily, it usually evolves over time into the most optimal version.

The nervous system can be visualized. Figure 3 shows a simple squid nervous system with two arms. The network contains five neurons, indicated by orange circles, and two output neurons, indicated by blue circles. When the output signal of a neuron increases, the neurons become brighter. When the output signal decreases, the circles become transparent. Axons are visualized by dashed lines connecting the corresponding neurons. When the axon transmits a signal (and affects the target neuron), the dashed line becomes more visible, and the dashed line moves in the direction of the signal.


Figure 4: Evolved floating squids. Note that not all agents are particularly effective; some contain useless mutations.

Figure 4 shows several squids floating in a simulation environment. These squid use two simultaneously swaying tentacles. Among the agents, variability is noticeable:

• There are various lengths of tentacles, but their patterns of movement are approximately the same.
• The two agents at the bottom of the image have evolutionarily received additional tentacles, which are not yet particularly useful. They increase the mass of squid, thus reducing its score.

Conclusion

Simulation in its current state demonstrates the effectiveness of neuroevolution and forms the foundation for the further development of the system:

• . , .
• . , .
• , , .
• . . , . , .

These additions will not require any changes in the basic mechanism of neuroevolution, they will simply allow more diverse strategies to appear. The universality and adaptability of the shown framework of neuroevolution make it an interesting tool for many other areas, in particular, studies of artificial life .