Get best of the week

Brain prosthesis: synchronization of artificial and biological neural networks



The concept of prosthetics, i.e. an attempt to replace the missing part of the body with an artificial analogue has existed for a very long time. The first mention of prosthetics can be found in records dating back after 1500 BC. And this is not surprising, since the simplest forms of prosthetics are really simple, and therefore could have been done artificially even in those days (remember the pirates with their hooks and wooden legs). However, prosthetics is not limited to apparently superficial health problems. We all know about artificial joints, vessels, valves, etc. But even these augmentations are nothing compared to prosthetics of a part of the brain, because the brain is the most complex organ of our body. Today we will meet with you a study in which scientists from the University of Tokyo found a way to get real neurons to work together with artificial ones.What technologies and techniques were involved in the development, how effective is the relationship between synthetic and biological, and what is the practical application of this discovery? A report by scientists will tell us about this. Go.


The human brain is regularly compared to a computer and this comparison is justified. Any disease or brain injury directly affects the structural and functional properties of brain networks and circuits, causing cell death, loss of synapses and axon loss. Such processes impair the ability of local information processing and its exchange between remote circuits, disrupting the process of segregation and integration of information in the brain. It is logical that such problems need to be addressed. Some methods are more focused on restoration - cell regeneration, while others are inclined to replace - prosthetics of the missing “highways”. The regenerative techniques being developed at the moment, quite successfully cope with defects at short distances. However, when it comes to losing communication between distant parts of the brain,they are powerless due to the complexity of reprogramming and reconstructing neural circuits.

If something doesn’t come out, then it needs to be replaced. According to scientists, in recent decades, impressive progress has been made in the implementation of neuro prosthetics, when artificial pulsed neural circuits are locally capable of receiving and processing input data in real time. In this case, the input data can be provided both locally and remotely; both by electrical and optogenetic stimulation.

There are a lot of variants of techniques in neuromorphic engineering for creating pulsed neural networks (SNNs from spiking neural network) and artificial synapses. Neuroinduced SNNs are very different from their biological progenitors, but are great for computing and developing artificial intelligence. But neuromimetic SNNs more successfully mimic the activity of real nerve cells and work on an accelerated or biological (real) time scale. The disadvantage of this SNN option is that it can be reproduced programmatically, but in reality it does not work. In contrast, there are hardware SNNs that work in real time, have low power consumption and are built-in. Such characteristics are the most attractive for creating a hybrid system, i.e. for neuro prosthetics.

Hardware SNH15-21 can be classified into two groups: analog implementation and digital implementation. Digital implementation has the advantage that it is customizable and easier to process, despite the higher power consumption.

It's all great, scientists say, but all of these SNN systems are nothing if they cannot work in tandem with real biological systems. To create a connection between the artificial and the biological has not yet succeeded.

In their report, the researchers demonstrate the first operational implementation of real-time communication and transmission of information from hardware SNN implemented on the FPGA and biological neural network (BNN) circuit board by dynamically encoding SNNs using patterns used for optogenetic BNN stimulation.

Activity patterns are generated using SNN, then they are encoded in real time into unique blue light patterns - binary images (8x8 pixels) generated using digital light processing (DLP) using a modified video projector with microprojection onto a two-dimensional neural network (culture) grown on a multi-electrode matrix (MEA).

The neurons used in the experiments were transduced * using adeno-associated virus * (AAV) to express the ChIEF27 protein.
Transduction * - the transfer of DNA from one cell to another.
Adeno-associated virus * is a small virus that infects human cells, but does not cause any disease, therefore, causes a weak immune response.
In view of this procedure, neurons were excited upon stimulation with blue light, and their activity was recorded using both an MEA device and calcium imaging (a CCD camera (CCD from a charge-coupled device ) with an electron multiplier was mounted on a microscope).

Research results



Image No. 1 The

experimental setup ( 1A ) consists of three main components located around an epi-fluorescence microscope.


Schematic of an epi-fluorescence microscope.

The main components of the installation:
  • pulsed neural network ( 1B ), operating on an FPGA (field-programmable gate array, i.e. user-programmable gate array);
  • SNN-FPGA ( 1C);
  • , ChIEF-mCitrine ( hSyn), .



Table No. 1

In order to simulate the activity of a real biological neural network, SNN generated spontaneous activity characterized by neural synchronizations with similar characteristics (in terms of duration, frequency and number of recruited neurons) that were generated using cortical BNNs (from 0.1 to 1 Hz).

Four different SNNs (table 1) used in 12 experiments were composed of 100 Izhikevich neurons (80 excitatory and 20 inhibitory) implemented in FPGA (table 2), and provided a dynamic range with network synchronization (NS), which spanned from 0.25 up to 1 Hz (table 3).


Table 2: FPGA resources. LUT and FF are the main components of logic blocks in FPGA; LUTRAM and BRAM - memory technology; DSP (Digital Signal Processing) - circuits used for complex digital computing, such as multiplication.


Table No. 3: experiment parameters. For each experiment, several variables are selected at once: one of four SNNs, threshold value (N) of neuron pulses, and duration (T) of time windows for calculating network synchronization.

SNN generated activity with a time resolution of 1 ms, and NSs were determined when at least N of 64 neurons generated an impulse in time interval B. Four SNNs represent different activity, since their neurons, synapses, and connectivity parameters changed in different experiments.

The spontaneous SNN activity was converted in real time into 8x8 pixel binary matrices, where each matrix element was equal to zero (i.e., without light) if its corresponding assigned impulse neurons did not start, or one (i.e., light emission ) if neurons are activated.

Once the NS was identified, the corresponding transformed image was illuminated. Further, based on the SNN activity, the network synchronization detector module created a transistor-transistor logic (TTL) signal to the stimulator device and formed the illumination of the VGA image corresponding to the 8x8 matrix.

As we have already guessed, one of the main parts of the installation is a video projector system. The 8x8 binary image generated as SNN output activity was converted to an 800x600 pixel image via the VGA port of the video projector, where instead of the original lamp, a powerful blue LED was used. The 8x8 binary matrix (0 = black, 1 = blue) was displayed in the central part of 800x600 pixels, and all other pixels were zero (black).

The image generated by the digital micromirror device (DMD) of the video projector was projected into an epi-fluorescence microscope through an additional optical path passing between the camera and the filter cube located above the sample ( 1C) The focus of the DMD image on the microscope port was optimized so that all generated images in the field of view of the microscope could be projected through 10x magnification with sufficient power to trigger action potentials in neurons expressing ChIEF. The projected DMD image was located in the focal plane of the lens with a focal length of 250 mm, which made it possible to enlarge the image by about fourteen times.

So, while there is an artificial neural network and a means of fixing data. The next integral component of this experiment is, of course, BNN, i.e. biological neural network.

For BNN, neural cultures from 21 to 28 DIV were used (days in vitro - days “in glass”, ie the number of days spent in vitro culture or in a petri dish). Neuronal activity was recorded using a standard (8x8) MEA * cup with an interelectrode distance of 200 μm ( 2A ).
MEA * (Microelectrode array) - a microelectrode array is a device in which there are several (from tens to thousands) microelectrodes through which neural signals are received or sent. MEA is a neural interface between neurons and electronic circuits.


Image No. 2

Excitatory and inhibitory neurons were transduced (under the hSyn promoter) at 7 DIV for expression of ChIEF-mCitrine ( 2B ). The expression rate in the entire population of neurons at 21 DIV was 70 ± 13%. The results of calcium imaging ( 2B ) were obtained by a 10-fold magnification with an EMCCD camera mounted on a microscope with a field of view of 800x800 μm, which is approximately equal to the space between 4x4 MEA electrodes ( 2A ).

The projected image (i.e., stimulus) was applied in the same field of vision, but on a slightly smaller area ( 2C ).

The time synchronization of the various devices was performed through the MEA data acquisition system, in which the signal from each of the 60 electrodes was simultaneously recorded on the TTL signal, activating the stimulation protocol, controlling the LED driver (turning on and off the blue light) and a single-frame signal received by the camera.

It should be noted that the MEA system detected BNN activity before, during and after the activation of stimulation (i.e., when the potential link between SNN and BNN was activated).


Images No. 3

During the study, 12 experiments were conducted, in each of which a specific version of SNN and other parameters were used to detect network synchronization (Table 3), which was supposed to increase the number of stimuli per minute of time.

Due to the change in parameters, SNN generated different output data (OUTPUTs) with different frequency ranges (measured as the interval between stimuli, 3A ) and intensity (100% stimulus intensity meant that all 64 squares of the 8x8 matrix were turned on, 3B ).

The frequency interval for network synchronization for SNN has been set [0.25; 1] Hz. Such an accurate choice of these values ​​made it possible to neutralize any overlap of stimulations, since the stimulation protocol lasts 310 ms, and cortical BNNs generate an average neuron synchronization between 0.1 and 1 Hz.

Information transfer (IT from information transmission) between SNN and BNN was quantified taking into account the correlation of similarity between INPUT pairs (SIP, 4A ) and similarity between OUTPUT pairs (SOP,4C ).


Image No. 4

The bottom line is that when information is transmitted, two similar INPUTs for the BNN should call two similar OUTPUT patterns in the BNN.

Next, information transmission was evaluated over 12 experiments with different parameters (linearity of the network response, average frequency / stimulation intensity, etc.; 5C - 5G ) and different metrics (entrainment coefficient BNN and coefficient of suppression of network synchronization, SIP were measured by the Jacquard coefficient) .

Figure 4A shows the INPUT affinity matrix for a representative experiment during which about 200 stimuli were delivered from SNN to BNN.

SOPs were calculated based on the BNN ( 4B ) vector response network . In particular, the number of pulses recorded by each electrode within the time window T after the SNN stimulus delivery was calculated. Next, for each stimulus, a vector network characteristic (VNR, on the left on 4B ) was constructed and a matrix representing SOP ( 4C ) was calculated . In addition, the response of the scalar network (SNR) to each stimulus, calculated as the sum of VNRs (on the right on 4B ), was also considered.


Image No. 5

To begin with, an assessment was made of how the SNR varied depending on the intensity of the stimulus. When examining the entire response time window (500 ms) after the stimulus, it was found that the response was ambiguous: the SNR was approximately linear with respect to the stimulus intensity with a correlation of 0.70, when only responses not exceeding 1/8 of the maximum were considered, while NR were approximately evenly distributed above such a threshold. Similarly, when a shorter response time (50 ms) was chosen and when focusing on NR with a threshold below 1/6 of the maximum, a correlation of 0.57 was observed between NR and stimulus intensity.

Therefore, for each experiment, the optimal response time window T was determined (from 1 to 50 ms, 5C - 5E) and the optimal threshold response network ( 5C ). Due to this, it was possible to maximize the transfer of information between SNN and BNN. Optimal information transfer was obtained in 8 of 12 experiments ( 5G ).

The scientists further examined how the transmission of information is related to the intensity and frequency of the stimulus from the SNN, which are both formed by spontaneous SNN network synchronizations that mimic those that occur in the BNN (image below).


Image No. 6

The transmission of information strongly correlated with the average stimulus intensity and showed a bell-shaped curve as a function of the average stimulus frequency, which reached a maximum at 0.56 Hz.

Researchers also note that spontaneous network synchronizations in the absence of an external stimulus can also occur, because they further examined the relationship between network synchronization (NS) in BNN and information transfer. Spontaneous NSs in the BNN (in the absence of stimuli, i.e., when the connection between SNN and BNN was disconnected) occurred with an average frequency of 0.37 ± 008 Hz. A high correlation was also observed between NS suppression and information transfer (image No. 8).


Image No. 7

Spontaneous NS suppression was quantified as the ratio of the frequency of spontaneous NS BNNs under basic conditions (i.e., when SNNs and BNNs were disabled) to the frequency of spontaneous NSs when communication between SNNs and BNNs was turned on. Spontaneous NSs in the presence of stimuli coming from the SNN were considered as arising at least 500 ms after delivery of the last stimulus (image No. 7).


Image No. 8 The

general results showed that information transfer can be achieved only with a linear response of network responses to stimuli. In addition, the best results were achieved during the early reaction period, i.e. during the first hundred milliseconds from the start of stimulation.

The maximum value of information transfer was obtained when the frequency of stimuli was about 0.56 Hz, which is only slightly higher than the frequency of biological neural networks (0.37 Hz).

The above results, according to the author of the study, confirm the theory that BNN activity should be strongly carried away by incoming stimuli from SNN in order to reliably process it in the linear mode.

For a more detailed acquaintance with the nuances of the study, I recommend that you look into the report of scientists and additional materials to it.

Epilogue


This study in practice confirmed that the relationship between the artificial neural network and the real one is quite possible. Of course, as the scientists themselves admit, among the thousands of neurons used in the experiments, only hundreds entered the synchronization mode.

Comparing an artificial network with a biological one is quite difficult. One of the problems is spontaneity, which is inherent in real biological neural networks. Spontaneous activity in neurons causes synchronous activity, which corresponds to a certain rhythm, which can be affected by the connection between neurons, the types of neurons in this connection, as well as their ability to adapt and change due to new working conditions. In other words, neural networks are sometimes very unpredictable when they create such a complex synchronization system, as if creating order in chaos.

Therefore, in order to achieve synchronization of artificial and biological networks, it was necessary to adjust the artificial to this rhythm. During the study, good results were achieved in this difficult matter, although several unsuccessful experiments were required.

The main goal of their work, scientists call the development of neural prostheses that can successfully replace damaged areas of the brain. It sounds very futuristic, remotely reminiscent of the movie "Johnny Mnemonics." Nevertheless, the goal is noble, and therefore I want to believe that the authors of this work will be able to successfully improve their development in the future.

Thank you for your attention, remain curious and have a good working week, friends! :)

A bit of advertising :)


Thank you for staying with us. Do you like our articles? Want to see more interesting materials? Support us by placing an order or recommending to your friends, cloud VPS for developers from $ 4.99 , a unique analog of entry-level servers that was invented by us for you: The whole truth about VPS (KVM) E5-2697 v3 (6 Cores) 10GB DDR4 480GB SSD 1Gbps from $ 19 or how to divide the server? (options are available with RAID1 and RAID10, up to 24 cores and up to 40GB DDR4).

Dell R730xd 2 times cheaper at the Equinix Tier IV data center in Amsterdam? Only we have 2 x Intel TetraDeca-Core Xeon 2x E5-2697v3 2.6GHz 14C 64GB DDR4 4x960GB SSD 1Gbps 100 TV from $ 199 in the Netherlands!Dell R420 - 2x E5-2430 2.2Ghz 6C 128GB DDR3 2x960GB SSD 1Gbps 100TB - from $ 99! Read about How to Build Infrastructure Bldg. class c using Dell R730xd E5-2650 v4 servers costing 9,000 euros for a penny?

More articles:


All Articles