From galaxy models to atoms - simple tricks in AI implementation speed up simulations billions of times

To model such extremely complex natural phenomena, such as the interaction of subatomic particles or the influence of fog on the climate, it takes many hours to spend even on the fastest supercomputers. Emulators, algorithms that quickly approximate the results of detailed simulations, offer a way to get around this limitation. A new work published online shows how AI can easily produce accurate emulators that can accelerate simulations in various fields of science billions of times.

“This is a major achievement,” says Donald Lucas, a climate simulator at Livermore National Laboratory who was not involved in this work. He says that the new system automatically creates emulators that work better and faster than those that his team manually develops and trains. New emulators can be used to improve the models they simulate and increase the efficiency of scientists. If the work passes the peer review, says Lucas, “it will change a lot.”

A typical computer simulation can calculate, step by step, how physical effects affect atoms, clouds, galaxies - everything that they simulate. Emulators based on such a variant of AI as machine learning (MO) skip the stage of laborious reproduction of nature. Having received the required input and output data of a full simulation, emulators look for patterns and learn to speculate on what the simulation will do with the new set of input data. However, to create training data, it is necessary to run a full simulation many times - that is, to do exactly what emulators should get rid of.

New emulators are based on neural networks - MO systems inspired by the structure of the human brain - and they need to learn a lot less. Neural networks consist of simple computing elements that are connected to each other in a certain way to perform certain tasks. Typically, the connectivity of the elements changes in the learning process. However, a technique called "search for neural architecture" allows you to determine the most effective connection scheme for a given task.

Based on this technique, Deep Emulator Network Search (DENSE) technology relies on a generic neural architecture search scheme developed by Melody Guan, a computer scientist at Stanford University. She randomly inserts computational layers between input and output, and then checks and trains the resulting connection on a limited data set. If the added layer improves operational efficiency, then the likelihood of its appearance in future network variations increases. Repeating the process improves the emulator. Guan says that with "enthusiasm" he follows how her work is used "for the purpose of obtaining scientific discoveries." Muhammad Qasim, a physicist at the University of Oxford who led the study, says his team based its work on Guan’s work,as this approach achieves a balance between accuracy and efficiency.

Researchers used DENSE to develop emulators of 10 simulations - in physics, astronomy, geology and climatology. One simulation, for example, models how soot and other suspended particles in the atmosphere reflect and absorb sunlight, changing the global climate. Her work can take thousands of hours of computer time, so Duncan Watson-Parris, an atmospheric physics specialist from Oxford and co-author of the study sometimes uses an emulator with MO. However, according to him, the emulator is difficult to configure, and it cannot produce high-resolution results, regardless of the amount of data it receives.

DENSE emulators show excellent results despite the lack of data. When they were equipped with special graphics chips, they showed acceleration from 100,000 to 2 billion times compared with the corresponding simulations. Such acceleration is often characteristic of emulators, but their results were also extremely accurate: in one comparison, the results of an astronomy emulator were more than 99% identical to the results of a full-fledged simulation, and according to the results of 10 simulations, emulators based on neural networks showed better results than usual ones. Qassim says he thought DENSE simulators would need tens of thousands of training examples to achieve similar accuracy for each simulation. But in most cases only a few thousand examples had to be used,and in the case of suspended atmospheric particles - only a few dozen.

“A very cool result,” said Lawrence Perrault-Levasier, an astrophysicist at the University of Montreal, pretending to simulate galaxies whose light undergoes gravitational lensing caused by other galaxies. “It is impressive that the same methodology can be applied to such different tasks, and that they were able to train it on such a small number of examples.”

Lucas says that DENSE emulators, besides being fast and accurate, have another interesting use. They can solve "inverse problems" - to determine the best parameters of the model for the correct prediction of results. And then these parameters can be used to improve full-fledged simulations.

Qasim says DENSE may even allow scientists to interpret data on the fly. His team is studying plasma behavior under extreme conditions created by a giant x-ray laser at Stanford, where the experiment time is very valuable. It is impossible to analyze their data in real time — for example, to simulate the temperature and density of a plasma — since the required simulations can take several days, which the researchers using the laser do not have. However, according to him, the DENSE emulator could interpret the data fast enough so that it could change the experiment. “We hope that in the future we will be able to analyze almost immediately.”