"We will not give rise to conspiracy theories." Talk about ML conferences with people from science and IT companies

Now almost no public events take place, but we hope that the situation will change soon (and we are making efforts to do this). We expect that the transferred events will take place on other dates, and future events will not be canceled, since there will be no need to cancel them. Including in 2020, dozens of large conferences on machine learning should be held.

NeurIPS (formerly NIPS) is considered the most prestigious of them. Each year, NeurIPS gathers thousands of researchers and engineers who present their scientific results in different areas of ML: deep learning, reinforced learning, scalable optimization, Bayesian methods and more.



Based on the NeurIPS held a few months ago in Vancouver, we had a discussion where we invited experts from the scientific world and the IT world. Each of them is interested in different things in machine learning, the experience of visiting NeurIPS is also different. Under the cut - a brief squeeze of those parts of the discussion that relate to the reasons for attending the conference.

Members

Dmitry Vetrov . Candidate of Phys.-Math Sci., professor-researcher, Faculty of Computer Science, HSE, head of the Samsung Laboratory at HSE and professor of ShAD. The author of more than 130 scientific papers. Dima has repeatedly made publications at top ML conferences, including NeurIPS. He has been at the conference five times since 2011: as a speaker and as a participant.

Mikhail Burtsev . PhD in Physics and Mathematics, Head of the Laboratory of Neural Systems and Deep Learning MIPT. Co-author of more than 20 scientific papers. Since 2017, she has been conducting competitions in dialogue systems on NeurIPS, she is interested in everything related to conversational artificial intelligence.

Sergey KolesnikovScitator. R&D lead at Tinkoff, project lead at Catalyst, researcher at Yandex and MIPT. The last five years has been engaged in DL and RL. NeurIPS first visited in 2019.

Mikhail Bilenko . Head of Machine Intelligence and Research at Yandex, previously worked at Microsoft. Area of ​​interest - distributed learning. Participates in NeurIPS since 2005, missed it only three times.

Konstantin Lahmanklakhman. Head of Computer Vision and ML Applications at Yandex. Interested in CV and NLP. Rides the NeurIPS every year, starting in 2014.

Alexey Natekinnatekin. Creator and dictator of Open Data Science (ODS) - the largest online community of data scientists in the CIS. The areas of interest in ML are meta learning, federated learning, transfer learning and everything that lies around auto-ML. He went to NeurIPS for the first time in 2016, was a participant two times and twice helped to organize competitions.

Rest with mathematical engineering load

Why do you ride NeurIPS?

Sergey Kolesnikov :
For the last three years I have been promoting RL, this is one of the reasons why I wanted to get there. I won the competition, I was invited, but it is difficult to make a visa for a month in Canada. And in 2019, I finally caught on: there was a visa from the past NeurIPS, which was done for six months. This is a success story and three missed visas.

Mikhail Bilenko :
There I find articles and people whom I would not find otherwise, I communicate with colleagues. In 2016, I went to NeurlPS out of work, just then we met Kostya Lahman. The conference is an excellent engineering and scientific tourism: rest, but with a mathematical and engineering load. Allows you not to get fat, to correct the shape and be in good shape. You can not be distracted by work tasks - you just get pure intellectual pleasure.

Dmitry Vetrov :
When I was younger and smarter, I went to learn new directions and new machine learning techniques. After about ten years, it began to seem that the reports were no longer so cool, and soon I could not find out anything valuable for myself. But so far every time I take away from the conference a lot of interesting things - something that cannot be read on arXiv.

Mikhail Burtsev :
I join Dima. On the one hand, it seems strange to come and listen to what you read about a long time ago. On the other hand, the conference helps to pay attention to work that you may not have noticed. Communicating with other people, you can get a completely new look at your work, see something that you yourself had not seen before. And unlike articles, here people still discuss what they are doing now or planning to do. This helps to adjust the course of their work, not to make mistakes. You can suddenly find out that you don’t need to do what you were going to do.

Konstantin Lahman :
In addition to networking, there is another important point - the conference helps to check where the wind blows. When the NIPS was slightly smaller, the wind direction was more understandable. First, the wind blew into the sails of the GAN networks, then into the sails of the NLP, then into the RL.

Now sometimes there is a feeling that the wind blows in all sails simultaneously, but not so much. Nevertheless, it is useful to compare yourself with what is happening in the world in your sphere. Matching is optional, but interesting to compare.

Modern science is designed so that negative results are not published. Only positive ones are published, unfortunately. Such conferences are a great way to find out what and why didn't work.

Alexey Natekin :
There is an average temperature in the room: what they say at the conference itself and what on the sidelines on the afterparty. Backstage conversations are perhaps more interesting than those in the hall. Something truly new can be found mainly at workshops, because applications for participation in them are submitted in a month, and applications for participation in the conference itself - in six months.

From year to year the conference grows and changes a little, but nothing fundamentally new is added. Don't you think that scientific novelty is getting smaller?

Dmitry Vetrov :
To quote Axiom Cole: “The sum of the mind on the planet is a constant value, and the population is constantly growing.” It seems to me that the scientific novelty at the conference is also a constant value, and there are more and more conferences. Therefore, in my opinion, the average level of reports at NeurIPS is falling.

Mikhail Bilenko :
Dima correctly said, but it is always important for me to take out something useful from that percentage of the material that remains interesting. There is Sturgeon’s law that 90% of everything is complete nonsense. I focus on the remaining ten. And in the quantitative ratio of nonsense it became more, I agree.

Konstantin Lahman :
By the way, it seems to me that what happens with NeurlPS is not the same as with all conferences. Due to the administrative framework, it is trying to be such a reserve where the minimum necessary population of all regions is artificially maintained. CV conferences are organized differently: there is no work on some topic - well, a lot of work on another topic is also good.

Mikhail Bilenko :
I was on the NeurIPS program committee for several years, and yet in fact there is a fairly organic distribution of areas. Let's not spawn conspiracy theories.

Academy and IT companies

How visible is the problem with NeurlPS that models need to be learned on tons of GPUs, and companies and not academies can afford it?

Mikhail Bilenko :
In the past few years, the organizers have been holding an open meeting. At it, academics say that comrades from large IT companies write articles only because they have cool hardware and they can conduct gigantic experiments. Our resources are really not equal. So there are high demands for the results to be reproducible.

Question to scientists. How do you feel about the fact that many people from the industry come to the conference who listen to reports, communicate, but do not bring anything new?

Dmitry Vetrov :
I feel a huge buzz when I can talk about what I’ve done to a large number of people - it doesn’t matter to scientists or specialists from the industry. The main thing is that they like my work as much as scientists.

Mikhail Burtsev :
It doesn’t matter to me either who is from the industry, and which from the scientists, especially since now many scientists have moved into the industry, and everything is mixed up. It is important that it is interesting to communicate with the person. In this regard, I like workshops more - there the circle of people is limited and there are more opportunities for communication.

There is always a gap between what science does and what goes into production. How close are the articles presented at the conference to what can be quickly rolled out into production?

Konstantin Lahman :
It is incorrect to say that about NeurIPS, because originally there is practically nothing on it that can be immediately applied in production. The conference budget consists of contributions from sponsors and participants. Most of the participants are from the industry. It turns out that IT companies pay for a scientific conference. And this raises an interesting question - should the scientific community take this into account somehow? The industry pays more for his work. Maybe he also needs something to help the industry?

NeurlPS is about finding inspiration and understanding where machine learning develops.

Sergey Kolesnikov :
Yes, NeurIPS is interesting precisely because in one week you manage to talk with a large number of people, you can immerse yourself in various industrial and academic cases, get a wide view of everything that happens. All ideas from there can be put in a black box and then periodically open it. And at the next conference it’s already to check what worked and what didn’t. In this regard, divine.

Top ML Conferences

If you could only attend two conferences of the year, what would you choose?

Alexey Natekin :
Data Fest and Data Fest in Minsk!

Konstantin Lahman :
NeurIPS and CVPR.

Mikhail Burtsev :
NeurIPS and NLP.

Mikhail Bilenko :
ICML and NeurlPS.

Sergey Kolesnikov :
NeurlPS and Data Fest.

Dmitry Vetrov :
ICML and NeurlPS. Swap them when there are no traffic jams on NeurlPS!

Video