Get best of the week

Who else needs Selenium? Does anyone use BDD in 2020? Machine Learning at Selenium

Every time I come across a new article about how to use the BDD approach and how it improves the life of development, testing and management, I grab onto the face. (And now I don’t know what to do. I don’t grab my face, but just sad). However, the tasks of writing Selenium tests can cause the same sensations . I wanted to talk about it with those who are faced with Selenium tests and are developing various tools for convenient work with them.

image

We (Vsevolod Brekelov and Daria Manukhina, Heisenbug conference program committee) talked with Anna Chernyshova, developer of the Akita library and the new Healenium tool ( her talk about BDD ), and Ivan Krutov, developer of Selenoid. Ivan has been engaged in infrastructure for Selenium tests for several years ( One of his master classes ). Discussed:

  • BDD approach in 2020;
  • Selenium and its development;
  • Career;
  • Machine Learning and Selenium.


The interview consists of two parts: in the second, we talked about the infrastructure of Selenium tests, the tester’s tools and answered the question that was raised at the end of this post.


A lot of letters about interlocutors


- Anya, Vanya, tell us about yourself. Let's start with Ani: tell us briefly what you mainly do.


Anya : I now have two main activities: the first - I work in the EPAM testing competency center at EPAM. This is such a group of people who are being consulted for help in starting a project, selecting technologies. I do a lot of audit testing on projects. I understand what is wrong, what is good, what is bad, I recommend improvements, I make PoC. Basically, such consulting within the company. The second direction is the accelerator, our new Healenium , we have compensated for it relatively recently, this is the thing that can repair broken locators in runtime and keep tests up to date.

- Great! Vanya, what about you? I know that you are a Selenoid developer.


Vania: I am a specialization developer. Java, Go are basically such languages. Although it started a hundred years ago. I also wrote in PHP when the simplest thing was there. At school, he wrote in all sorts of pascals, like everyone else. As a result, the last 6 years I have been doing what is now called the buzzword DevOps, this is such a thing at the junction. According to the position in Russian, this can be called, for example, a software engineer, it is both a developer and a person who is involved in maintaining the infrastructure. For example, I am responsible for the performance of a large Selenium cluster with a sufficiently large number of browsers, it is fault tolerant, distributed, and so on. This is the first part. The second part - I do all sorts of open source and not only projects. We write Selenoidnotorious for everyone in open source. We also have commercial tools. For example, Selenium for Kubernetes is a separate product, we make cloud solutions for Selenium, well, and much more. There is also a product line, we are professionally developing tools for Selenium: open and closed are different. Such an activity. Of course, I go to conferences, speak, educate people, how to effectively build Selenium based on my experience.

- You said a little about Selenoid. I know that Selenoid is a project of the Aerokube organization. Can you tell me how it is? Is this a community or is it an organization? A commercial organization or how did it even appear?

Vania: It is both. In fact, initially it was just a community, they started an organization on GitHub in order to create open-source, effective tools for Selenium. This was done in his spare time from work, from the main work. It turned out to be the first Selenoid tool, then in the end we realized that in order to be able to properly ensure, so to speak, the effective development of this Selenoid you need one of two things.

All good open source products are developing in one of two directions. The first is when they are made in some large company that simply sponsors them, pays the salary to the developers, and the second option is when there is a free open source solution, and there is some paid product, similar or different, due to which development is monetized. As a result, when Selenoid flew with us, we just made a decision and designed it in the form of a company. Now it is already a commercial organization that has open source and has its own products.

- So you work in it, it turns out?

Vanya : Yes.

- Good. Anya, you wrote the Akita library , which was being developed at Alfa Bank. Are you still using it somewhere actively? Or did she stay in Alpha and that's it?

Anya: It was developed just as part of Alfa Bank. First of all, she was for their needs. Then we opened it up, because it seemed that it might be useful to someone else. As far as I know, the bank is now using it, of course, and maybe also about a dozen third-party organizations. But I don’t work with her right now.

- You promoted BDD . Are you promoting it somewhere now, or is this not the topic that you are actively working on?

Anya: It also depends on the situation. If the customer comes and says that I want to be aware of what is happening, then yes, they are good at this approach. If feature teams do this, or testing is carried out separately as a service, then in these cases I do not promote it. I have not worked with BDD for some time, now it is mainly self-healing automation.

Use of BDD in 2020. This is a joke?


- BDD in 2020 is okay?

Anya : I think that okay, but it depends on the situation. Now we are testing self-healing with BDD in the same Alfa Bank and everything is fine. Type it can be updated, it is this BDD approach that is made more stable and then it works well.

Vanya : In fact, BDD - it was for managers originally. So that a person who does not have very deep competencies in the development of tests can nevertheless be quite easy to read how the product is tested, to understand. It seems to me, as long as the whole development is organized in such a way that there are executive managers who have different competencies and there are developers, I think that BDD makes sense, why not?

Anya: There are still situations, for example, how the bank had it, that the teams simply did not have a tester, and it was necessary to test it somehow, and the analysts said: “here we want!”. For them, just this tool including was developed. They came almost to this ideal formulation of BDD, that the analysts were also involved in writing the scripts, and it was convenient for them to be guided by this upper-level description of the steps in order to compile the tests, and they could automatically run.



- This holivar is still coming, who should write tests for BDD? For example, there is some organization that does not yet have BDD, and it is unclear whether it is needed or not, and who should write these tests. In an ideal world, what would you recommend?

Vanya : I think it depends a lot on the amount of everything.

Anya : Yes, and it depends a lot on the situation. If this is a feature team, where there are all representatives of these roles and some tester with good hard skills, and there are many such teams, and this is how the organization is built, then they will hardly need BDD. If this is a team where there is no tester, or the tester has weak hard skills, for example, this is mainly manual testing, then this approach will also be applicable, because the tester will also be able to do automation immediately.

- That is, if you have a lot of money and you have hired a lot of people who are technically incompetent, then is BDD suitable for you?

Anya : In principle, you can say that.

Vanya : In general, it seems to me that not only BDD, but any testing during the planning should be strongly based on money.
It so happened with us that most conferences nevertheless move in the technical or philosophical direction. It’s not customary to say that in any case, we work in companies that have budgets, which have some limited amount of money. All the decisions that management makes, it all the same in the end rests on this very money. There is no getting anywhere. Just programmers, testers and analysts, we brew in such a "limited" world, where we are not told about them, they can only say "we do" or "do not." It seems to me that testing, development planning, and everything else - you always need to build on the existing budget, nothing more. All the same, everything rests there.

Is BDD outsourced or hosted?


- If we talk about whether BDD is for an outsourced project and for an internal one, is there any difference or not? If the budget allows, do you need BDD for outsourcing?

Anya : Here again, yes, everything rests on the budget. I don’t know, I probably wouldn’t give BDD personally to outsourcing, but if, of course, the customer doesn’t really want to. If he says: “I want BDD and that's it, I don’t care!” - then yes. But this will require such skilful guys so that they beautifully wrap it all up in some beautiful framework. Typically, customers come to outsource who say: “I quickly want some kind of automation,” or “I don’t have automation, build,” or “I’m doing badly, let's update something somehow.” Usually in such situations, their solution requires a deeper technical understanding and automation at a technical level.
Although if it falls well to the client and you can quickly start automation, it also probably depends on the time. If it is possible in this way to quickly start and achieve the goal that the client wants, then yes, he can lie down.

- It seems that BDD for outsourcing is not only about how to write a project to them, finding out the requirements, but also about teaching customers to generally understand what is there. Because the training of some analysts or even managers in understanding how they should make scenarios, if they are in the future, is also a different story.

Anya : Yes, this may have its own difficulties.

Vanya : Returning to the question of whether BDD is normal in 2020, it seems to me that there is some definite evolution, I’m leaving so philosophically ...
Over time, we have all sorts of new approaches and technologies, and it seems to me that it doesn’t happen that technology completely dies. For example, 40 years ago, when computers were developing, there were mainframes of these huge machines that have now almost died, nevertheless, certain niches remained where they still exist and work perfectly.

Photo from the site www.kmd.dk

I just recently spoke with a person, he is somehow connected with banks, with some old American ones, he says: “Everything costs forty years ago, everything works, because it’s cheaper to maintain it than rewrite to a new one. " In the same way, I think, in testing, new approaches appear, BDD has appeared. I do not think that it will die at all, it will remain in some niches, even if they come up with something better than this. So with any product, with any technology. It is impossible, on the one hand, to completely, one hundred percent seize the entire market, on the other hand, to die completely too - you need to try to make this happen, there are always some niches. The question is what is their share in the total mass of people.

Career


— - , , , , , , .

: People often ask the question: how to grow in the company? Junior developers, juniors come and, naturally, in their head the goal is how to grow. In the end, at some point, you begin to feel a business demand that money should be counted there and so on. In fact, the way to grow into a higher level of a chief developer is to start thinking even in such a low position about how you would start to do it from a business point of view. Not from the point of view, what would I try a cool new language or something else, this is cool for a personal examination of my own, but from a business point of view, if you want to grow, it seems to me that you need to think more about this.

- - — , .


At one time I heard a smart person. He said that there are different ways to grow a career. He listed three main ways. The first way is to be someone's relative. This is usually not enough, a small number of people who have high-ranking relatives there. The second way is to do manipulation. All sorts of intrigues weave in the company and so on. There are few such people who know how to do this. And the third way is to prove by the work your utility to the company. The main idea of ​​growth is to find a niche in the company that has problems. And suggest to your management how to solve these problems effectively, but in this case you can greatly grow.

— , , . ? , , , : «, , , ! , , ».

: It all depends very much on the person and on the team. By and large, if a person wants to develop and grow, he will still have to do it at some point. Until a certain point in time, the manager will communicate for him, if he has one, but if he eventually wants to develop, he will have to do it. How is a lead developer different from a senior? The fact that he has a very good expertise and the fact that he knows how to solve such already half-managerial tasks. He knows how to bear full responsibility for the product that he makes from drawing up requirements to directly developing, this is the difference, it seems to me.

Anya : Yes, here comes the understanding of why you are doing something, and not just doing it - it will be cool.

Selenium Is he still needed?


— Selenium, . : « - Selenium? , , , Selenium?»

: Now it is often said that Selenium is dying. Some other tools come in its place. It seems to me that this is not entirely true and all the same it all depends on the expertise. In any case, most of the projects now at Selenium are done and done on it. You will need support, and if as a testing specialist you want to get somewhere, then most likely it will be Selenium. There is probably much more expertise in this area, plus when some new technologies enter the market, it can bomb and then slide somewhere. Selenium covers, I would say, all the possibilities of jujine testing. And this is such a powerful thing, which in the near future will definitely not die.

If as a testing specialist you want to get somewhere, then most likely it will be Selenium.


Vanya : I also want to supplement. There is always a certain inertia with technology. Selenium, as you know, has existed since 2004, it is already 15 years old. Of course, it also changes on its own, however, during this time a rather large number of ready-made projects have accumulated. For programming, 15 years ago is a stone age compared to what it is now. And more experience has been gained precisely in Selenium. Just start even moving out to some new technology, if you are already an experienced automation engineer, it’s difficult, it’s really inertia.
Since the product has existed for a long time, it has accumulated a large knowledge base, pitfalls, even if it’s not only Selenium’s problem, it’s generally a problem of any new technology - in order for it to fly, you need to gain some critical mass of knowledge, users, and so on .

Selenium has a critical mass of users, there are forums, communities, Stack Overflow, a lot of documentation, a lot of conferences and specialists who can constantly transfer this knowledge. As soon as a new technology arises, it can fly now only if there is a lot of money in it - and everything depends on money.

There is an example not about testing, but about development. There is a programming language Kotlin. He now, as it turned out, flew in the mobile direction, straight flies, flies. They say that there are some rumors that just a pretty decent amount of money has been invested in its promotion, so that it is mentioned everywhere in conferences, so that people develop frameworks. So much is happening. To overcome inertia, you need to spend a certain amount of resources. Just making a good product, even if it's really better than Selenium, is hard to promote.

Why Selenium? Because it is now standard. There is a specific World Wide Web Consortium, which took and fixed the standard Selenium. It is always better to work with tools that are standardized. Choosing between two tools, we can say: “Everything is fixed here, the API is stable, everything is fine, but there is some obscure part on the side that some company also does, it is unclear how long it will last.” When choosing between these two tools, as a rule, a reasonable manager usually chooses something more standard. It seems to me that writing tests on Selenium makes sense, because even some beginner is now easier to do. The entry threshold will be lower - more than anyone you can ask about Selenium.

The second reason why you need to continue to write tests on Selenium, why you generally need to write tests on UI. In fact, writing tests on the UI, such end-to-end, is the only way to verify that the application works exactly the way the user sees it. Even if we write all sorts of units and everything else, cover with different types of testing, this does not give a one hundred percent guarantee. Only after losing the same actions that the application user will do, we can be one hundred percent sure that the application is working with us. It is clear that you do not need to completely pass one hundred percent of the code this way. Usually the most critical scenarios in the application are covered, but nonetheless it is important.

It is always better to work with tools that are standardized.


Anya: I would like to add more about new tools. Once I heard a smart guy at a conference. He talked about the launch of new technologies and how to understand which technology will die, which will be delayed. In any technology that enters the market, it begins to gain what Vanya is just saying, a critical mass. Then she has a boom, and this boom needs to be waited, as it were, because after this boom either this technology remains, it stands, it will have its own development and leap forward, or she will die after that. It also depends on investments, on whether the development team manages to cope with the stream of users who come to them, the stream of bugs, or directions for further development, or not. So right away, grab onto something that’s on the hype and transfer your tests to this technology, it seems to mewill be unjustified.

- In your opinion, how much has changed in Selenium? Has anything changed dramatically in the last two years in this instrument and is it evolving somehow?

Vanya : I think it changes quite a lot. Two or three years ago, there was a feeling that there was some kind of stagnation in it, because in fact Selenium was invented in 2004, then it developed, developed, developed, then in 2008-2009 they came up with Selenium WebDriver, and they came up with Selenium Grid. After five years, they just finished it, rolled out some small features, in general, nothing changed much. Everyone had a Selenium Grid, everyone had a standard Selenium library.

But the last 2-3 years, there has been a boom in all sorts of new products based on Selenium. We, for example, made Selenoid. The guys who worked at that time at Zalando , based on the Selenium Grid, made such a tool as Zalenium, also a Selenium-compatible solution. In fact, both tools were made in order to solve any specific problems for which there was no solution in the original code. Then others appeared, competitors of Selenium itself. Now, it seems to me, for the last two or three years, some kind of moving movement has gone.

It seems to me that Selenium is developing, plus at the beginning of 2018 they fixed the standard. Therefore, Selenium, of course, on the one hand, as a tool, has quite a lot of maturity. On the other hand, the team that originally developed the core code succeeded in imposing responsibility for maintaining compatibility with the standard to other teams of browser developers. Previously, virtually all of Selenium was a single unit in which everything lay, all the drivers, but now they managed to agree with all the teams. Apple for Safari does support because there is a standard and there is no need to coordinate with anyone. Google does support for Chrome, Mozilla does for Firefox. In fact, now it is already a distributed ecosystem, there are several different points. Now Microsoft also makes its own for its browsers, for Edge it made its own driver,and they managed to distribute the responsibility. Due to this, it seems to me that there has been a pretty decent leap in quality.

- Yes, three years ago, I remember that the tests for Selenium on Safari ran, it was necessary to unscrew yourself like that. In general, it hurt.

Vanya : And now everything is done more or less according to the standard. There, of course, there are some peculiarities of their own, but in essence everything became uniform, that is, there was such a unification of how it all works inside. I think it's cool, there a lot of effort was put into it all. Therefore, I do not think that there is any stagnation in Selenium now. Now there is a healthy competition of different decisions.

We are in some ways competing with the guys who make the main Selenium code. We come to the conference, we say that guys, yours is not effective because, therefore, and therefore. In order to somehow prove that they are also well done, they have to do their own thing. There is healthy competition.

Anya : Regarding the Selenium IDE, I also want to add, about which in general, it seems to me, everyone has long forgotten. With the advent of new tools for script-less automation, they again took up the Selenium IDE. During 2019, it was developed very much and a large-scale release is planned in early 2020, where they will present something. The IDE’s interface has changed a lot and it has become more stable and convenient, let's say.

Vanya : I also wanted to add that now they started trying to apply different technologies to Selenium that had not been applied before, for example, the same machine learning.
This is one of the areas that still did not exist in Selenium. People, having used Selenium for about 10 years, found a certain number of tasks that, as it turned out, can be partially or completely solved with the help of machine learning. All sorts of routine operations.

- Anya, tell me what problems do you solve by machine learning? Will they write everything for us? In our country, artificial intelligence is beginning to penetrate everywhere. If you drive “AI and Selenium” into Google, people begin to develop their own libraries there, upload something else, that is, development is underway.

Anya: Regarding Selenium, I can say that recently it has been much more convenient to make all kinds of decisions on the basis of it, well, in particular, due to its standardization. Regarding features, the most important thing you need is to interact with the elements on the page. And accordingly, this part already lies with the testers how to describe this interaction using Selenium. If you describe it purely using Selenium, then there may be unstable tests, because any changes there will occur on the UI, some element will not wait, and so on. Machine learning is beginning to be applied to this place so that these tests can be stabilized. So that these tests can be generated based on the information that we have on the page, and based on user behavior.

The trend was Record and Play, what exactly the Selenium IDE does: we go to the page, click on the action, we create tests. These tests must be stable. Machine learning is needed there just for that. Usually locators are hardcoded, so if you change the UI, you need to update them. The Healenium, which we are currently doing, for example, allows us to update them in runtime, and our test will pass accordingly.

Now there is such a trend not purely for Record and Play tests, but for the possibility of generating tests. We know what elements we have on the page. We can draw some kind of logical diagram of our site. We know that we have the main page, from it the login page and something else. Based on these logical connections and actions that we can do on this page, we can generate tests. But I don’t know such instruments on the market that did it very well. Healenium we want to develop just in this direction.

With the development of these approaches and knowledge in the field of machine learning, in the near future there will be a bias in the development and appearance of such tools. For example, crawlers of sites that can automatically test instead of writing test cases.

— , Selenium IDE . , . IDE .
, , ?


I propose to discuss this issue in the comments. And in the continuation of this interview we will find out the answers of Anna and Ivan. By the way, at the Heisenbug 2020 Piter conference , which will be held online , you can chat with them and learn more about Healenium, Selenoid and the use of the Chrome DevTools protocol in the Kubernetes cluster.

Recently, the second part of the interview with the answer to the question about ML in testing was released: they found out who the person is the marker , and figured out whether Selenium should be studied in 2020.

For those who want to broaden their horizons and attend more than one conference, but immediately 8, we have prepared something .

More articles:


All Articles