It often happens that a web application consists of a large number of dynamically restructuring forms with different text and controls. Testing such an application turns into a nightmare.You need to click 100500 pages and check all the functionality ... And before the next release, check the same again ... And again ... And tomorrow again. At some point, the verification begins to take longer than the development of new functionality. “What about e2e tests?” - you ask. But, firstly, they still need to be written. And secondly, before you start writing them, you need to write test cases. A lot of test cases.If while reading these lines your forehead is covered with perspiration, do not worry. In this article I will share with you the idea of how we at Tinkoff automated the testing of one of the web applications without writing a single test case and e2e test.
Automatically writing test cases
It just so happened that testing our web application is mainly related to interface checks. You need to check that the button is present on the screen, the desired title and text are displayed, and when you enter an invalid value in input , an error message appears.Accordingly, when writing a test case, you need to record all the actions:- “Pressed the button”
- "Entered the value of XXX"
- “Selected the value YYY in the drop-down list”
and checks:- “The text appeared: XXX”
- "Error message appeared: YYY"
- "The heading appeared: ZZZ"
After analyzing all the functionality of our web application, we identified about 30 unique actions and checks. It became clear that this process can be automated. To do this, you just need to track all the actions of the tester on the page and the reaction of the site to these actions.Let's start by intercepting events. To track interaction with controls such as buttons, radio buttons and check boxes, you need to subscribe to the click event. Each framework has its own methods for this. For example, fromEvent in Angular and document.addEventListener in JavaScript and React. For input controls, such as calendar or input, only the type of event you want to subscribe to will change: instead of click, focusout will be.fromEvent(this.elementRef.nativeElement, 'click')
.subscribe(tagName => {
if (tagName === 'BUTTON') {
this.testCaseService.addAction(` "${action.name}"`);
} else if (tagName === 'INPUT-CALENDAR') {
this.testCaseService
.addAction(` "${action.name}" "${action.value}"`);
}
});
And finally, the most important thing is checking. The way the site should behave in response to the actions of the tester.What does the tester usually check? For example, he entered an invalid value in input , the site responded with an error message. Or, let's say we clicked on a button and in response a new screen appeared, the title changed, new text appeared, and the controls were rebuilt. All these changes are related to the change in the DOM tree. There are many options to track them. You can, for example, use MutationObserver in React and JavaScript or ngAfterViewInit in Angular (putting a directive on the form elements of interest on the site).ngAfterViewInit() {
const tagName = this.nativeElement.nodeName;
const text = this.nativeElement.textContent;
if (['SPAN', 'P'].includes(tagName)) {
this.testCaseService.addContent(`** ** "${text}"\n`);
} else if (tagName === 'H1') {
this.testCaseService.addContent(`** ** "${text}"\n`);
} …
}
The code will be very dependent on layout. Let's take a look at the markup. These buttons are taken from the Google translator.
<div class="tlid-input-button input-button header-button tlid-input-button-text text-icon" role="tab" tabindex="-1">
<div class="text"></div>
</div>
<div class="tlid-input-button input-button header-button tlid-input-button-docs documents-icon" role="tab" tabindex="-1">
<div class="text"></div>
</div>
Despite the fact that buttons are not represented as button tags , looking at the markup, you can select all buttons on the page using the “input-button” css class, and you can get button names from the nested “text” css class.Half the job was done, it remains only to write down everything that we tracked into a test case.We include the interception of all actions on the site for a specific key combination and only on the test circuit. We stop the interception of all events on the site by a certain key combination. This allows you to start and stop the automatic recording of the test case from anywhere.Automatically writing e2e tests
If you look at the automatically generated test case, then these are essentially user scripts reduced to the same form. So, they can be converted into e2e tests. You can even immediately write e2e tests after intercepting all the actions and checks, bypassing the test cases.Now there are a large number of different frameworks with gherkin notation based on behavioral scripts: SpecFlow, xBehave.net., Cucumber.js, CodeceptJS, etc.To get features from the test case, you need to add the When keyword before the actions and before all checks Then and And.Get the automatically generated e2e test:Feature: 2-
Background:
When "" ""
Scenario:
When " - "
Then " "
And " - "
When " " ""
When "" " "
When ""
To run the test run, the generated features are few - you need to write a handler for all actions and checks.There is good news: you do not need to write a handler for each feature. As I said, despite the large number of different forms on the site, we got only 30 unique actions and checks, which means that there will be exactly the same number of methods in the common processor for all e2e tests. The code will be slightly different - depending on the chosen framework with gherkin notation and layout on the site. But writing the handler itself does not take much time.When(' {string}', async function (button: string) {
const xpath = "//button"
const btn = await getItemByText(xpath, button)
await waitAndClick(btn)
})
When(' {string} {string}', async function (label: string, text: string) {
const xpath = "//*[contains(text(),'" + label + "')]/ancestor::outline"
await inputSendKeys(currentBrowser().element(by.xpath(xpath)), text)
})
Now, checking the next task, a test case is automatically written for the tester and the automatically generated e2e test is run.In short, you need to:- Subscribe to events of interaction with controls and the site’s reaction to these actions (through tracking the rebuilding of the DOM tree).
- Convert data from paragraph 1 to e2e tests.
- Write a general handler for running e2e tests.
This approach will help you get away from the routine. You can automate the writing of test cases and e2e tests for simple checks related to the interface. We have gone even further and check automatically also the record in the database and sending to third-party services.I talked about this, as well as versioning, the technology stack, and even about the problems at the first stage of implementation and their solution at the Heisenbug-2019 conference in Moscow .In this article, I tried to convey the main idea without going into details.Conclusion
Now it takes us 2 minutes on average to write a test case and an e2e test - this is 60 times faster than the initial calculations, when we wanted to write test cases and e2e tests manually.We did not change the processes in the team. It was no longer necessary to allocate the test capacity for writing test cases and take it to the automation team.We are completely gone from the notion of regression. If earlier, with a two-week sprint, the regression took us more than 3 days, now the regression takes time to run all the e2e tests, and this is only 2 hours.When manually writing e2e tests, it is very difficult to go in parallel with testing. Now, e2e tests are written automatically during the testing of the task, and the tester does not need to check the same functionality twice.As a result, our team, without changing the composition, began to work much more efficiently.