Get best of the week

Test assessment: how to calculate the exact time to test the system or "When will the tests be ready ?!"

image

Good day to all! My name is Denis, I am the head of the testing service at BARS Group. This is my first post on Habré.

Having read a lot of interesting articles and got a lot of useful information from there, I wanted to give something in return. Then I began to analyze the topics: some were already voiced, others were too simple ("how to enter IT?"). PS I didn’t want to hurt anyone’s feelings :)

How to calculate time for tests - problem and solution


As the head of the service, I constantly come across a question from managers: “When will it be ready?” or "How long does it take to test?" It would seem that this is complicated, take an assessment of the previous project and plus or minus the same thing ... but no. I realized that the task is not trivial and requires detailed study. And I want to share her decision.

Our company has many business centers and each has its own development approach - mainly Kanban and Scrum. Therefore, teams of automated testers have been singled out, which are synchronized with the development team with their methodology.

Due to different approaches to development management, difficulties arise in the uniformity of task formation and planning. The use of Kanban and Scrum in its pure form did not give an answer how long it would take to test. In design decisions, each time it is necessary to evaluate the new functionality and cover it with tests. It took me a lot of time to calculate. Therefore, I decided to take the methods for estimating time costs for software development (for testing automation) as a basis and modify them to fit my realities. The principle of weighted average assessment and calculation based on typing formed the basis. The estimates will be temporary indicators for the automation of typical elements of the system, and the level of specialist training will be used as weights. When forming the values ​​of the weights, I chose the accuracy of the assessment when performing the task, i.e., the more experienced the specialist,the smaller the estimation error. The following values ​​were obtained:

  • "Senior" - 95% accuracy, factor 1.05
  • "Middle +" - 80% accuracy, factor 1.2
  • "Middle" - 70% accuracy, coefficient 1.3
  • "Junior +" - 60% accuracy, coefficient 1.4
  • Junior - 50% accuracy, factor 1.5

Next, we will need to multiply the time estimate t n by the corresponding coefficient W n . Our calculation method is performed according to the formula, where the sum of the weights is not equal to 1 (100%).

image

W avg = (w 1 * t 1 + w 2 * t 2 ... + w n * t n ) / (w 1 + w 2 + ... + w n )

For calculation, I took two tests - functional and UI testing, because they total about 85%.

To get the final result, we need to collect a weighted average score for each element into a larger object for calculations - a category.

UI testing


When testing the UI, it is required to emulate the user's work through the Selenium.Webdriver framework. When using this approach, there are difficult to construct elements on forms: tabs, documents with online editing, huge grids with lines, a tree structure, etc. In addition to these elements, there are also factors that affect the test development time:

  • Structure of forms (typical constructor or custom)
  • AJAX requests (their number)

Based on this, 3 categories of UI forms were distinguished by their difficulty in implementing tests:

1 category



2 category



3 category



As a result, I received the following results, which are presented in the table:



Functional testing


For functional tests, the situation is similar to UI - categories for systematization of cases are highlighted. In addition to REST services, it is worth mentioning about SOAP, it will be similar to 3 categories of REST.

Integration testing involves testing several methods in one service, for an approximate assessment we took the presence of 5 methods per 1 service.

1 category



2 category



3 category



Similar to the UI table:



Integration testing checks the operation of services built on both REST and SOAP. When designing a service, the number of methods used inside can vary. For calculations, we took an average of 5 methods.



With this calculation of the time spent on the project, the percentage of getting into this estimate was 81.

Instead of a conclusion


It took a week of hard work to count the first time. Therefore, I did the evaluation after testing and then compared the results with real time costs.

It is enough to do the main work once and then consider it according to the ready-made “formula”. But you need to take into account the fact that the level of employees is growing, so you need to understand the weight of each employee in order to know whether to recalculate indicators.
All of the above is my experience and does not claim to be true.

More articles:


All Articles