Get best of the week

Quality Architect: who is it and when is it needed

Every day in the field of IT there are more and more new tasks, including in the field of testing. If earlier a tester needed to simply test according to requirements (or without them), then now he needs to first understand how it can be tested at all, what technologies are needed for this, what can be automated, and how to include a release cycle and all this disgrace etc. Who should answer these questions? Who will communicate with the customer and clarify the requirements? Who will create the approaches and testing architecture, requirements?

Working as the leader and coordinator of testing on projects for large companies and resolving all these issues over the course of three years, I realized that it was important to still attract an individual who would answer the main question: “How to conduct testing?”.
I conducted a small investigation and found that such a role already exists, and it is called Quality Architect, but few people know about it. And the job descriptions of Quality Architect vacancies on employers' websites are striking in their list of responsibilities and skills, but I think that this is more likely due to a lack of understanding of who Quality Architect is.

Based on my experience in this direction, I decided to show by the example of one of the real projects who Quality Architect is and when it is needed.



Brief Project Description


The objective of the project was to replace one database with another, more precisely, Cassandra and its appendage in the form of FilloDB were replaced with SnowFlake, in the business process of daily, hourly and even minute-by-minute flow of delivery and data processing. There were more than 50 such flows, and as planned by the architects, this was supposed to solve a huge number of problems related to performance, data loss, maintenance costs and the purchase of additional licenses for Cassandra, etc. But which of these expectations were met and which are not - this is the topic of another article.

What test roles were on the project?


Historically, the following roles have been distinguished in testing: a manual tester or a functional tester, a testing automation tool (the one who writes the code and tests), and a testing manager (the one who solves all other issues). Our project was no exception. The project involved:

  • 1 Test Leader or Test Lead
  • 40 manual testers who worked with scrum teams (7 teams)
  • 2 developers (this is rather an exception, because developers do not like to work on automation tasks) and 2 test automation
  • 2 testers who did stress testing
  • 1 functional tester for testing products that was developed by developers and test automation

Manual testing


The main goal of functional testers was to test the application and say whether it meets the requirements of the customer. For this, testers usually:

  1. Create test plans.
  2. Create a test strategy.
  3. Create test cases with scheduled steps (with the expected and current result).
  4. Perform test-cases and conclude on the quality of the application.

We did not have customer requirements or system descriptions. There was only the old system and the new one and the wish: “Make it work, but with a new base.” Therefore, we could only use the like-for-like approach - comparing the results of the old and new systems. Everything was complicated by the fact that we worked with a production system and daily updated data. Unfortunately, we could not start our system at the same time as the production system, and started it later, and this led to the fact that the data in the old and new systems were different.

At this stage, questions began to arise:

  1. How to conclude that our system worked correctly if, due to a time shift, we get results that differ from those that we get in the old system?
  2. But can we get away from data production and work on stable data? What are the risks?
  3. And if we still come to stable data, then how to prove that our system will work correctly with daily updated data?

This is only the tip of the iceberg of the list of similar questions that arose on the project during functional / manual testing.

Test automation


Test automation (and developers) had the goal of writing applications that can test automatically and / or facilitate the work of functional testers:

  • self-tests that would be integrated with CI / CD;
  • applications that helped test functional testers;
  • and other developments that were needed for the internal needs of the project.

On the project, all applications that were developed for testing should have been a separate product that obeyed all the development rules and would have been delivered to the customer as a result. And, accordingly, questions arose:

  1. Who together with the customer will develop non-functional requirements for testing applications? Solution Architect said that this is not their job, as SAs work with the application.
  2. Who will develop the functionality requirements for testing applications? There are obvious requirements, but there are non-obvious ones. For example, do we need to store the logs of our applications and analyze them?
  3. Who will work out the environment for applications with the client and fix these requirements? For example, specifically on this project there was a restriction on spark 1.4, and this was discovered only after creating the application.

And this is also far from all the issues that arise on the project in terms of test automation.

Test Leader, aka Test Lead


Test Lead should organize the testing processes on the project. Its functions included the distribution of tasks, holding internal meetings with testers, organizing work with the customer (finding out details, reviewing the results), etc. That is, he was focused on the current process and solving daily problems / tasks, and not in working out the tasks of the approach, strategy, and testing architecture.

If we are talking about the technical Test Lead, then it was in charge of technical solutions: organizing the development of applications for testing as a process of creating a separate software product that obeys all development rules, for example, creating unit tests, integration with CI / CD processes and etc.

On our project, these roles were played by two people, and the schedule of each looked like this:


In both cases, Test Lead is busy working on an already selected testing strategy, but no one is responsible for choosing the strategy itself, justifying this choice to the client, adapting it to changes and other issues. For instance:

  1. Development of a plan with the client to enter the stage of acceptance testing by the client (UAT).
  2. Studying the requirements with the client when it is considered that the functionality can be transferred to the UAT.
  3. Study with the client of the assumptions of testing on different types of environments (very delicate point). Since the test environment usually does not correspond to production, all issues related to testing on another environment should be worked out with the client.
  4. Study with the client of an acceptable discrepancy of metrics in various types of testing. For example, what result does a client expect to see during performance testing? Maybe it will be enough for him that the system works no worse.
  5. Collecting all possible metrics in numbers, for example, data discrepancies for this table are allowed by no more than 10%, or the script should run no more than eight hours.

What is wrong here?


The questions above are usually laid on the shoulders of Test Lead, but there is a flawed approach:

  1. Test Lead is not expected to work on all such issues with the customer. Most often, an experienced Test Lead understands this and, perhaps, even knows how, but most likely, such an important part as the goal of a business or specific improvements remains beyond its understanding. Accordingly, incorrect priorities may be set.
  2. Test Lead usually enters a project when development is already underway or just starting, i.e. he is faced with the conditions that there are already some agreements or even everything has already been fixed. It is lucky if SA is sufficiently experienced and competent person and will go towards testing - it will help to adapt the initial agreements with the customer for the needs of testing. In another case, the lead has to reinvent the wheel.

And this is not all the problems that fall on Test Lead when there is no Quality Architect.

So who is such a Quality Architect, and why is it needed?


Quality Architect is a person who considers the task from the point of view of business, information and technology, agrees and develops requirements with the customer, draws up a roadmap with deadlines and develops solutions at the interface level.

Projects now dictate other conditions. Projects have become larger, more complex in technical and process terms, can cover several industries and technologies. Testing has become not only a service, but also a supplier (developer) of software for the customer. Therefore, in testing there is a need to highlight a similar role. Moreover, this person should enter the project together with SA, who is involved in the production application, and begin to work on all issues at the same time.

Specifically, on our project, the role of Quality Architect was played by Test Lead, who joined the project 3 months after the start of development, and this entailed a lot of additional work on harmonizing requirements, environments, figuring out the vision of the final test results from the customer. As a result, the project did not meet the deadlines for most of its life, and the customers were unhappy with the supplies and the test result. Not because the work was done poorly, but because the result produced by the team did not meet the expectations of the customer.

When is Quality Architect not needed?


It is clear that such a role is not needed on all projects. Below I propose the simplest way to determine the need for Quality Architect on a project, depending on how the customer sees the testing approach.

The first type of project - the customer gives the formulation of the testing processes at the discretion of the company that he hired.

In this case, Quality Architect has been part of the project since the beginning of work on it, together with SA, it is developing testing requirements, developing a high level testing scheme, deciding what will be automated / developed as separate applications, determining the requirements for these applications, and, of course, agrees with the customer, prioritizes the tasks of the project in accordance with the expectations of the business.

The second type of project - the customer has his own understanding of testing, a clear picture and streamlined processes.

In this case, Quality Architect may not be needed; testing is only required to fit into the existing picture of the world and support it. But if the understanding is superficial, then Quality Architect needs not only to perform all those actions as for the projects of the first type, but also to prove to the customer that this is all necessary.

The third type of project - the customer does not need the services of Quality Architect.

The reasons may be different:

  1. A small and short-term project with simple functionality.
  2. The customer has his own Quality Architect.
  3. The customer refused to test, etc.

Quality Architect Skills


In practice, the practice of decision architecture practice has long been successfully applied. Based on this successful experience, and also considering the volume and complexity of projects, issues that exceed the Test Lead area of ​​responsibility, we can say that highlighting the role of Quality Architect is a natural development of events. Moreover, Quality Architect must be trained in the same processes, techniques and approaches as SA.

In short, in my opinion, Quality Architect should have:

  1. With technical knowledge, both in general and in the domain area where he works, to know the problems of this area, typical errors, problems and ways to check them.
  2. , , , , , .
  3. , .. .
  4. , , , , ..;
  5. : , , CI/CD . .

Based on the foregoing, Quality Architect should possess all the same knowledge as SA, but be focused on the tasks of providing the customer with a quality product. Quality Architect's work is complicated by the fact that often you have to change the customer’s opinion about testing as exclusively about the internal kitchen of the project.

It was precisely the knowledge that I received at the Solution Architecture Mentoring school that allowed me to restore customer confidence, solve many testing issues and successfully complete the delivery of all functionality almost on time on the project that was described above, as well as to establish processes and sign contracts for future work .

Conclusion


The IT industry is developing, new tasks appear (challenges if you want), and in the case of testing, the highlighted position of Quality Architect has become urgent. Naturally, it all depends on the project, but in projects where Quality Architect is needed, you need to be involved in the work at the very initial stage, this will avoid problems in the future and help the development of the project.

Source: https://habr.com/ru/post/undefined/

More articles:


All Articles