Get best of the week

Book “How to test on Google” - free electronic version

imageHello, habrozhiteli!

The book describes testing software products at Google: how processes are organized, how teams are organized, what techniques are used, who is responsible for quality. The principles that Google testing is built on apply to projects and companies of any size. The authors of the book themselves worked on Google products, creating testing tools, customizing processes and doing testing directly.

The book is intended for professionals from the software development industry: testing specialists, programmers, managers.

Excerpt. Risk reduction


It is rarely possible to completely eliminate the risks. We drive a car, although this is dangerous, but do we need to get to work? In general, the possibility of an accident does not mean that it will happen, and, most likely, nothing terrible will happen. Why? Because by our actions we reduce the possible risk. For example, we do not drive while intoxicated and do not drive in conditions of insufficient visibility. Thus, we reduce risks.

In software product development, the simplest thing is to avoid risky areas: the less code, the less risk. But besides using the “ax and ax”, we can do a lot more to reduce the risks:

  • , , .
  • -, , .
  • , .
  • .
  • , . , .

The specific solution depends on the features of the application, on the expectations of the user regarding its safety and reliability. As testers, we, of course, can be involved in the process of risk reduction, but we are certainly involved in the process of identifying them. We start by prioritizing the features marked in red in the table. We want to test in order to reduce risks. This is important: if you can’t test everything, first test the most important thing. And the most important thing is that it is most exposed to the most serious risks.

In some projects, it is testers who ask about the readiness of the product for release. It is enough for a good tester to glance at the heat map to determine whether it is still worth holding the product in the oven or it's time to serve it on the table. If we are talking about launching an experimental Google Labs, then the presence of red risk zones is not so significant, if they are not related to security, of course. And if this is the release of a new version of Gmail, then even the yellow zones are a serious danger. Such a simple color gradation is clear to everyone, even top managers.

Concerns about risks subside over time, and the large volume of successful testing is a good sign that the risks are at an acceptable level. Here we benefit from linking test cases to individual product features, and then to attributes and components in the risk table. “ACC analysis” is ideal for this case, and that’s why we created this tool just like that.

James Whittaker Prescription Test Plan in Ten Minutes


, , . , , — ? , , . Google, , -. , , : «», « » — « » ( ). ’, , - , .

— . -, , — , , ( ), , , . : , ,
?

- , , , . , — . : ?

- , - . . : -.

, : - . - , .

, . : . , , .
-, .

. : « , - ». , , . , , , , .

, , , (Google Docs, App Engine, Talk Video . .), .

, ACC-. , . — , — . , . - — . , , .

’, - . . ,
- .

, . , . 80% . ? , , ? , (, , ) . , , .

. -!


Google Test Analytics takes as a basis the risk assessment criteria described above ("very rare", "rarely", "sometimes", "often"). We specifically do not want to turn risk analysis into a difficult task, otherwise it will not be completed. We are not interested in the exact mathematical details, because the numbers mean little. It is enough to know that “A” is more risky than “B”, not paying attention to the exact significance of the risks. Simple knowledge of which opportunity is riskier than another will allow the test manager to more efficiently distribute the work of testers. And people like Patrick Copeland can easily decide how many testers to assign to each development team. Understanding the risks benefits the entire company.

Risk analysis is an independent scientific field respected in many industries. We use a simplified version of the methodology, but this does not prevent us from becoming interested in new research in order to improve our approach to testing. If you want to learn more about risk analysis, start with the article “Risk Management” on Wikipedia.

GTA helps identify risks, and testing helps reduce them. The tester serves as an intermediary in this process. He can perform internal tests in some of the most risky areas or task developers and developers in testing so that they add regression tests. There are other tools in his arsenal: research testing, attracting internal and beta users and the strength of the external community.

It is the responsibility of the tester to know all areas at risk. He should try to reduce risks in any way that is subject to him. Here are a few recommendations that we find useful in dealing with risks.

  1. For the most risky features and attribute / component pairs marked in red, write a set of user stories, use cases, or a test guide. At Google, the responsibility for the most risky opportunities lies with the tester. He can coordinate his work with colleagues, use different tools, but personal responsibility is still on him.
  2. , . , GTA? ? ? . , , .
  3. , / , , . .
  4. — . , . , . , : «!», . , .
  5. , . , , . , « ?» « ?». Google , , .
  6. 6 , , , . ! .




, . , , , .

, , - . - , , , . , , . , , .

, , . -, - — !

— . - . — . Google , . -: Google Documents — , .

, , , . — .

We will not be too faulty with low-risk opportunities. We may decide that writing test cases for these areas is too expensive. Instead, we can limit ourselves to research testing or to crowdsource testing. To manage the work of testers from the external community, we often use the concept of tours - these are high-level instructions for research testing. Simply put, this approach gives your request the specifics you need. For example, asking the community: “Have a FedEx tour for such a set of features” - we will get a much better result than just giving the application and hoping for the best. We immediately determine the features that need to be tested, and give instructions on how to do this.

Crowdsourcing




— . , , - ! , . . ?

, , . , , — , , -. , Chromium, . , , . , .

( ) — . , , . , , ? — .

, , , . , : -1000 , : Chrome, : 1 = 1000 20 = 50 . .

, , , . , , . Chrome, , , ( « Chrome»). . , . « , » , .

- — Google: , , . , . , , , (, uTest). .

So, the strength of ACC analysis is that we get a list of product features that can be sorted by risk and assigned to different performers. Testers working on the same project may receive different sets of test capabilities. Internal users, “twenty percent” participants, testers, community testers, developers, developers in testing will all receive their lists of features, and, to the tester’s joy, important areas will be covered with less overlap than if we just handed out the application for testing for everyone.

The work of the tester, in contrast to the work of the developer in testing, does not end with the inlet of the product.

» Download epub and pdf

More articles:


All Articles