Get best of the week

Is TDD effective?

During an interesting discussion, one very respected person “trumped” the “indestructible” argument:
There are many studies demonstrating the effectiveness of TDD.

Really. If you go to Google Scholar, score the keywords “TDD” and “Efficiency” - there will be many scientific articles , but is it that simple? Although I myself am a big fan of TDD, I also consider myself a skeptic, and decided to check whether it has been scientifically proven that TDD is so cool.

I find your lack of skepticism disturbing

Test-driven development (TDD) has been with us for a long time, and all this time it has been the subject of heated debate. The essence of TDD is that you need to start development by writing a falling Unit test, after which code is written that will make this test “green”, and so on in a loop. For those who are not up to date, you can see the TDD guide in the IDE Idea .

Some developers are confident that TDD improves efficiency and quality by at least 146%. Some developers know for sure that TDD is complete nonsense, and writing tests before writing code is like skiing on asphalt. A compromise between these development groups is not worth the wait.

But back to science. By a lucky coincidence, it so happened that I was studying at the Wrexham Glyndŵr University Master's program in Computer Science with Big Data Analytics (by the way - I recommend it to everyone, and I will write about it separately) and right now I'm going through the Critical Research module, where they teach you to read scientific articles and analyze their validity.
What is our evidence?

There is indeed a lot of research on the effectiveness and usefulness of TDD. However, if you read the studies themselves, you will notice that they come to conflicting conclusions.

For example, in the report “An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development”, published in 2007, it was concluded that TDD improves the productivity of developers without affecting the quality [1].

In another report, which examined the findings of several other studies and conducted its own study, “Evaluating the Effectiveness of Test Driven Development: Advantages and Pitfalls”, 2017 concluded that TDD can improve product quality, but leads to lower productivity. Additionally, it has been observed that most developers find TDD unnatural [2].

According to another study, “A Replicated Experiment on the Effectiveness of Test-First Development”, 2013, researchers were not able to find a convincing and significant effect of Test-First or Test-Last approaches on developer productivity or product quality [3].

To hammer a nail into the coffin of “evidence” of the effectiveness and usefulness of TDD, you can read the overview report “Overview of the Test Driven Development Research Projects and Experiments”, 2012 [4], in which, based on a large number of available studies, they tried to come to a conclusion about the evidence efficiency and positive effect of TDD. The authors concluded that there is no convincing evidence of the beneficial effects of TDD. Differences in the results of previous studies, the authors substantiated Confirmation bias - studies that showed the effectiveness of TDD were originally based on the hypothesis that TDD is effective.
So why do people (including me) consider TDD a great practice?
After reading the research, I have only one logical answer - the effectiveness and applicability of TDD depends primarily on the particular developer. Studies on the effectiveness of TDD, apparently, have not sufficiently taken into account that developers are not robots, and TDD is not a program. For some developers, TDD will bring good results, for some - there will be no significant difference between TDD and simple writing Unit tests, for some - TDD will lead to a drop in quality, productivity and motivation.

Programming is social activity. The problem with social interactions is that it is not an exact science. It remains to figure out how to conduct a more or less scientific experiment to confirm / refute this hypothesis. To begin, I offer a survey.

If you suddenly want to share this publication with your English speaking colleagues, there is a slightly more formal English version: medium.com/@alexspush/is-tdd-effective-2474c0471ac4

List of sources

[1] A. Gupta, P. Jalote “An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development ”presented at First International Symposium on Empirical Software Engineering and Measurement, Madrid, Spain, 2007
[2] Z. Khanam, M. Ahsan,“ Evaluating the Effectiveness of Test Driven Development: Advantages and Pitfalls ” International Journal of Applied Engineering Research vol. 12, no 18, p. 7705, 2017. Available: www.ripublication.com/ijaer17/ijaerv12n18_81.pdf . [Accessed May 13, 2020].
[3] D. Fucci, B. Turhan, “A Replicated Experiment on the Effectiveness of Test-first Development” presented at ACM / IEEE International Symposium on Empirical Software Engineering and Measurement, Baltimore, MD, USA, 2013
[4] A. Bulajic, S. Sambasivam and R. Stojic “Overview of the Test Driven Development Research Projects and Experiments” presented at Proceedings of Informing Science & IT Education Conference (InSITE), 2012. Available: pdfs.semanticscholar.org/3278/971bb53a25822171df127d47a57243dd3bbd.pdf . [Accessed May 13, 2020].

More articles:


All Articles