Findings from a multi-method study on test-driven development

Abstract

Context: Test-driven development (TDD) is an iterative software development practice where unit tests are de ned before production code. A number of quantitative empirical investigations have been conducted about the practice. The results are somehow contrasting and inconclusive. In addition, previous studies fail to analyze the values, beliefs, and assumptions that inform and shape TDD. Objective: In this paper, we present a study designed, and conducted to understand the values, beliefs, and assumptions of TDD. Participants in the study were novice and professional software developers. Method: We conducted an ethnographically-informed study with 14 novice software developers, i.e., graduate students in Computer Science at the University of Basilicata, and six professional software developers (with one to 10 years work experience). The participants worked on the implementation of a new feature for an existing software written in Java. We immersed ourselves in the context of our study. We collected qualitative information by means of audio recordings, contemporaneous eld notes, and other kinds of artifacts. We collected quantitative data from the integrated development environment to support or refute the ethnography results.Results: The main insights of our study can be summarized as follows: (i) refactoring (one of the phases of TDD) is not performed as often as the process requires and it is considered less important than other phases, (ii) the most important phase is implementation, (iii) unit tests are almost never upto- date, and (iv) participants rst build in their mind a sort of model of the source code to be implemented and only then write test cases. The analysis of the quantitative data supported the following qualitative ndings: (i), (iii), and (iv). Conclusions: Developers write quick-and-dirty production code to pass the tests, do not update their tests often, and ignore refactoring.