Design, Implementation, and Validation of Zorro for Test-Driven Development
Test Driven Development (TDD) can be summarized as two basic rules: (1) Write new code only if an automated test has failed; and (2) Eliminate duplication. TDD is becoming increasing popular in industry and is a core feature of many agile software processes. Millions of dollars have been spent converting to agile software processes in general and TDD in particular. Unfortunately, many of the claimed benefits of TDD are based on anecdotal evidence rather than rigorous scientific research. The controlled research findings, thus far, are mixed and sometimes contradictory. Perhaps the claimed benefits of TDD are simply due to increased testing, as opposed to the “test first” strategy? If same amount of tests are created, is TDD really better than other development methods? Just how important is it to rigorously follow the test-first philosophy of TDD?
To make headway into this important question for the software engineering community, it is necessary to find some way to rigorously identify TDD behaviors during development, as opposed to other, similar development methods such as Iterative Test Last (ITL). This can be thought of as a “process conformance” problem. In this thesis, I implemented the Zorro software system to infer process conformance of TDD automatically using a rule-based system.
In the course of my research, I designed a pilot study, a classroom case study and an external industrial case study to evaluate Zorro’s ability to collect development activities and use them to infer the presence of TDD behaviors. The initial pilot study concluded that Zorro recognized TDD development correctly 88% of the time. I then enhanced the system to improve Zorro’s inference capability and and its accuracy in real-world situations. In the classroom case study, both participant observation using video recording and cross-validation from participants themselves provided evidence that Zorro can identify TDD when it occurs. The external industrial case study was conducted by a researcher from Norway who compared TDD against Test-Last using the assistance of Zorro. The researcher found that Zorro is useful for conducting empirical research of TDD with only minimal data collection overhead.
In addition, my research resulted in a more general framework for “micro process” identification named SDSA (Software Development Stream Analysis). SDSA has the potential to facilitate the study of broad classes of software development behaviors.
Friday, September 7, 2007
POST 302
2:00-4:00pm
