Students versus professionals as experiment subjects:an investigation on the effectiveness of TDD on code quality

• Main Author: Salman, I. (Iflaah)
• Format: Dissertation
• Language: English
• Published: University of Oulu 2015
• Subjects: Information Processing Science
• Online Access: http://urn.fi/URN:NBN:fi:oulu-201504021273; http://nbn-resolving.de/urn:nbn:fi:oulu-201504021273

Background: Most of the software engineering empirical studies use students as subjects for conducting experiments. This raises the concern that whether results acquired through students are applicable to the industry or not. The researchers argue that experiment with students lack realism. This situation not only causes the threats to the generalizability of the research results but also becomes a potential barrier to the adoption of novice research in industry. Aim: The objective of the study is to investigate, whether students are representative of professionals in software engineering experiments. We are addressing the objective by investigating the difference in the code quality of the two subject groups — students and professionals — in the context of the effectiveness of Test Driven Development (TDD) on the internal code quality. The study tests two hypotheses; the first one aims to test the difference in the code quality of the two subject groups. The second one aims to test the difference in the code quality of tasks implemented following TDD and Test-Last Development (TLD) methodologies. Method: The study follows a quantitative research approach with the experimentation methodology. The study involved graduate students from academia and professionals from the industry in their respective environmental settings. Both of the subject groups implemented tasks in TDD and TLD approaches. The treatments of the first hypothesis are professionals and students for studying the difference in the two subject groups. Furthermore, TDD and TLD are the treatments of the second hypothesis for assessing the difference in the code quality of the two methodologies. Results: Students differed professionals when compared for TLD and TDD1 implementations. The results couldn’t show the difference for TDD2 task. The code quality differed as an effect of TDD for professionals in the cases of TLD vs. TDD1 and TLD vs. TDD2. Students’ data showed the difference in the code quality for TLD vs. TDD1 but couldn’t for TLD vs. TDD2 tasks. The null hypotheses were refuted with the p-value < 0.05. Conclusion: The results of the study are confined to the context of TDD and to some extent with the experimental design. The future extension can be to compare the two subject groups by designing the same experiment. Additionally, comparison can also be studied in other areas of software engineering for a better generalizability of the comparison results.