Effective and Appropriate Use of Controlled Experimentation in Software Development Research
Although there is a large body of research and publication on software development, only a limited amount of this work includes empirical demonstration of its effectiveness. Yet, it is this empirical understanding which will help move software development from a craft to an engineering discipline. O...
| Main Author: | |
|---|---|
| Format: | Others |
| Published: |
PDXScholar
1996
|
| Subjects: | |
| Online Access: | https://pdxscholar.library.pdx.edu/open_access_etds/5289 https://pdxscholar.library.pdx.edu/cgi/viewcontent.cgi?article=6361&context=open_access_etds |
| Summary: | Although there is a large body of research and publication on software development, only a limited amount of this work includes empirical demonstration of its effectiveness. Yet, it is this empirical understanding which will help move software development from a craft to an engineering discipline. Of the empirical methods for research, controlled experiments are the most commonly thought of in scientific studies, and yet the least used to study software development. This thesis begins with a brief review of the different empirical methods commonly used to study software development. This review provides a quick introduction to each empirical method, compares the main advantages and weaknesses of each method, and provides a context for how controlled experimentation compares to other empirical methods for studying software development. Using empirical methods to study software development is not easy or straightforward. There are limitations which appear to be inherent in the nature of software and issues due to the improper understanding or application of empirical methods. These limitations and issues are identified, specifically for controlled experiments, and approaches for dealing with them are proposed. A controlled experiment was designed and conducted to demonstrate the method and explore the limitations and issues for empirical research in software development. This experiment and its results are presented. This example experiment demonstrates that conducting even a simple experiment in software development is challenging. Lessons learned from this experience are reported. Controlled experiments require that the researcher have a high degree of control over the environment where the experiment is carried out. This can be costly and difficult to achieve. This thesis concludes by discussing how controlled experiments can be used effectively in studies of software development. |
|---|