Consider the following hypothetical report in experimental software engineering:
Professor Walia has developed a new programming technique, “Suspect-Oriented Programming” (SOP). To evaluate SOP, he directs half of the students in his “Software Methodology” class to do the project using traditional techniques, and the others to use SOP. He finds that projects by the students using SOP have, on the average, 15% fewer bugs than the others, and reports that SOP increases software reliability.
What’s wrong with this story?
Provide at least 4 reasons as to why Professor Walia’s attempt at empirical software engineering is problematic.
