Test Cost-Effectiveness and Defect Density: A Case Study on the Android Platorm
The Android operating system is one of the most popular open-source platforms in the mobile operating system market. It had a worldwide smart-phone market share of 68% at the second quarter of 2012. However, there has been little research on test coverage and test cost-effectiveness in this platform. The goal of the case study to be presented in the presentation is to assess test coverage, fault detection effectiveness, test cost-effectiveness and defect density in code-base of version 2.1 of the Android platform. We raise and address five research questions (RQs) in this study. Among our results are: (1) in contrary to what one would expect, for packages with larger coverage values (meaning more rigorous testing), it is not necessarily true that less defects have been reported by the users after release. Also, it is not necessarily true that components with low coverage have more defects; (2) We re-confirm (replicate) the existence of correlation between code coverage and mutation score, similar to existing studies; and (3) The package with the highest defect density (DD) in the Android code-base is Music (DD=0.19 per 1 KLOC) and the package with the lowest DD value is ContactsProvider (DD=0.0003). Results of our study will help us and other researchers to get a better view on test coverage, fault detection effectiveness, test cost-effectiveness and defect density in Android code-base.
Vahid Garousi is an Associate Professor at the University of Calgary, Canada and a Visiting Associate Professor in the Middle East Technical University (METU) in Ankara, Turkey. His research expertise include: software engineering (in general), software testing and quality assurance, software maintenance and model-driven development. He has been involved as an organizing or program committee member in many software engineering conferences such as ICST, ICSP, CSEE&T, and MoDELS. He is a member of the IEEE and the IEEE Computer Society, and is also a licensed professional engineer (PEng) in the Canadian province of Alberta. He has been recently selected a Distinguished Visitor (speaker) for the IEEE Computer Society’s Distinguished Visitors Program (DVP) for the period of 2012-2015. Among his awards is the Alberta Ingenuity New Faculty Award in June 2007. For more information, visit: www.ucalgary.ca/~vgarousi