Software Development Analytics in Practice: A Systematic Literature Review

Abstract

Software development analytics is a research area concerned with providing insights to improve product deliveries and processes. Many types of studies, data sources and mining methods have been used for that purpose. This systematic literature review aims at providing an aggregate view of the relevant studies on Software Development Analytics in the past decade, with an emphasis on its application in practical settings. Definition and execution of a search string upon several digital libraries, followed by a quality assessment criteria to identify the most relevant papers. On those, we extracted a set of characteristics (study type, data source, study perspective, development life-cycle activities covered, stakeholders, mining methods, and analytics scope) and classified their impact against a taxonomy. Source code repositories, exploratory case studies, and developers are the most common data sources, study types, and stakeholders, respectively. Testers also get moderate attention from researchers. Product managers’ concerns are being addressed frequently and project managers are also present but with less prevalence. Mining methods are rapidly evolving, as reflected in their identified long list. Descriptive statistics are the most usual method followed by correlation analysis. Being software development an important process in every organization, it was unexpected to find that process mining was present in only one study. Most contributions to the software development life cycle were given in the quality dimension. Time management and costs control were less prevalent. The analysis of security aspects is even more reduced, however, evidences suggest it is an increasing topic of concern. Risk management contributions are also scarce. There is a wide improvement margin for software development analytics in practice. For instance, mining and analyzing the activities performed by software developers in their actual workbench, i.e., in their IDEs. Together with mining developers’ behaviors, based on the evidences and trend, in a short term period we expect an increase in the volume of studies related with security and risks management.