An investigation of entropy and refactoring in software evolution

As software evolves, the source code tends to become more complex and therefore harder to maintain, something that is exacerbated by poor development practice, where a disorderly development process is more likely to yield a disorderly and more complex result. Disorder in source code evolution may be quantified by using source code change entropy, a measure of the scattering pattern of code changes, i.e., how widely spread throughout the codebase are individual changes. Refactoring is an important activity for improving the structure of existing code and reducing its complexity, but it is unclear if refactoring can effectively counteract software entropy. Understanding how or if refactoring can reduce entropy could provide insights on software evolution and help reduce maintenance cost. We empirically investigate how entropy at the system and file level evolves in software systems and how it is influenced by refactoring activities as well as non-refactoring changes. We identify refactorings in the version history of open-source projects through the analysis of source code and change metadata. We observe that system-level entropy fluctuates up and down, but remains relatively stable over time on average. We also observe that file-level entropy tends to decrease over time in response to both refactorings and non-refactoring changes, challenging the findings of previous studies. We observe factors which may lessen the utility of existing entropy metrics and suggest future avenues for exploring this intriguing but little-studied concept in software evolution.