A Mutation Analysis Based Model Clone Detector Evaluation Framework
MetadataShow full item record
Model-Driven Engineering is becoming increasingly prevalent and mature. As software projects developed through this methodology age, the need for analysis of Model-Driven projects becomes imperative. One form of analysis is Model Clone Detection, which involves finding similar or identical model fragments in a given context. There are a number of techniques intended for Model Clone Detection and for different types of models. One hindrance to the growth of this field is the ability to objectively and quantitatively compare different model clone detectors and settings of the same detector. In this thesis, our original contribution to knowledge includes a framework utilizing Mutation Analysis to evaluate and compare model clone detectors. It is our proposition that, through distinguishing edit operations on models as mutations, we can create such a framework. In order to demonstrate the plausibility of our framework, we develop a Simulink implementation of the framework. We begin by outlining our initial, qualitative, attempts evaluating our Simulink model clone detector. This includes challenges encountered that are addressed by our framework. We outline the framework and describe each step in its process in an example-driven manner through creation of a framework prototype that works on Simulink model clone detectors. We choose Simulink because it is the most mature form of Model Clone Detection, it is of interest to our industrial partners, and we previously created a Simulink model clone detector. An additional contribution is a taxonomy of Simulink model mutations intended to inject the various types of model clones, while still being representative of realistic Simulink model evolution, which we verify through a case study. We run our Simulink framework prototype on leading Simulink clone detectors to ascertain their recall and precision. We observe high recall for Simone, lower recall for ConQAT because it is intended for only a subset of clone types, and high precision for both tools. It is our hope that having such a framework in place will help facilitate gains in Model Clone Detection research as engineers in this area can now refine their own tools and new detectors can be compared against existing ones.