Rouwayd Hanna

Test Case Dependency Detection Using Syntactic Analysis of Code for Test Optimization Purposes

Abstract

It is not possible to develop high quality software for large systems without a rigorous testing process. However, testing tends to be costly and time-consuming, which is why research in test optimization has received a great deal of attention. Test optimization is often seen as a multi-criteria decision making problem, where dependencies between test cases are one of the criteria. Since dependent test cases directly influence the execution results of each other, ignoring the dependencies can cause unnecessary test execution failures. Recognizing dependencies and similarities between test cases is beneficial in many aspects of test optimization, such as test minimization and test prioritization.

The dependency information is typically derived from requirements and design artifacts which are not always present in the testing phase. One artifact that is always available during a testing process is the test code that executes the test cases. In this thesis, an approach for automatically detecting test case dependencies by analyzing test code is proposed, applied, and evaluated in the context of an industrial case study at Ericsson AB in Sweden.

The proposed approach involves syntactic analysis of test code to produce abstract syntax trees, which are converted into feature vectors and fed into machine learning models to classify the dependent test cases into clusters. Two clustering algorithms HDBSCAN and K-means were used and their results were compared. The proposed approach was able to detect dependencies using the test code and the best achieved results were obtained when using the HDBSCAN clustering algorithm, yielding an F1 score of $70.7\%$. The approach proposed in this degree project can be used in industrial settings to help testers in identifying dependencies between test cases. Making use of the identified dependencies during the testing process can reduce the risk of unnecessary failures, thus saving time and costs.