Investigating Selenium Usage Challenges and Reducing the Performance Overhead of Selenium-based Load Tests
Shariff, Shahnaz Mohammedi
MetadataShow full item record
Selenium automates the testing of web applications. Quality Assurance (QA) engineers usually write Selenium scripts to test the content that is rendered in a browser and simulate user interactions with the web content for web applications. Apart from test automation, Selenium is also used to perform load testing and web scraping tasks. Selenium has gained a great amount of attention from users as observed from the trend of Selenium-tagged questions on StackOverflow. In this thesis, we first explore the Selenium-related questions on StackOverflow to understand users' questions about Selenium, programming-language specific issues, browser-specific concerns and the difficult aspects of Selenium. Some of the notable findings from our empirical study are: Questions about Python have become the most frequent and the fastest growing programming language related questions in Selenium, while questions on Java have become stable in recent years. We also find that Chrome tag is the most frequently used browser tag among other browser related tags associated with Selenium. Several users discuss performance issues in Selenium tests on StackOverflow. Prior work on Selenium also points out that Selenium tests are resource intensive. As Selenium needs to launch a browser for each instance (e.g., a client) of a test, it is usually very resource consuming to run a load test using Selenium as one would need to run thousands of user instances. We propose an approach to reduce the performance overhead of Selenium-based load tests. Our approach shares browser instances between user instances, thereby reducing the performance overhead that is introduced by launching many browser instances. The main contribution of this thesis is that our approach can significantly increase the number of user instances that can be tested on a test driver machine without overloading the machine. We improve the testing efficiency of Selenium-based load tests by at least 20%. The insights that we share in this work can help the developers of Selenium understand how Selenium is used in practice and how they can offer better support for Selenium. Further, software practitioners can benefit from our approach to reduce the performance overhead of Selenium-based load tests.