10 Open Source Automation Tools For Your Automation Needs

Introduction

Test automation services has proven to increase the speed, quality, and efficiency of the software in many ways. Listed below are top 11 open source tools for your automation needs.

Selenium

Selenium was originally developed by Jason Huggins in 2004 at ThoughtWorks. It is a software testing framework which can be used to test the web applications. It provides driver to automate web applications across different browsers.

Pros:

• Selenium reduces execution time since it supports parallel execution. 
• Selenium supports different operating systems such as Windows, Linux, and Mac OS X. 
• Selenium supports various scripting languages such as Java, C#, Python, Ruby, Perl, PHP, Kotlin and JavaScript. 
• Selenium uses fewer hardware resources. 

Cons:

• Selenium cannot be used to test desktop-based applications. 
• Selenium provides less support for image testing. 
• Selenium doesn’t have a built-in reporting facility. 
• Since it is open-source software, nobody provides reliable support. 
• Selenium does not provide any built-in IDE for script generation, it needs other IDE like Eclipse for writing scripts. 

Appium

Dan Cuellar originally developed Appium in 2011. It is a mobile automation testing framework that supports native, hybrid, web application and cross-platform testing. It allows testing on physical devices, emulators and simulators

Pros:

• Appium supports both Android and iOS. It translates all selenium commands into XCUITest (iOS) if the MAC OS version exceeds iOS 9.3 or UIAutomator (Android), depending on the device type. 
•  Appium supports scripting languages such as Java, Objective-C, JavaScript with node.js, PHP, Ruby, Python, C#, etc. 
• It supports Android and IOS Native apps, Mobile Web, and Hybrid applications. 
• Community support is available. 
• Appium supports both physical devices and Simulator/Emulator testing. 

Cons:

• Some of the gesture supports are not available 
• Appium does not support the android version lower than 4.2. 
• Appium does not support image comparison. 

Robotium 

Robotium was introduced and developed by Renas Reda. It is an Android test automation framework which provides full support for native and hybrid applications. Test scripts are written using Java. It can be used to write functional, system and user acceptance test cases. 

Pros: 

• It has a simple API, so less time is required to write test cases.  
• It can be easily integrated with Maven and ANT. 
• It allows us to take screenshots during test execution and save them in internal memory. 
• Robotium recorder allows the tester to generate a test case quickly. 

Cons: 

• Robotium can’t test flash components. 
• Tied to one app process. 

Robot framework 

The Robot framework was introduced and developed by Pekka Klarck. It is a generic automation testing framework for acceptance testing and test-driven development. It follows a keyword-driven approach. 

Pros: 

• Robot framework is an application and operating system independent.  
• Robot framework test cases can be written using the natural language itself. A test report gets generated once the execution is complete. Reports are available in a HTML/XML format. 

Cons: 

• Robot framework doesn’t support parallel execution directly. 

Cucumber 

Cucumber is a testing framework based on Behaviour Driven Development (BDD). It uses a simple grammar defined by a language called “Gherkin” to define application behaviour. This framework is written using Ruby. It writes its test scenarios in Given/When/Then format. This scenario file is called a feature file, which is the entry point for any cucumber tests. 

Pros: 

• Cucumber supports scripting languages like Java, Perl, PHP, Python and DotNet. 
• Cucumber is available as a plugin in eclipse software. 
• Cucumber focuses on end-user experience. 
• Cucumber is a behavior-driven development tool. 

Cons: 

• The Cucumber framework may lead to step definition redundancy if not properly followed, which will lead to unnecessary or repeated lines of code. 

Sikuli

Sikuli was introduced by Raimund Hocke (aka RaiMan) in 2011. It is an automation tool used to test the GUI using the “Visual Image Match” method. It identifies the image using Image Recognition. This tool is preferred when there is no access to the internal source code of the application.   

     Pros: 

• Sikuli allows automating flash objects. 
• Picture recognition using Sikuli is accurate. 
• Sikuli can automate window applications as well as web applications.  

     Cons: 

• Sikuli is not preferred when UI elements are constantly changing. 
• Sikuli scripting is platform-dependent. 
• Sikuli is resolution-dependent, meaning a specific resolution script won’t work for other resolutions. 

Espresso  

Espresso is a testing framework for android to write reliable test cases for mobile application user interfaces. This framework mostly suits white box automation test cases where the test can directly interact with the application under test implementation code. It has major three components. 

• ViewMatchers  
• ViewActions 
• ViewAssertions 

Pros: 

• Easy to use. 
• Expresso gives developers fast feedback on their code changes. 
• The Stability of the expresso tests is very high. 
• Development of espresso test cases is easy. 

Cons: 

• It is only for UI testing. 
• It supports only Android UI. 
• You require access to the app’s source code, which needs to be tested with espresso. 

JUnit/NUnit 

JUnit is an open-source testing framework. As its name indicates, this framework is for performing a unit test for a small unit of code. It identifies each test methods using @Test annotation. It provides assertions class for checking expected results. NUnit is open source unit testing framework for .Net applications. It provides services as JUnit provides for the java applications.

Pros: 

• JUnit can be easily integrated with Ant and Maven. 

Cons: 

• JUnit cannot do dependency testing. 
• It is not suitable for larger test suite creations. 

TESTNG 

TestNG is introduced by Cedric Beust. It is an automation testing framework in which NG stands for “Next Generation”. It provides a detailed report which provides information about passed, failed and skipped test case details. This framework is inspired by JUnit framework. This framework is better than JUnit framework. 

Pros: 

• TestNG supports parallel execution.  
• TestNG allows us to create dependent tests, prioritize tests, and group the tests.   
• With the help of @DataProvider annotation, data-driven approach can be achieved through TestNG framework. 
• TestNG generates logs. 
• TestNG produces HTML reports. 

Cons: 

• TestNG supports only Java. 

FlaUI: 

FlaUI is developed as part of the open-source community. It is a framework designed to support almost all types of Windows applications. It is based on native UI Automation libraries from Microsoft and, therefore, a wrapper around them. It is a .NET based open-source library which internally consumes Microsoft’s UI Automation libraries and supports C# programming language for writing test scripts. It can be downloaded from NuGet package manager available in Visual Studio. FLAUI can be easily integrated with unit testing frameworks such as Nunit, Xunit and MSTest. This framework Supports well with BDD framework, spec flow. 

Pros: 

• It is compatible with BDD framework. 
• Allows for using any version of libraries UIA: MSAA, UIA2, or UIA3. 
• Supports Windows 7, 8, and 10 (likely XP too, but we have not checked). 
• Good community: helps solve problems not only in issues but also in chats. 
• It does not require running an external service (unlike WinAppDriver, which requires winappdriver.exe to be running). 
• Faster and more reliable compared to other tools. 
• Reduce efforts of writing xpath through its locator strategy to find specific or all child/descendants. 
• Manages to get window focus on maximum cases when minimized or overlapped. 
• Easy to handle windows/components inside and outside the application through their built-in commands. 

Cons: 

• Debugging specific UI components could be challenging as the application may not come into focus. 
• Application may not withstand the tool’s speed(waits should be used intelligently to control the flow). 
• Few of the grid components won’t be accessible if the developers do not specify a default action. 
• Visual studio 2017 or above versions are required to support properly with the BDD tool (Specflow). 

Playwright: 

Playwright enables reliable end-to-end testing for modern web apps. It is an open-source NodeJS based framework. It was built by the same team which created Puppeteer (Headless Browser Testing Framework for Google Chrome). Though built on the node JS framework, it can also support other programming languages. It supports multiple languages using their native IDE or adding respective extensions in VS code. It Supports both TDD and BDD frameworks. Easy to Install and less maintenance required. 

Pros: 

• Open source and free of cost. 
• Has inbuilt test framework (@playwright/test). 
• Supports multiple languages such as JavaScript, Java, Python, and .NET C#. 

• Supports multiple OS such as Windows, Linux, and Mac OS. 
• It can be integrated with major CI/CD servers such as Jenkins, CircleCI, Azure Pipeline, Dockers, TravisCI, etc. 
• It provides wider coverage for cross-browser testing by supporting Chrome, Firefox, and WebKit. 
• Supports parallel testing. 
• Supports API testing by getting access to the REST API of the application. 

• Supports android web testing. 
• Supports both headless and headful modes. 
• Playwright automatic waits help to eliminate flaky tests. 
• Playwright assertions are created specifically for the dynamic web. 
• Wide coverage, accuracy, and high speed. 

Cons: 

• Iteration of web elements is not straightforward (as it is strictly based on index, foreach can’t be used). 
• Does not support real android  devices but supports emulators for mobile testing. 
• Since Playwright is new, the community’s support is limited compared to Selenium. 
• Running a test in multiple threads (parallel execution) might cause the app to freeze. 

• Multiple locator strategies might confuse the beginners on choosing the right one.