testing starter kit

Collection of free open source software testing tools with some links on how to use them

Marketionist
16
3
JavaScript

testing-starter-kit

QA Engineers and Developers now have a wide variety of free open source tools for testing websites and portals, but it can be a challenging task to find them all and make informed decision on what and how to use.

This is a collection of free open source software testing tools with some links on how to use them.

The scope of this particular list is limited to things that Test Automation Engineers, Software Development Engineers in Test (SDET), Web Developers, etc. are likely to find useful.

You can help by sending Pull Requests to add more tools and links to “how to use” instructions.

We love all the free things out there, but it would be good to keep it on topic. It’s a bit of a grey line sometimes, so this is a bit opinionated.

Table of Contents

Code Editors and IDE

  • VS Code: initial release date: April 29, 2015, written in: TypeScript, JavaScript, Cascading Style Sheets (CSS). Visual Studio Code is a source code editor developed by Microsoft for Windows, Linux and macOS. It includes support for debugging, embedded Git control, syntax highlighting, intelligent code completion, snippets and code refactoring. Plugins and extensions for it can be written in JavaScript (a lot of them are built and maintained by community).

  • Sublime Text: initial release date: January 18, 2008, written in: C++, Python. Sublime Text is a cross-platform source code editor. It natively supports many programming and markup languages. Plugins and extensions for it can be written in Python (tons of them can be installed and used).

  • IntelliJ IDEA: initial release date: January 2001, written in: Java. IntelliJ IDEA is a Java integrated development environment for developing computer software. It is developed by JetBrains and is available as an Apache 2 Licensed community edition and in a proprietary commercial edition. Both can be used for commercial development.

  • CodePen: initial release date: 2012. CodePen is an online community for testing and showcasing user-created HTML, CSS and JavaScript code snippets. It functions as an online code editor and open-source learning environment, where developers can create code snippets, creatively named “pens”, and test them.

Automated Browser Testing (end-to-end)

Testing pyramid by CircleCI

  • Selenium: initial release date: 2004, written in: Java. Selenium automates browsers. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that. Selenium has the support of some of the largest browser vendors who have taken (or are taking) steps to make Selenium a native part of their browser. It is also the core technology in countless other browser automation tools, APIs and frameworks. Selenium has 2 incarnations:

    • Selenium IDE: is more simple and fast way to create automated testing scripts by clicking and recording your actions with record-and-playback Firefox add-on. It is mostly used to create simple scripts to aid in automation-aided exploratory testing. It requires some knowledge of HTML.
    • Selenium WebDriver: is more sophisticated and powerful tool to create robust, browser-based regression automation suites and tests. It requires some knowledge of programming languages - can be used one of the following: Java, JavaScript, Python, Ruby, C#.

  • RSpec: initial release date: August 2005, written in: Ruby. RSpec is a ‘Domain Specific Language’ testing tool written in Ruby to test Ruby code. It is a behavior-driven development framework which is extensively used in the production applications. It utilizes easy-to-read syntax (describe, it). Here is a free, once-weekly e-mail on how to use Selenium with RSpec (also see the full archive of useful tips).

  • Protractor: initial release date: Jun 12, 2013, written in: JavaScript. Protractor is an end-to-end test framework for Angular and AngularJS applications (can be used for non-Angular websites too). Protractor runs tests against your application running in a real browser, interacting with it as a user would.

    Note: the Angular team announced that Protractor was officially sunsetted at the end of 2022.

  • Nightwatch: initial release date: January, 2014, written in: JavaScript. Nightwatch was created as a tool to enable writing of automated UI tests in a straightforward way with as little configuration and additional libraries as possible.

  • WebdriverIO: initial release date: May 17, 2012, written in: JavaScript. WebdriverIO is a test automation framework that allows you to run tests based on the Webdriver protocol and Appium automation technology. It provides support for your favorite BDD/TDD test framework and will run your tests locally or in the cloud using Sauce Labs, BrowserStack or TestingBot.

    Useful links:

  • Puppeteer: initial release date: August 16, 2017, written in: JavaScript. Puppeteer is a Node.js library maintained by Chrome’s development team from Google. Puppeteer provides a high-level API to control Chrome or Chromium (by default headless, but can be configured to run full non-headless Chrome or Chromium) or interact with the DevTools protocol.

    Useful links:

  • TestCafe: initial release date: October 17, 2016, written in: JavaScript. TestCafe is Node.js tool to automate end-to-end web testing. TestCafe runs on Windows, MacOS, Linux and supports desktop, mobile, remote and cloud browsers (UI or headless).

    Useful links:

  • Cypress: initial release date: September 10, 2017, written in: JavaScript. Cypress is a next generation front-end testing tool built for the modern web. It is most often compared to Selenium, however Cypress is both fundamentally and architecturally different. Cypress is not constrained by the same restrictions as Selenium. This enables you to write faster, easier and more reliable tests. At the same time Cypress has a set of limitations/trade-offs.

  • Playwright: initial release date: 31 January, 2020, written in: JavaScript. Playwright is a framework for Web
    Testing and Automation. It allows testing Chromium, Firefox and WebKit with a
    single API. Playwright is built to enable cross-browser web automation that is
    ever-green, capable, reliable and fast.

    Useful links:

Useful links:

API Testing

  • cURL: initial release date: 1997, written in: C. The name stands for “Client URL”. cURL is providing a library and command-line toolfor transferring data using various protocols. Installing and using cURL on Mac OS:

    • /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
    • brew install curl
    • curl http://localhost:3000 # Server will receive a GET HTTP request
    • curl -X POST http://localhost:3000 # Server will receive a POST HTTP request
    • curl -X PUT http://localhost:3000 # Server will receive a PUT HTTP request
    • curl -X DELETE http://localhost:3000 # Server will receive a DELETE HTTP request

  • Postman: initial release date: February 19, 2016, written in: JavaScript. Postman is a powerful API testing suite which has become a must-have tool for many developers. It has the ability to make various types of HTTP requests, i.e. GET, POST, PUT, PATCH. It is available for Windows, macOS, and Linux. To query an API endpoint, you’ll need to do the following steps:

    1. Enter the URL that you want to query in the URL bar in the top section.
    2. Select the HTTP method on the left of the URL bar to send the request.
    3. Click on the “Send” button - Postman will then send the request to the application, retrieve any responses and display it in the lower window.

    Also Postman has a CLI collection runner - Newman. It allows you to effortlessly run and test a Postman collection
    directly from the command line.

    Useful links:

  • Dredd: initial release date: October 25, 2013, written in: JavaScript. Dredd is a language-agnostic command-line tool for validating API description document against backend implementation of the API. Dredd reads your API description and step by step validates whether your API implementation replies with responses as they are described in the documentation.

    Useful links:

Unit Testing

  • Mocha: initial release date: 22 November 2011, written in: JavaScript. Mocha is a feature-rich JavaScript test framework, making asynchronous testing simple and fun. Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases.

  • Chai: initial release date: January, 2014, written in: JavaScript. Chai is a BDD / TDD assertion library that can be delightfully paired with any JavaScript testing framework. Chai has several interfaces that allow the developer to choose the most comfortable: the chain-capable BDD styles provide an expressive language & readable style, while the TDD assert style provides a more classical feel.

  • Jasmine: initial release date: September 14, 2010, written in: JavaScript. Jasmine is a behavior-driven development framework for testing JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM. And it has a clean, obvious syntax so that you can easily write tests with easy-to-read syntax (describe, it). It has built in assertion library and can be used as a testing framework both for unit tests and end-to-end tests.

  • Karma: initial release date: March 11, 2012, written in: JavaScript. A simple tool that allows you to execute JavaScript code in multiple real browsers. Karma is not a testing framework, nor an assertion library. Karma just launches an HTTP server, and generates the test runner HTML file you probably already know from your favourite testing framework. So for testing purposes you can use pretty much anything you like. There are already plugins for most of the common testing frameworks: Jasmine, Mocha and many others.

  • Jest: initial release date: April, 2016, written in: JavaScript. Jest is a delightful JavaScript Testing
    Framework with a focus on simplicity. It was developed by Facebook.

    Useful links:

  • AVA: initial release date: April 14, 2018, written in: JavaScript. AVA is a test runner for Node.js with a concise API, detailed error output, embrace of new language features and process isolation that let you write tests more effectively.

Useful links:

Load Testing

  • JMeter: initial release date: December 15, 1998, written in: Java. Apache JMeter is a load testing tool for analyzing and measuring the performance of a variety of services, with a focus on web applications.

    Useful links:

  • Tsung: initial release date: May 29, 2006, written in: Erlang. Tsung is an open-source multi-protocol distributed load testing tool. It can currently stress test HTTP, WebDAV, LDAP, MySQL, PostgreSQL, SOAP and XMPP servers. Tsung can simulate hundreds of simultaneous users on a single system. It can also function in a clustered environment.

    Useful links:

  • Gatling: initial release date: January 13, 2012, written in: Scala. Gatling is a highly capable load testing tool designed for ease of use, maintainability and high performance.

  • Artillery: initial release date: November 17, 2015, written in: JavaScript. Artillery is a modern, powerful, easy-to-use load-testing toolkit. Artillery has a strong focus on developer happiness & ease of use, and a batteries-included philosophy. Our goal is to help developers build faster, more resilient and more scalable applications.

  • k6: initial release date: January 18, 2017, written in: Go, but scripts are written in JavaScript (see the running k6 guide). k6 is an open-source load testing tool that makes performance testing easy and productive for engineering teams. k6 is free, developer-centric, and extensible. Using k6, you can test the reliability and performance of your systems and catch performance regressions and problems earlier. k6 will help you to build resilient and performant applications that scale.

Security Testing

Useful links:

Command Line Tools (CLI)

  • Grep: initial release date: November, 1974, written in: PDP-11 assembly. Grep is a command-line utility for searching plain-text data sets for lines that match a regular expression. Its name comes from g/re/p (globally search a regular expression and print), which has the same effect: doing a global search with the regular expression and printing all matching lines.

    Useful links:

Useful links:

Linting and Code Quality

Version Control Systems (VCS)

  • Git: initial release date: April 7, 2005, written in: C, Perl, Tcl, Python. Git is a version-control system for tracking changes in computer files and coordinating work on those files among multiple people. It is primarily used for source-code management in software development, but it can be used to keep track of changes in any set of files.

    Useful links:

Continuous Integration (CI / CD)

  • Jenkins: initial release date: February 2, 2011, written in: Java. Jenkins is commonly used for: building projects, running tests to detect bugs and other issues as soon as they are introduced, static code analysis, deployment, execute repetitive tasks, save time, and optimize your development process. It usually has to be installed on your servers / machines.

  • Travis: initial release date: November, 2010, written in: Ruby. Travis is famous for: quick setup (just add .travis.yml configuration file, login with Github, tell Travis CI to test a project and then push to Github), live build views, pull request support, pre-installed database services, auto deployments on passing builds, clean VMs for every build, etc. With Travis CI testing your open source projects will always be free! It does not need to be installed on your servers / machines, it just needs to be enabled for your GitHub repository.

  • GitLab: initial release date: October, 2011, written in: Ruby, Go, JavaScript. GitLab is a web-based Git-repository manager providing wiki, issue-tracking and CI/CD pipeline features. It usually has to be installed on your servers / machines.

  • GitHub Actions: initial release date: October, 2018, written in: Ruby, Java, JavaScript. GitHub Actions are used to automate, customize, and execute your software development workflows right in your repository. You can create actions to perform any job you’d like, including CI/CD, and combine actions in a completely customized workflow.

    Useful links:

Containerization Tools

Bug Tracking and Task Management

  • Redmine: initial release date: June 25, 2006, written in: Ruby. Redmine is a free and open source, web-based project management and issue tracking tool. It allows users to manage multiple projects and associated subprojects. It features per project wikis and forums, time tracking, and flexible, role-based access control.

  • Mantis: initial release date: Nov 6, 2008, written in: PHP. Mantis Bug Tracker is a free and open source, web-based bug tracking system. The most common use of MantisBT is to track software defects. However, MantisBT is often configured by users to serve as a more generic issue tracking system and project management tool.

  • Jira: initial release date: 2002, written in: Java. Jira used to be an open source tool, but then became a commercial product developed by Atlassian for issue tracking and agile project management. The name Jira comes from Godjira or Japanese for Godzilla. Jira is used by a large number of companies globally for project, time, requirements, task, bug, change, code, test, release and sprint management.

Useful links:

Node.js

Useful links:

Leadership in QA

Useful links:

Job Search

Thanks

If this list of free open source software testing tools and links was helpful to
you, please give it a ★ Star
on GitHub.