Open Source Test Reporting Tools Guide

Running automated tests is one of the great things for software quality assurance. We now have to understand the effects of the modern test run. We could also be interested in seeing how our test results evolve. To achieve these goals, we need a tool for producing meaningful test reports, and some open-source tools are available in the market to help us do this.

Some of the main open-source test reporting tools available in the market today are listed in this article. Below are some of the features that we can find in open source test reporting tools:

1. Data management of one or many languages and test automation frameworks.
2. Test report creation in different formats.
3. Report distribution via email.
4. A graphical and textual representation of results of a single test run.
5. Presentation of the evolution of results for different test runs using a timeline graph.
6. Integration in the CI/CD process.

What are Generic Tools

Open source test reporting tools that work with multiple programming languages and test automation frameworks are listed in the section:

1. Allure Framework

Allure framework is a manageable and lightweight open-source multi-language test reporting tool used for providing clear graphical reports. It allows every person involved in the development procedure for extracting the maximum of information from the everyday software testing procedure. 

Many languages, including Java, JavaScript, .NET, Python, Ruby, etc., are supported by the allure framework. Reporting process could also be triggered by a continuous integration tool like Jenkins or Gradle. One of the earlier specific features is linking user stories or effects defined in behavior-driven development tools.

2. ExtentReports

ExtentReports is also an open-source library that enables users to create interactive and detailed reports for their tests. We can also add events, screenshots, tags, devices, or any other relevant information that we decide is essential for creating descriptive reports that we have total control of. This framework can operate with Java and C# languages and the Cucumber, SpecFlow, and TestNG testing tools.

ExtentReports is associated with Klov, a tool that allows us to track each of our launches across all the projects. We can observe and analyze how our test performs over time and how the execution category can be compared in detail, track exceptions, create topics for discussion on reminders, and search entities from a host of options. This open-source framework also supports a commercial version.

3. ReportPortal

ReportPortal is an open-source reporting service that 8f odds increase capabilities for speeding up test result analysis and reporting through built-in analytic features. ReportPortal seamlessly blends with mainstream CI/CD and project management platforms like Jenkins, Jira, and many other functional and unit testing frameworks like Cucumber, JBehave, SpecFlow Jasmine. 

Execution of test case results is stored by following the same structure we have in our reporting suites and test plans. All the test cases are sold together with or related data in one place right where we need it. The performance pipeline of certain test cases is also available to view previous execution results in one click. 

ReportPortal also provides the ability to analyze the test automation results collaboratively. We can also associate particular test cases with the product bug, an automation issue, a system issue or could be submitted as an issue ticket directly from the execution result. 

ReportPortal will automatically figure out the root cause of a fail with each execution. ReportPortal will be marking a test result with a flag as a result of this analysis. Then all the queue engineers will be alerted about this issue for providing further analysis if the bag has been resolved already or which tests are required for actual human analysis.

What are Specific Tools

Open source test reporting tools designed to work with a limited number of programming languages and test automation frameworks are listed in this section:

1. CI::Reporter

CI::Reporter is an add-on for Ruby testing frameworks like Test::Unit or RSpec that will generate XML reports of our test runs. The resulting files could be read by continuous integration systems that understand Ant’s JUnit report XML format and allow our CI systems to track test successes and failures.

CI::Reporter works only with projects that utilize standard rake tasks for running tests. It hooks into testing frameworks by using environment variables recognized by the custom task to inject the reporter code into the test run in this fashion.

2. Cluecumber Report Maven Plugin

Cluecumber Report Maven Plugin is an open-source Maven plugin for simple and concisely reporting Cucumber BDD tests. This plugin provides aggregated test results from Cucumber files that are JSON compliant with the Cucumber BDD format, Karate, and other frameworks.

3. Jasmine Reporters

Jasmine Reporters is a JavaScript collection of Jasmine Reporter classes that we can include in the Test System of JasmineBDD. JUnitXmlReporter is included for producing JUnit XML output in CI environments like Jenkins.

4. jest-html-reporter

Jest-HTML-reporter is an open-source repository of test results for HTML generation. Jest is an open-source JavaScript testing framework that aims to work out of the box on certain JavaScript projects. Jest Reporter is an open-source processor for test results to produce an HTML overview. Jest is an open-source JavaScript testing framework intended to operate out of the box on certain JavaScript projects.

5. karma-htmlfile-reporter

Karma-htmlfile-reporter is an HTML-styled karma plugin for unit test reports. It is a plugin for the Karma Test Runner. Adding the reporter to our karma setup would export the unit test results as a styled HTML format. A separate table is created for each test browser.

6. ReportBuilder

Report builder is a Ruby gem that blends Cucumber JSON reports and develops mobile-friendly HTML test reports, JSON reports, and retry files.

7. ReportNG

ReportNG is an uncomplicated HTML open-source reporting plugin used for the TestNG unit testing framework. It is used as a replacement for the default TestNG HTML report. A default report is very comprehensive but not easy to understand at a glance. 

ReportNG provides a simple and colour-coded view of the test results, and it also generates 100% valid XHTML 1.0 files. We can also customize the output by overriding a default style sheet with our CSS file.

Related Articles

Open Source Forensic Tools

Open Source Bug Tracking Tools

Open Source Security Testing Tools