Test Reports in Selenium Automation

Test Reports are an important part of software testing and Selenium automation. A test report presents the results of automated test execution in an understandable format so that testers, developers, managers, and other stakeholders can identify passed tests, failed tests, skipped tests, execution time, errors, and other useful information.

In Selenium automation, the browser automation is performed by Selenium WebDriver, while a testing framework such as TestNG or JUnit manages test execution. Reporting tools and built-in reporters then convert execution information into reports that can be reviewed after or during a test run.

TestNG provides built-in reporting capabilities and also supports custom listeners and reporters. Its generated results can include information about suites, tests, methods, execution times, failures, skipped methods, and reporter output. :contentReference[oaicite:0]{index=0}

Course Resource: Selenium Training | Register for Course Demo

1. What is a Test Report?

A test report is a document or web-based result generated after test execution that summarizes what happened during testing. It provides information about the tests executed, their status, failures, execution duration, and other relevant evidence.

A typical automation test report can contain:

• Total number of tests.
• Passed tests.
• Failed tests.
• Skipped tests.
• Test execution duration.
• Test method names.
• Failure messages.
• Exception details.
• Stack traces.
• Browser and environment information.
• Screenshots for failures.
• Execution logs.
• Test data information.
• Build or release information.

2. Why are Test Reports Important?

Test reports are important because automated test execution can produce a large amount of information. A report organizes this information so that the team can quickly understand the result of the test run.

• Provides a clear summary of test execution.
• Helps identify failed test cases.
• Helps developers investigate defects.
• Provides execution evidence.
• Improves communication between QA and development teams.
• Helps monitor regression testing results.
• Supports CI/CD automation.
• Provides historical execution information when the reporting system supports history.
• Helps identify recurring failures.
• Can provide screenshots and logs for debugging.

3. Test Reporting Flow

The basic reporting flow in a Selenium TestNG framework can be represented as follows:

Test Data

    |
    v
TestNG Test Case
    |
    v
Selenium WebDriver
    |
    v
Application Under Test
    |
    v
Assertions / Validation
    |
    v
Test Result
    |
    +----------------+
    |                |
    v                v
Pass             Fail
    |                |
    +--------+-------+
             |
             v
       Reporting Layer
             |
             v
       HTML / XML Report
             |
             v
     Screenshots / Logs
             |
             v
        Test Summary

4. Test Reports in Selenium

Selenium WebDriver is primarily responsible for browser automation. It does not itself function as a complete test reporting framework. Reporting is normally handled by the test framework and/or additional reporting libraries.

A Selenium automation framework can therefore contain several layers:

5. TestNG Reporting

TestNG includes built-in reporting functionality. After a test suite is executed, TestNG can generate an HTML report in its output directory. The generated report provides information about the execution and links to additional result files. :contentReference[oaicite:1]{index=1}

For example, a typical TestNG output directory may contain:

test-output/

|
|-- index.html
|-- emailable-report.html
|-- testng-results.xml
|-- testng-failed.xml
|-- old/
|-- classes/
|-- testng.css

The exact generated files can vary with the TestNG version and reporting configuration.

6. TestNG index.html Report

The index.html file is one of the main HTML reports generated by TestNG. It provides an overview of the test execution and can link to additional report information.

The report can contain information such as:

• Suite name.
• Test name.
• Number of methods.
• Passed methods.
• Failed methods.
• Skipped methods.
• Execution time.
• Reporter output.
• Chronological execution information.

TestNG's sample reports demonstrate suite-level information, execution times, reporter output, and method-level results. :contentReference[oaicite:2]{index=2}

7. Emailable Report

TestNG can generate an emailable HTML report designed to provide a compact summary of the test execution.

An emailable report is useful when test results need to be shared with team members through email or attached to a test execution summary.

A report can typically communicate information such as:

Test Execution Summary

----------------------
Total Tests : 50
Passed      : 44
Failed      : 4
Skipped     : 2
Duration    : 12m 35s

8. TestNG XML Reports

TestNG can also generate XML test results. XML reports are particularly useful when another tool needs structured machine-readable test results. TestNG provides an XML reporter capable of capturing TestNG-specific information. :contentReference[oaicite:3]{index=3}

XML reporting is useful for:

• CI/CD integration.
• Automated result processing.
• Test management systems.
• Custom reporting utilities.
• Result aggregation.

9. Understanding Test Status

Most test reports organize test executions into different statuses.

TestNG determines whether a test is successful or failed based on test execution and exceptions/assertions. :contentReference[oaicite:4]{index=4}

10. Basic TestNG Test Example

import org.testng.Assert;

import org.testng.annotations.Test;
 
public class LoginTest {
 
    @Test
    public void validLoginTest() {
        String actualTitle = "Dashboard";
        String expectedTitle = "Dashboard";
 
        Assert.assertEquals(actualTitle, expectedTitle);
    }
}

When this test executes successfully, TestNG records it as a passed test. If the assertion fails or an unexpected exception occurs, the test is reported as failed.

11. TestNG Report with Multiple Tests

import org.testng.Assert;

import org.testng.annotations.Test;
 
public class LoginTest {
 
    @Test
    public void validLogin() {
        Assert.assertTrue(true);
    }
 
    @Test
    public void invalidLogin() {
        Assert.assertTrue(true);
    }
 
    @Test
    public void logoutTest() {
        Assert.assertEquals("Logout", "Logout");
    }
}

The report can display each test method separately along with its execution status.

12. Reporter.log()

TestNG provides the Reporter class for adding messages that can appear in generated TestNG reports.

import org.testng.Reporter;

import org.testng.annotations.Test;
 
public class ReporterExample {
 
    @Test
    public void loginTest() {
        Reporter.log("Opening login page");
        Reporter.log("Entering username");
        Reporter.log("Entering password");
        Reporter.log("Clicking login button");
        Reporter.log("Login completed");
    }
}

This can make reports more informative by showing important execution messages. TestNG documents Reporter.log() as a mechanism for adding messages to generated HTML reports. :contentReference[oaicite:5]{index=5}

13. Why Logging is Useful in Reports

When a test fails, a simple FAILED status may not provide enough information. Logs can explain what the test was doing before the failure.

Start Login Test

    |
    v
Open Login Page
    |
    v
Enter Username
    |
    v
Enter Password
    |
    v
Click Login
    |
    v
Verify Dashboard
    |
    v
PASS / FAIL

14. Test Reports with Selenium WebDriver

Test reports become more useful when Selenium execution information is combined with test status and browser evidence.

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.Reporter;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
 
public class SeleniumReportTest {
 
    WebDriver driver;
 
    @BeforeMethod
    public void setup() {
        driver = new ChromeDriver();
        driver.manage().window().maximize();
        Reporter.log("Chrome browser started");
    }
 
    @Test
    public void verifyPageTitle() {
        driver.get("https://example.com");
 
        Reporter.log("Opened application");
 
        String title = driver.getTitle();
 
        Reporter.log("Page title: " + title);
 
        Assert.assertFalse(title.isEmpty());
    }
 
    @AfterMethod
    public void tearDown() {
        if (driver != null) {
            driver.quit();
            Reporter.log("Browser closed");
        }
    }
}

15. Test Reports and Assertions

Assertions are an important source of test status information. A test report should clearly communicate whether expected and actual results matched.

String expected = "Dashboard";

String actual = driver.getTitle();
 
Assert.assertEquals(actual, expected);

If the assertion succeeds, the test can be marked as passed. If the assertion fails, TestNG records the failure and includes relevant failure information in its results.

16. Expected vs Actual Result

17. Capturing Screenshots for Reports

Screenshots are valuable evidence when a Selenium test fails. A screenshot can show the state of the browser at the time of failure.

import java.io.File;

import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
 
public class ScreenshotUtil {
 
    public static File capture(WebDriver driver) {
        TakesScreenshot screenshot =
                (TakesScreenshot) driver;
 
        return screenshot.getScreenshotAs(
                OutputType.FILE
        );
    }
}

The screenshot can then be stored and attached to a custom report or test-result system.

18. Screenshot on Test Failure

A common framework design is to capture a screenshot automatically whenever a test fails.

Test Starts

    |
    v
Execute Selenium Steps
    |
    v
Assertion
    |
    +----------------+
    |                |
    v                v
 PASS              FAIL
                     |
                     v
              Capture Screenshot
                     |
                     v
                 Add to Report

19. TestNG ITestListener

ITestListener is a TestNG listener interface that can be used to react to test execution events such as test start, success, failure, and skip. TestNG documents listeners as a way to receive execution events in real time. :contentReference[oaicite:6]{index=6}

Listeners are particularly useful for implementing centralized reporting behavior.

import org.testng.ITestListener;

import org.testng.ITestResult;
 
public class TestListener implements ITestListener {
 
    @Override
    public void onTestStart(ITestResult result) {
        System.out.println(
            "Test Started: " +
            result.getName()
        );
    }
 
    @Override
    public void onTestSuccess(ITestResult result) {
        System.out.println(
            "Test Passed: " +
            result.getName()
        );
    }
 
    @Override
    public void onTestFailure(ITestResult result) {
        System.out.println(
            "Test Failed: " +
            result.getName()
        );
    }
 
    @Override
    public void onTestSkipped(ITestResult result) {
        System.out.println(
            "Test Skipped: " +
            result.getName()
        );
    }
}

20. Listener Execution Flow

Test Starts

    |
    v
onTestStart()
    |
    v
Test Executes
    |
    +--------------------+
    |          |         |
    v          v         v
Success     Failure    Skip
    |          |         |
    v          v         v
onTest      onTest     onTest
Success     Failure    Skipped
    |          |         |
    +----------+---------+
               |
               v
          Report Update

21. Registering a Listener

A TestNG listener can be registered using the @Listeners annotation.

import org.testng.annotations.Listeners;

 
@Listeners(TestListener.class)
public class LoginTest {
 
    // Test methods
}

Another approach is to configure listeners through the TestNG suite configuration when appropriate.

22. IReporter

TestNG also provides the IReporter interface for creating custom reports after the suites have completed. Unlike an event listener that receives execution events in real time, an IReporter receives information about the completed test run and can generate a report from it. :contentReference[oaicite:7]{index=7}

import org.testng.IReporter;

import org.testng.ISuite;
import java.util.List;
 
public class CustomReporter implements IReporter {
 
    @Override
    public void generateReport(
            List<ISuite> suites,
            String outputDirectory) {
 
        System.out.println(
            "Generating report in: " +
            outputDirectory
        );
    }
}

23. ITestListener vs IReporter

24. Custom HTML Reports

Organizations may create custom HTML reports when the built-in TestNG report does not contain all the information required by the project.

A custom report can contain:

• Project name.
• Build number.
• Environment.
• Browser.
• Test suite.
• Test case.
• Test steps.
• Status.
• Execution duration.
• Error message.
• Screenshot.
• Logs.
• Timestamp.

25. Example Custom Report Structure

Test Automation Report

======================
 
Project       : E-Commerce Application
Environment   : QA
Browser       : Chrome
Build         : 1.0.25
 
Total Tests   : 20
Passed        : 17
Failed        : 2
Skipped       : 1
 
Test Results
------------
Login Test       PASS
Search Test      PASS
Cart Test        PASS
Checkout Test    FAIL
Logout Test      PASS

26. Extent-Style Reporting Concept

Third-party reporting libraries can provide richer dashboards than a basic console or default HTML report. Depending on the project, a reporting solution may provide dashboards, logs, screenshots, categories, test steps, and other execution evidence.

When using an external reporting library, the reporting library should be integrated with the chosen test framework and lifecycle.

27. Report with Test Steps

Detailed reports can display the individual steps performed during a test.

Login Test

|
|-- Open Login Page             PASS
|-- Enter Username              PASS
|-- Enter Password              PASS
|-- Click Login                 PASS
|-- Verify Dashboard            PASS
|
+-- Overall Result              PASS

This structure makes a report easier to understand than simply displaying a single PASS or FAIL status.

28. Logging Test Steps

Reporter.log("Step 1: Open application");

driver.get("https://example.com");
 
Reporter.log("Step 2: Enter username");
driver.findElement(By.id("username"))
       .sendKeys("testuser");
 
Reporter.log("Step 3: Enter password");
driver.findElement(By.id("password"))
       .sendKeys("testpassword");
 
Reporter.log("Step 4: Click login");
driver.findElement(By.id("loginButton"))
       .click();
 
Reporter.log("Step 5: Verify dashboard");

29. Test Report with Failure Details

A useful report should not only say that a test failed. It should provide enough information to help identify the reason for failure.

30. Capturing Failure Information

ITestResult can be used inside listeners to obtain information about a test execution.

@Override

public void onTestFailure(ITestResult result) {
 
    System.out.println(
        "Failed Test: " +
        result.getName()
    );
 
    System.out.println(
        "Failure Reason: " +
        result.getThrowable()
    );
}

31. Screenshot and Listener Integration

A common Selenium framework pattern is to capture screenshots from a listener whenever a test fails.

@Override

public void onTestFailure(ITestResult result) {
 
    System.out.println(
        "Test Failed: " +
        result.getName()
    );
 
    // Obtain the WebDriver from the
    // framework's driver management layer.
 
    // Capture screenshot.
 
    // Store screenshot path.
 
    // Attach screenshot to report.
}

The exact implementation depends on how WebDriver instances are managed by the framework.

32. Report File Organization

A good automation framework should keep generated reports and supporting evidence organized.

reports/

|
|-- html/
|   |-- index.html
|   |-- dashboard.html
|
|-- screenshots/
|   |-- login_failure.png
|   |-- checkout_failure.png
|
|-- logs/
|   |-- execution.log
|
|-- xml/
|   |-- testng-results.xml
|
|-- videos/
    |-- test_execution.mp4

33. Test Reports and Maven

Maven can be used to execute TestNG-based Selenium tests as part of a build. Reports can then be generated as part of the test execution process.

mvn test

A typical Maven-based automation flow is:

pom.xml

   |
   v
Maven
   |
   v
TestNG
   |
   v
Selenium Tests
   |
   v
Test Results
   |
   v
Reports

34. Test Reports in CI/CD

Test reports are especially important in CI/CD because tests may execute automatically after every code change or build.

Developer Commit

      |
      v
CI/CD Pipeline
      |
      v
Build
      |
      v
Automated Tests
      |
      v
Selenium WebDriver
      |
      v
TestNG
      |
      v
Test Results
      |
      v
Generate Reports
      |
      v
Publish Reports

35. Test Reports in Jenkins

Jenkins and other CI systems can consume structured test results and publish test execution information. A common automation setup is to generate XML results during the build and configure the CI server to process those results.

The general workflow is:

Jenkins

   |
   v
Maven Test
   |
   v
TestNG
   |
   v
XML Results
   |
   v
Jenkins Test Result
   |
   v
Build Dashboard

36. Test Reports and Regression Testing

Regression testing can produce hundreds or thousands of test executions. Reports help the QA team identify which tests passed, failed, or were skipped during the regression cycle.

For example:

Regression Suite

|
|-- Login Tests       25
|-- Search Tests      30
|-- Cart Tests        20
|-- Checkout Tests    25
|-- Profile Tests     15
|
+-- Total Tests      115

37. Test Reports for Smoke Testing

Smoke tests are commonly executed after a deployment to verify that major application functionality is available.

Smoke Suite

|
|-- Application Launch       PASS
|-- Login                    PASS
|-- Search                   PASS
|-- Product Details          PASS
|-- Add to Cart              PASS
|-- Checkout                 PASS
|
+-- Build Validation         PASS

38. Test Reports for Failed Tests

Reports should make failed tests easy to identify.

39. Rerunning Failed Tests

TestNG creates a testng-failed.xml file when tests fail. This file can be used to rerun failed methods instead of rerunning the complete suite. :contentReference[oaicite:8]{index=8}

Full Test Suite

      |
      v
Test Execution
      |
      v
Failures
      |
      v
testng-failed.xml
      |
      v
Rerun Failed Tests

This can significantly reduce the time required to reproduce failures during debugging.

40. Test Reports and Screenshots

A strong Selenium report often combines test status with visual evidence.

Failed Test

    |
    +-- Failure Message
    |
    +-- Stack Trace
    |
    +-- Browser Information
    |
    +-- Test Logs
    |
    +-- Screenshot
    |
    +-- Timestamp

41. Test Reports and Page Source

For some failures, the screenshot alone may not be enough. Saving the page source can provide additional debugging information.

String pageSource = driver.getPageSource();

 
System.out.println(
    "Page Source Length: " +
    pageSource.length()
);

In a framework, the page source can be stored as an artifact when appropriate.

42. Test Reports and Browser Information

Including browser information can make failures easier to investigate, especially when tests run against multiple browsers.

Browser      : Chrome

Browser      : Firefox
Browser      : Edge

A report can also include operating system, environment, application version, and build number.

43. Cross-Browser Test Reports

When the same test is executed against multiple browsers, the report should make browser-specific results easy to identify.

44. Test Reports with Data Providers

When a TestNG test uses a Data Provider, the same test method may execute multiple times with different data sets. Reports can therefore contain multiple invocations of the same test method.

Login Test

|
|-- admin / valid password       PASS
|-- manager / valid password     PASS
|-- employee / valid password    PASS
|-- invalid / wrong password     FAIL

When sensitive information such as passwords is used, reports and logs should avoid exposing the secret values.

45. Test Reports and Parameters

Browser, environment, URL, and other configuration values can be included in reporting output.

Environment : QA

Browser     : Chrome
URL         : https://qa.example.com
Suite       : Regression

46. Test Reports with Page Object Model

Page Object Model separates page interaction logic from test logic. Reporting can remain in the test framework or listener layer while Page Objects focus on application interactions.

Test Class

    |
    v
Page Object
    |
    v
Selenium WebDriver
    |
    v
Application
    |
    v
Assertion
    |
    v
Test Report

47. Reporting in a POM Framework

LoginTest.java

    |
    +-- LoginPage.java
    |
    +-- DriverFactory.java
    |
    +-- ScreenshotUtil.java
    |
    +-- TestListener.java
    |
    +-- ReportManager.java
    |
    v
HTML Test Report

This design helps keep reporting utilities separate from page-specific Selenium actions.

48. Report Manager Concept

A centralized report manager can be used in larger automation frameworks to initialize reports, create test entries, record steps, log results, and finalize reports.

ReportManager

|
|-- initializeReport()
|-- createTest()
|-- logInfo()
|-- logPass()
|-- logFail()
|-- attachScreenshot()
|-- finalizeReport()

49. Example Report Manager Structure

public class ReportManager {

 
    public static void initializeReport() {
        System.out.println("Report initialized");
    }
 
    public static void logInfo(String message) {
        System.out.println("INFO: " + message);
    }
 
    public static void logPass(String message) {
        System.out.println("PASS: " + message);
    }
 
    public static void logFail(String message) {
        System.out.println("FAIL: " + message);
    }
 
    public static void finalizeReport() {
        System.out.println("Report finalized");
    }
}

50. Test Report Lifecycle

Initialize Report

       |
       v
Create Test Entry
       |
       v
Execute Test
       |
       v
Log Steps
       |
       v
Assertion
       |
       +----------------+
       |                |
       v                v
     PASS             FAIL
       |                |
       |                +--> Screenshot
       |                |
       |                +--> Exception
       |                |
       +----------------+
                |
                v
        Finalize Report

51. HTML Report Example

A simple custom HTML report can contain a table of test results.

<html>

<head>
    <title>Automation Test Report</title>
</head>
<body>
    <h1>Automation Test Report</h1>
 
    <table border="1">
        <tr>
            <th>Test Name</th>
            <th>Status</th>
        </tr>
        <tr>
            <td>Login Test</td>
            <td>PASS</td>
        </tr>
        <tr>
            <td>Search Test</td>
            <td>FAIL</td>
        </tr>
    </table>
</body>
</html>

52. Report Summary Dashboard

A reporting dashboard can summarize the execution using totals and percentages.

Automation Test Dashboard

=========================
 
Total Tests : 100
Passed      : 85
Failed      : 10
Skipped     : 5
 
Pass Rate   : 85%
Fail Rate   : 10%
Skip Rate   : 5%

Dashboards are useful for quickly communicating the overall state of a test execution.

53. Calculating Pass Percentage

A simple pass percentage can be calculated using:

Pass Percentage =

(Passed Tests / Total Executed Tests) * 100

For example, if 90 out of 100 executed tests pass:

(90 / 100) * 100 = 90%

54. Test Reports and Execution Time

Execution duration is another important reporting metric.

Execution time can help teams identify unusually slow tests and investigate performance issues in the automation suite.

55. Test Reports and Flaky Tests

A flaky test is a test that does not consistently produce the same result under the same expected conditions. Historical reports can help identify repeated patterns of intermittent failures.

Execution 1  : PASS

Execution 2  : PASS
Execution 3  : FAIL
Execution 4  : PASS
Execution 5  : FAIL

Reporting systems that preserve execution history can make such patterns easier to investigate.

56. Test Reports and Historical Results

Historical reporting allows teams to compare results across multiple builds or releases.

Build 101

Passed : 92
Failed : 8
 
Build 102
Passed : 95
Failed : 5
 
Build 103
Passed : 98
Failed : 2

Historical trends can help the team monitor changes in test execution results over time.

57. Test Reports and CI Build Information

Reports generated in CI environments should ideally identify the build and environment in which they were created.

Project      : E-Commerce

Build Number : 245
Branch       : release
Environment  : QA
Browser      : Chrome
Execution    : Regression
Date         : 2026-10-01

58. Test Reports and Environment Information

Environment information can be particularly useful when the same automation suite runs against QA, staging, and other environments.

59. Test Report Folder Structure

automation-project/

|
|-- src/
|   |-- test/
|       |-- java/
|           |-- tests/
|           |-- pages/
|           |-- listeners/
|           |-- utilities/
|
|-- reports/
|   |-- html/
|   |-- screenshots/
|   |-- logs/
|   |-- xml/
|
|-- testng.xml
|-- pom.xml

60. Complete Practical Test Reporting Example

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.Reporter;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
 
public class TestReportExample {
 
    WebDriver driver;
 
    @BeforeMethod
    public void setup() {
        driver = new ChromeDriver();
        driver.manage().window().maximize();
 
        Reporter.log(
            "Browser started successfully"
        );
    }
 
    @Test
    public void verifyApplicationTitle() {
 
        Reporter.log(
            "Opening application"
        );
 
        driver.get("https://example.com");
 
        Reporter.log(
            "Application opened"
        );
 
        String title = driver.getTitle();
 
        Reporter.log(
            "Actual title: " + title
        );
 
        Assert.assertFalse(
            title.isEmpty(),
            "Page title should not be empty"
        );
 
        Reporter.log(
            "Title validation completed"
        );
    }
 
    @AfterMethod
    public void tearDown() {
 
        Reporter.log(
            "Closing browser"
        );
 
        if (driver != null) {
            driver.quit();
        }
    }
}

61. Complete Reporting Architecture

                 Selenium Test Framework

                           |
        +------------------+------------------+
        |                  |                  |
        v                  v                  v
     TestNG             Page Objects       Utilities
        |                  |                  |
        +------------------+------------------+
                           |
                           v
                    Test Execution
                           |
                           v
                      Assertions
                           |
                           v
                    TestNG Results
                           |
              +------------+------------+
              |                         |
              v                         v
          Listener                  Reporter
              |                         |
              +------------+------------+
                           |
                           v
                    Reporting Manager
                           |
          +----------------+----------------+
          |                |                |
          v                v                v
       HTML              XML          Screenshots
          |                |                |
          +----------------+----------------+
                           |
                           v
                    CI/CD Dashboard

62. Common Mistakes in Test Reporting

• Generating reports without useful test-step information.
• Not capturing screenshots for important failures.
• Exposing passwords or tokens in reports.
• Using unclear test names.
• Not recording browser and environment information.
• Saving all screenshots with the same filename.
• Not organizing report files.
• Ignoring failed test details.
• Creating overly large reports without retention or cleanup.
• Not publishing reports in CI/CD.
• Sharing sensitive application information in publicly accessible reports.
• Failing to distinguish between test failures and infrastructure failures.

63. Best Practices for Test Reports

• Use meaningful test names.
• Include clear test-step logs.
• Capture screenshots for relevant failures.
• Include browser and environment information.
• Include execution duration.
• Keep reports easy to navigate.
• Separate screenshots, logs, and structured results.
• Do not expose credentials, tokens, or other secrets.
• Use listeners for reusable reporting behavior.
• Use centralized reporting utilities in larger frameworks.
• Publish reports automatically in CI/CD.
• Keep report artifacts for an appropriate retention period.
• Make failed-test diagnosis easy.
• Use structured XML results when integrating with CI tools.
• Use historical reporting when trend analysis is required.

64. Test Reports and Security

Reports can contain sensitive information such as usernames, URLs, request data, screenshots, application data, and error messages. Passwords, API keys, access tokens, and other secrets should not be written into reports or logs.

For example, avoid:

Reporter.log(

    "Password: " + password
);

Prefer masking sensitive values:

Reporter.log(

    "Password: ********"
);

65. Test Reports and Data Privacy

Automation reports should contain only the information required for testing, debugging, and compliance. Personal or confidential data should be handled according to the organization's security and data-retention policies.

66. Test Reports and Parallel Execution

When Selenium tests execute in parallel, reporting utilities should be designed so that one test does not overwrite another test's report information or screenshot.

Thread 1

   |
   +-- Test A
   +-- Screenshot A
   +-- Report Entry A
 
Thread 2
   |
   +-- Test B
   +-- Screenshot B
   +-- Report Entry B

Thread-safe driver and reporting management becomes especially important when parallel execution is used.

67. Test Reports with Data Provider and Parallel Execution

Data Providers can execute multiple data sets, and TestNG supports parallel Data Provider invocations. When this is combined with Selenium, each concurrent invocation should use an isolated WebDriver session and reporting context.

@DataProvider(

    name = "users",
    parallel = true
)
public Object[][] users() {
    return new Object[][] {
        {"user1"},
        {"user2"},
        {"user3"}
    };
}
 
@Test(dataProvider = "users")
public void loginTest(String username) {
    Reporter.log(
        "Testing user: " + username
    );
}

68. ReportNG

ReportNG is an HTML/XML reporting plug-in associated with TestNG. It was designed as an alternative presentation of TestNG's default HTML report and can also produce JUnit-format XML output. :contentReference[oaicite:9]{index=9}

When selecting a reporting solution for a current project, verify compatibility with the project's TestNG and Java versions because third-party reporting tools can have version-specific limitations.

69. Test Reports with JUnit XML

JUnit-style XML is widely used as a machine-readable result format for CI systems and other test-result consumers. TestNG can generate XML results, and reporting integrations can consume those results.

Automated Test

      |
      v
TestNG
      |
      v
XML Result
      |
      v
CI Server
      |
      v
Test Result Dashboard

70. Test Reports and Custom Listeners

Custom listeners allow framework developers to centralize common reporting actions.

TestListener

|
|-- onTestStart()
|-- onTestSuccess()
|-- onTestFailure()
|-- onTestSkipped()
|
+-- Screenshot Handling
+-- Logging
+-- Report Updates

71. Test Reports and Failure Diagnosis

A good report should answer the following questions:

• Which test failed?
• Which step failed?
• What was expected?
• What actually happened?
• Which browser was used?
• Which environment was used?
• When did the failure occur?
• What exception occurred?
• Is a screenshot available?
• Can the test be reproduced?

72. Test Report Checklist

73. Interview Questions on Test Reports

1. What is a test report?

A test report is a document or dashboard that summarizes automated or manual test execution results.

2. Why are test reports important in Selenium?

They provide evidence of test execution and help teams identify failures, understand errors, and communicate results.

3. Does Selenium WebDriver generate complete test reports?

Selenium WebDriver focuses on browser automation. Complete test reporting is normally handled by a test framework and reporting tools.

4. Does TestNG generate HTML reports?

Yes. TestNG provides built-in HTML reporting capabilities.

5. What is index.html in TestNG?

It is a main HTML result page generated by TestNG that summarizes the test execution.

6. What is an emailable report?

An emailable report is a compact HTML representation of test execution results that can be shared with stakeholders.

7. What is Reporter.log()?

Reporter.log() is a TestNG API used to add logging information to TestNG reports.

8. What is ITestListener?

ITestListener is a TestNG listener interface that receives test execution events such as start, success, failure, and skip.

9. What is IReporter?

IReporter is a TestNG interface used to generate reports after suite execution.

10. What is the difference between ITestListener and IReporter?

ITestListener is designed for receiving execution events, while IReporter is designed for processing completed suite information and generating reports.

11. Why are screenshots useful in Selenium reports?

They provide visual evidence of the browser state at the time of a failure.

12. How can screenshots be captured in Selenium?

Selenium provides the TakesScreenshot interface for capturing screenshots.

13. How can a listener capture screenshots on failure?

An ITestListener implementation can execute screenshot logic inside its onTestFailure() method.

14. Why should passwords not be included in reports?

Passwords are sensitive information and should not be exposed through logs or reports.

15. What information should a failure report contain?

It should ideally contain the test name, failure message, relevant logs, browser/environment information, execution time, and useful evidence such as a screenshot.

16. How are reports useful in CI/CD?

Reports allow teams to review automated test results after a pipeline execution and identify failed tests.

17. What is testng-failed.xml?

It is a TestNG-generated XML file that can be used to rerun failed test methods.

18. What is a custom report?

A custom report is a report created using project-specific reporting logic, listeners, reporters, or reporting libraries.

19. What is XML reporting?

XML reporting produces structured test-result data that can be consumed by other tools and CI systems.

20. What makes a good Selenium test report?

A useful Selenium report clearly communicates test status, failure details, execution information, logs, and relevant browser evidence while avoiding sensitive data exposure.

74. Quick Reference Table

75. Learning Roadmap for Test Reports

1. Understand Selenium WebDriver and TestNG basics.
2. Learn TestNG test execution and assertions.
3. Understand the TestNG output directory.
4. Learn the index.html report.
5. Understand emailable reports.
6. Learn XML test results.
7. Use Reporter.log().
8. Learn ITestListener.
9. Implement failure listeners.
10. Capture Selenium screenshots.
11. Attach screenshots to reporting systems.
12. Create reusable reporting utilities.
13. Understand IReporter.
14. Integrate reporting with Maven.
15. Publish reports through CI/CD.
16. Design thread-safe reporting for parallel execution.
17. Protect sensitive information in logs and reports.

76. Practical Exercises

1. Create a basic TestNG test and inspect the generated HTML report.
2. Create a Selenium login test and add Reporter.log() messages.
3. Create a test containing both passing and failing assertions.
4. Capture a screenshot when a Selenium test fails.
5. Create an ITestListener implementation.
6. Print test start, success, failure, and skip events.
7. Create a reusable Screenshot Utility.
8. Add browser and environment information to reports.
9. Create a custom report structure.
10. Generate XML test results and integrate them into a CI workflow.
11. Configure a Maven-based Selenium TestNG project.
12. Configure a CI pipeline to execute tests and publish results.
13. Implement reporting for Data Provider-based tests.
14. Design reporting for parallel Selenium execution.

77. Real-World Selenium Reporting Example

Consider an e-commerce automation suite containing Login, Search, Product Details, Cart, Checkout, and Logout tests.

E-Commerce Automation Suite

|
|-- Login Test
|      +-- PASS
|
|-- Search Test
|      +-- PASS
|
|-- Product Test
|      +-- PASS
|
|-- Cart Test
|      +-- PASS
|
|-- Checkout Test
|      +-- FAIL
|          |
|          +-- Error Message
|          +-- Screenshot
|          +-- Browser Details
|          +-- Execution Logs
|
|-- Logout Test
|      +-- SKIP
|
+-- Final Report
       |
       +-- Total Tests
       +-- Passed
       +-- Failed
       +-- Skipped
       +-- Duration
       +-- Failure Details

78. Recommended Reporting Architecture

TestNG

  |
  v
ITestListener
  |
  +--------------------+
  |                    |
  v                    v
Test Status       Screenshot Utility
  |                    |
  +----------+---------+
             |
             v
       Report Manager
             |
      +------+------+
      |             |
      v             v
    HTML           XML
      |             |
      +------+------+
             |
             v
          CI/CD
             |
             v
      Published Results

79. Summary

Test Reports are an essential component of a professional Selenium automation framework. They convert raw test execution information into a format that can be understood and analyzed by testers, developers, managers, and CI/CD systems.

TestNG provides built-in HTML and XML reporting capabilities and supports listeners and reporters for custom reporting requirements. TestNG also supports rerunning failed methods through the generated testng-failed.xml file. :contentReference[oaicite:10]{index=10}

In Selenium frameworks, reporting becomes more useful when test results are combined with execution logs, assertions, screenshots, browser information, environment details, and failure information.

For larger automation frameworks, reporting can be integrated with Page Object Model, Data Providers, Maven, CI/CD pipelines, parallel execution, screenshot utilities, and centralized listener/reporting components.

80. Final Takeaway

Test Reports provide visibility into automation execution. A well-designed Selenium reporting system should clearly show what was tested, whether it passed or failed, why it failed, and enough supporting evidence to help the team investigate the issue.

The overall reporting approach can be remembered as:

Execute Test

    |
    v
Validate Result
    |
    v
Capture Status
    |
    v
Capture Logs
    |
    v
Capture Screenshot if Required
    |
    v
Generate Report
    |
    v
Publish Report
    |
    v
Analyze Results

81. Course Resources

Learn more about Selenium automation and software testing:

• Selenium Training
• Register for Selenium Course Demo