Test Reports in Selenium and TestNG

Test Reports are an essential part of software testing and automation because they provide a structured summary of test execution results. A test report helps testers, developers, managers, and other stakeholders understand which test cases passed, failed, were skipped, or encountered errors.

In Selenium automation, TestNG is commonly used to execute automated test cases and generate execution reports. TestNG provides HTML and XML reporting capabilities, and its reporting system can be extended using listeners and reporters. :contentReference[oaicite:0]{index=0}

Test reports are especially useful in Selenium frameworks because a large automation suite may execute hundreds or thousands of test cases. Instead of manually checking console output, testers can use reports to identify test status, execution time, failures, skipped tests, error messages, and other useful information.

Course Resources: Selenium Training | Register for Course Demo

1. What are Test Reports?

A test report is a document or web-based output that presents the results of a test execution. It summarizes the execution status of test cases and provides information that helps the team analyze the quality and stability of an application.

For example, after executing 100 Selenium test cases, a report may show:

The report can then be used to investigate the 12 failed and 6 skipped tests.

2. Why are Test Reports Important?

Test reports provide visibility into the overall result of an automation test execution.

• Shows passed test cases.
• Shows failed test cases.
• Shows skipped test cases.
• Displays test execution details.
• Helps identify failures quickly.
• Provides execution timing information.
• Helps developers reproduce failures.
• Provides evidence of test execution.
• Supports regression testing.
• Helps monitor application quality.
• Can be integrated with CI/CD pipelines.
• Improves communication between QA and development teams.

3. Test Reporting Flow

The general Selenium and TestNG reporting process can be represented as follows:

Test Cases

    |
    v
TestNG Test Execution
    |
    v
Selenium WebDriver
    |
    v
Application Under Test
    |
    v
Assertions / Validation
    |
    v
Pass / Fail / Skip
    |
    v
TestNG Listener / Reporter
    |
    v
HTML / XML Report
    |
    v
QA / Developer / Management Analysis

4. Test Reports in Selenium

Selenium WebDriver itself is primarily responsible for browser automation. It does not provide a complete test execution reporting framework by itself.

In a Java Selenium framework, TestNG can be used for test execution, assertions, test organization, listeners, and reporting. TestNG generates result files after execution, including HTML and XML-based outputs. :contentReference[oaicite:1]{index=1}

The combination can be represented as:

Selenium WebDriver

        +
TestNG
        +
Listeners / Reporters
        |
        v
Automation Test Report

5. TestNG Default Reports

TestNG generates reports after test execution. The default output directory is commonly named test-output, although the output directory can be configured. TestNG documentation describes an HTML index report and related HTML/text result files. :contentReference[oaicite:2]{index=2}

A typical project may contain:

project

|
|-- src
|   |-- main
|   |-- test
|
|-- pom.xml
|
|-- testng.xml
|
|-- test-output
    |-- index.html
    |-- emailable-report.html
    |-- testng-results.xml
    |-- testng-failed.xml

6. TestNG index.html Report

The index.html report provides an overview of the test execution. It can contain information about suites, tests, classes, methods, results, timings, and other execution information.

A simplified report structure may look like:

TestNG Results

|
|-- Test Suite
|   |
|   |-- Test
|       |
|       |-- Passed Tests
|       |-- Failed Tests
|       |-- Skipped Tests
|
|-- Execution Time
|
|-- Reporter Output
|
|-- Chronological View

7. Emailable Report

TestNG provides an emailable HTML report that presents test results in a compact format suitable for sharing with team members.

Modern TestNG versions include an EmailableReporter2 reporter for generating a single-page HTML representation of test results. :contentReference[oaicite:3]{index=3}

An emailable report is useful when a QA engineer needs to quickly communicate execution results through email, project documentation, or team communication.

8. Test Result Status

Test reports generally categorize test execution into different statuses.

TestNG determines success and failure based on test execution and exceptions/assertions. :contentReference[oaicite:4]{index=4}

9. Passed Test

A test is marked as passed when the test method completes successfully without an unexpected exception or failed assertion.

@Test

public void verifyTitle() {
 
    driver.get("https://example.com");
 
    String actualTitle = driver.getTitle();
 
    Assert.assertEquals(actualTitle, "Example Domain");
}

If the actual title matches the expected title, the test can be reported as passed.

10. Failed Test

A test is marked as failed when an assertion fails or an unexpected exception occurs during execution.

@Test

public void verifyTitle() {
 
    driver.get("https://example.com");
 
    String actualTitle = driver.getTitle();
 
    Assert.assertEquals(actualTitle, "Expected Title");
}

If the actual title does not match the expected title, TestNG records the test as failed.

11. Skipped Test

A skipped test is a test that TestNG does not execute normally because it has been explicitly disabled or because a dependency/configuration condition prevents execution.

@Test(enabled = false)

public void skippedTest() {
    System.out.println("This test will not execute.");
}

Skipped tests appear separately in the TestNG results.

12. TestNG Assertions and Reports

Assertions are an important part of reporting because they determine whether an expected condition was satisfied.

import org.testng.Assert;

import org.testng.annotations.Test;
 
public class AssertionTest {
 
    @Test
    public void verifyLogin() {
 
        String expected = "Dashboard";
        String actual = "Dashboard";
 
        Assert.assertEquals(actual, expected);
    }
}

When the assertion passes, the test can be recorded as passed. When the assertion fails, TestNG records the failure and provides failure information in the report.

13. TestNG Reporter Class

TestNG provides the Reporter class for writing messages that can appear in generated HTML reports. The official TestNG documentation describes Reporter.log() as a mechanism for adding messages to generated reports. :contentReference[oaicite:5]{index=5}

import org.testng.Reporter;

import org.testng.annotations.Test;
 
public class ReportLogTest {
 
    @Test
    public void testLogin() {
 
        Reporter.log("Starting login test");
 
        System.out.println("Executing login test");
 
        Reporter.log("Login test completed");
    }
}

14. Reporter.log() with Selenium

Reporter logging can be used to record important Selenium execution steps.

import org.testng.Reporter;

import org.testng.annotations.Test;
 
public class SeleniumReportTest {
 
    @Test
    public void loginTest() {
 
        Reporter.log("Opening application");
 
        driver.get("https://example.com/login");
 
        Reporter.log("Entering username");
 
        driver.findElement(By.id("username"))
                .sendKeys("testuser");
 
        Reporter.log("Entering password");
 
        driver.findElement(By.id("password"))
                .sendKeys("password");
 
        Reporter.log("Clicking login button");
 
        driver.findElement(By.id("login"))
                .click();
 
        Reporter.log("Login action completed");
    }
}

15. Test Reports and TestNG Listeners

Listeners allow automation frameworks to react to test execution events. TestNG provides listener interfaces that can be used to monitor events such as test start, success, failure, and skip. :contentReference[oaicite:6]{index=6}

Important listener interfaces include:

• ITestListener
• ISuiteListener
• IInvokedMethodListener
• ITestResult-based event handling

16. ITestListener

ITestListener is commonly used to monitor the lifecycle of test methods.

Important methods include:

• onTestStart()
• onTestSuccess()
• onTestFailure()
• onTestSkipped()
• onStart()
• onFinish()

These methods can be used to add custom logging, screenshots, reporting information, notifications, and other actions.

17. Basic ITestListener Example

import org.testng.ITestContext;

import org.testng.ITestListener;
import org.testng.ITestResult;
 
public class TestListener implements ITestListener {
 
    @Override
    public void onTestStart(ITestResult result) {
        System.out.println(
            "Test Started: " + result.getName()
        );
    }
 
    @Override
    public void onTestSuccess(ITestResult result) {
        System.out.println(
            "Test Passed: " + result.getName()
        );
    }
 
    @Override
    public void onTestFailure(ITestResult result) {
        System.out.println(
            "Test Failed: " + result.getName()
        );
    }
 
    @Override
    public void onTestSkipped(ITestResult result) {
        System.out.println(
            "Test Skipped: " + result.getName()
        );
    }
}

18. Registering a Listener

A listener can be registered using the @Listeners annotation.

import org.testng.annotations.Listeners;

 
@Listeners(TestListener.class)
public class LoginTest {
 
    @Test
    public void loginTest() {
        System.out.println("Login Test");
    }
}

The listener can then receive execution events from TestNG.

19. Listener-Based Reporting Flow

Test Starts

    |
    v
onTestStart()
    |
    v
Test Execution
    |
    +------------------+
    |                  |
    v                  v
PASS               FAIL
    |                  |
    v                  v
onTestSuccess()    onTestFailure()
    |                  |
    +--------+---------+
             |
             v
        Report Update
             |
             v
        Test Finished

20. IReporter Interface

TestNG also provides the IReporter interface for generating custom reports after test execution. Unlike an event-oriented listener, an IReporter receives information about the completed suite and can generate a report from the collected results. :contentReference[oaicite:7]{index=7}

import org.testng.IReporter;

import org.testng.ISuite;
import org.testng.xml.XmlSuite;
 
import java.util.List;
 
public class CustomReporter implements IReporter {
 
    @Override
    public void generateReport(
            List<XmlSuite> xmlSuites,
            List<ISuite> suites,
            String outputDirectory) {
 
        System.out.println(
            "Generating report in: " + outputDirectory
        );
    }
}

21. Listener vs Reporter

22. Capturing Screenshots for Failed Tests

One of the most useful features of Selenium reporting is attaching or linking a screenshot when a test fails. This helps the QA engineer understand what the browser looked like at the moment of failure.

public void captureScreenshot(String testName) {

 
    TakesScreenshot screenshot =
            (TakesScreenshot) driver;
 
    File source =
            screenshot.getScreenshotAs(
                OutputType.FILE
            );
 
    File destination =
            new File(
                "screenshots/" + testName + ".png"
            );
 
    FileUtils.copyFile(source, destination);
}

23. Screenshot on Test Failure

A listener can capture a screenshot automatically whenever a test fails.

@Override

public void onTestFailure(ITestResult result) {
 
    String testName = result.getName();
 
    System.out.println(
        "Test failed: " + testName
    );
 
    // Screenshot capture logic can be called here.
}

This creates a useful relationship between:

Failed Test

    |
    v
Failure Listener
    |
    v
Screenshot
    |
    v
Report
    |
    v
Failure Analysis

24. Why Screenshots are Important in Reports

• Shows the exact browser state during failure.
• Helps identify incorrect page navigation.
• Helps identify missing elements.
• Helps diagnose validation errors.
• Provides visual evidence.
• Reduces debugging time.
• Improves defect analysis.

25. Capturing Page Source

Page source can also be captured during failures when useful.

String pageSource = driver.getPageSource();

 
Files.write(
    Paths.get("page-source.html"),
    pageSource.getBytes()
);

Page source can help investigate DOM-related problems, missing elements, unexpected HTML, or application rendering issues.

26. Capturing Current URL

The current URL is another useful piece of information for a report.

String currentUrl = driver.getCurrentUrl();

 
Reporter.log(
    "Current URL: " + currentUrl
);

27. Capturing Browser Information

Cross-browser test reports should identify which browser was used for each execution.

Reporter.log("Browser: Chrome");

Reporter.log("Test Environment: QA");
Reporter.log("Platform: Windows");

Useful report information can include:

• Browser
• Browser version
• Operating system
• Application environment
• Application URL
• Test suite
• Test method

28. Test Reports with Data Providers

When a test uses a Data Provider, the same test method can be executed multiple times with different data sets. Reports should make it possible to identify the individual invocation and its result.

@DataProvider(name = "users")

public Object[][] users() {
    return new Object[][] {
        {"admin"},
        {"manager"},
        {"employee"}
    };
}
 
@Test(dataProvider = "users")
public void loginTest(String username) {
 
    Reporter.log(
        "Testing user: " + username
    );
}

This is particularly useful for identifying which input caused a failure.

29. Test Reports with Parameterization

TestNG parameters can also appear in generated reports. TestNG documentation notes that parameters used to invoke test methods can be shown in HTML reports. :contentReference[oaicite:8]{index=8}

@Parameters({"browser"})

@Test
public void browserTest(String browser) {
 
    Reporter.log(
        "Browser: " + browser
    );
}

30. Test Reports with TestNG XML

TestNG can generate XML result information that can be consumed by reporting tools and CI systems. TestNG also provides an XML reporter for TestNG-specific information. :contentReference[oaicite:9]{index=9}

A simplified XML structure can look like:

<testsuite name="AutomationSuite">

    <testcase name="LoginTest"/>
    <testcase name="SearchTest"/>
</testsuite>

31. HTML Reports vs XML Reports

32. Custom HTML Reports

Automation frameworks can generate custom HTML reports containing project-specific information such as test status, screenshots, execution times, environment details, logs, and failure information.

A custom report can contain:

• Project name
• Build number
• Execution date
• Environment
• Browser
• Total tests
• Passed tests
• Failed tests
• Skipped tests
• Execution duration
• Failure details
• Screenshots
• Logs

33. Example Custom Report Structure

=========================================

        SELENIUM AUTOMATION REPORT
=========================================
 
Project       : E-Commerce Application
Environment   : QA
Browser       : Chrome
 
Total Tests   : 50
Passed        : 42
Failed        : 5
Skipped       : 3
 
Execution Time: 12 Minutes
 
Failed Tests:
1. LoginTest
2. CheckoutTest
3. SearchTest
4. PaymentTest
5. LogoutTest
 
=========================================

34. Extent Reports

Extent Reports is a popular third-party reporting approach used in Java automation frameworks to create detailed and visually rich test reports.

Depending on the framework and version being used, Extent-style reports can include:

• Test status
• Test steps
• Logs
• Screenshots
• Execution time
• Environment information
• Categories
• Authors
• Exception details

The exact API and dependency configuration should be selected according to the Extent Reports version used by the project.

35. Extent Report Basic Concept

TestNG Test

    |
    v
Extent Listener / Reporting Code
    |
    +------ Test Status
    |
    +------ Logs
    |
    +------ Screenshot
    |
    +------ Exception
    |
    v
HTML Report

36. Reporting with Allure

Allure is another reporting ecosystem commonly integrated with automated testing frameworks. It can provide detailed test execution information, steps, attachments, and failure details.

A typical architecture is:

Selenium

    |
    v
TestNG
    |
    v
Allure Integration
    |
    v
Allure Results
    |
    v
Allure Report

37. TestNG Reports vs Third-Party Reports

38. Test Reports and Maven

Maven can execute TestNG-based Selenium tests and can work with reporting plugins and result files.

mvn test

A typical Maven project may generate test-related output under the build output directories depending on the Maven configuration and reporting tools being used.

39. Test Reports and CI/CD

Test reports become particularly important when Selenium tests run automatically in CI/CD pipelines.

Developer Commit

      |
      v
Source Control
      |
      v
CI/CD Pipeline
      |
      v
Build
      |
      v
Selenium Tests
      |
      v
TestNG
      |
      v
Reports
      |
      v
Pipeline Result
      |
      v
Team Notification

40. Test Reports in Jenkins

Jenkins can consume test result files generated by automation frameworks and can publish or display test results as part of a CI job.

A common workflow is:

Jenkins

   |
   v
Maven Build
   |
   v
TestNG Execution
   |
   v
Selenium Tests
   |
   v
Test Results
   |
   +------ HTML Report
   |
   +------ XML Results
   |
   v
Jenkins Build Report

41. Test Reports and GitHub Actions

GitHub Actions can also execute Selenium and TestNG tests as part of automated workflows. Generated reports can be retained as build artifacts depending on the workflow configuration.

Push Code

   |
   v
GitHub Actions
   |
   v
Build
   |
   v
Run TestNG
   |
   v
Generate Reports
   |
   v
Upload / Publish Results

42. Test Reports and Failure Analysis

A useful test report should help answer:

• Which test failed?
• When did it fail?
• Which browser was used?
• Which environment was used?
• What was the current URL?
• What exception occurred?
• What was the expected result?
• What was the actual result?
• What did the browser look like?
• Which test data was used?

43. Exception Details in Reports

Exception information is important for understanding automation failures.

org.openqa.selenium.NoSuchElementException

    |
    +-- Element was not found
    |
    +-- Locator: By.id("loginButton")
    |
    +-- URL: https://example.com/login
    |
    +-- Browser: Chrome

A detailed report should provide enough context to investigate the failure without requiring the entire test to be rerun immediately.

44. Common Selenium Exceptions in Reports

45. Test Reports and Execution Time

Execution time can help identify slow tests and performance-related issues within the automation suite.

LoginTest       : 2.1 seconds

SearchTest      : 4.3 seconds
CheckoutTest    : 8.7 seconds
PaymentTest     : 6.2 seconds
LogoutTest      : 1.5 seconds

If one test consistently takes much longer than others, the automation team can investigate its synchronization, application response, or test design.

46. Test Reports and Test Steps

Detailed reporting can record important test steps.

Step 1: Open application

Step 2: Enter username
Step 3: Enter password
Step 4: Click login
Step 5: Verify dashboard
Step 6: Logout

Step-level logging makes failures easier to understand.

47. Test Reports and Test Data

When tests are data-driven, the report should identify the relevant test data without exposing sensitive information.

Test: LoginTest

User Type: Admin
Environment: QA
Browser: Chrome
Result: PASS

Passwords and other secrets should not be exposed in reports.

48. Test Reports and Environment Details

Environment information is especially important when the same test suite runs against multiple environments.

49. Test Reports and Cross-Browser Testing

When a Selenium suite runs on multiple browsers, reports should clearly identify the browser used for each execution.

LoginTest

|
|-- Chrome
|     |-- PASS
|
|-- Firefox
|     |-- PASS
|
|-- Edge
      |-- FAIL

This makes browser-specific failures easier to identify.

50. Test Reports and Parallel Execution

When tests execute in parallel, reports should distinguish individual test invocations and environments.

Thread 1 -> Chrome  -> LoginTest   -> PASS

Thread 2 -> Firefox -> LoginTest   -> PASS
Thread 3 -> Edge    -> LoginTest   -> FAIL
Thread 4 -> Chrome  -> SearchTest  -> PASS

Thread-safe reporting and isolated WebDriver instances are important when tests execute concurrently.

51. Test Reports and TestNG Groups

TestNG supports grouping tests. Reports can then help identify results by functional or execution group.

@Test(groups = "smoke")

public void loginTest() {
}
 
@Test(groups = "regression")
public void checkoutTest() {
}
 
@Test(groups = "sanity")
public void searchTest() {
}

Groups can help teams organize smoke, sanity, regression, integration, or other test categories.

52. Test Reports and Test Dependencies

TestNG supports dependencies between tests. A dependent test may be skipped when its dependency does not complete successfully. TestNG documents dependency behavior and how it appears in reports. :contentReference[oaicite:10]{index=10}

@Test

public void loginTest() {
    System.out.println("Login");
}
 
@Test(dependsOnMethods = "loginTest")
public void dashboardTest() {
    System.out.println("Dashboard");
}

53. Test Reports and Chronological View

TestNG reports can provide chronological information about test execution. This can help understand the order and timing of executed methods. The official TestNG sample report includes chronological execution information. :contentReference[oaicite:11]{index=11}

10:00:01 -> LoginTest started

10:00:03 -> LoginTest passed
10:00:04 -> SearchTest started
10:00:07 -> SearchTest passed
10:00:08 -> CheckoutTest started
10:00:14 -> CheckoutTest failed

54. Test Reports and Logs

Logs provide additional information about what happened during test execution.

2026-09-30 10:00:01 - Opening browser

2026-09-30 10:00:03 - Opening login page
2026-09-30 10:00:04 - Entering username
2026-09-30 10:00:05 - Entering password
2026-09-30 10:00:06 - Clicking login
2026-09-30 10:00:08 - Dashboard verification passed

55. Test Reports and Screenshots

A strong Selenium reporting framework commonly captures screenshots for failed tests and may also capture screenshots at important checkpoints.

Test Execution

      |
      v
Failure Detected
      |
      v
Capture Screenshot
      |
      v
Save Screenshot
      |
      v
Add Reference to Report
      |
      v
Analyze Failure

56. Test Reports and Video Recording

In some automation environments, browser sessions can also be recorded as video. Video recording can provide additional evidence for difficult or intermittent failures, especially in remote or distributed execution environments.

Video recording should be used selectively because it can increase storage requirements and execution overhead.

57. Test Reports for Regression Testing

Regression testing often executes a large number of automated tests. Reports make it possible to compare the overall execution status and identify failures after application changes.

Regression Suite

|
|-- Login
|-- Registration
|-- Search
|-- Product
|-- Cart
|-- Checkout
|-- Payment
|-- Order
|-- Logout
|
v
Regression Report

58. Test Reports for Smoke Testing

Smoke testing usually contains a smaller set of critical tests. A smoke report can quickly indicate whether the build is suitable for further testing.

Smoke Suite

|
|-- Application Launch
|-- Login
|-- Search
|-- Add to Cart
|-- Checkout
|
v
Smoke Result
|
+-- PASS
+-- FAIL

59. Test Reports for Sanity Testing

Sanity test reports can focus on a smaller functional area after a specific change or bug fix.

For example:

Bug Fix

   |
   v
Sanity Tests
   |
   +-- Login
   +-- Dashboard
   +-- Logout
   |
   v
Sanity Report

60. Custom Report Data

A professional automation report may contain the following information:

61. Sample Selenium Test with Reporting

import org.openqa.selenium.By;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.Reporter;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
 
public class LoginReportTest {
 
    WebDriver driver;
 
    @BeforeMethod
    public void setup() {
 
        Reporter.log("Starting browser");
 
        driver = new ChromeDriver();
 
        driver.manage().window().maximize();
 
        Reporter.log("Opening login page");
 
        driver.get("https://example.com/login");
    }
 
    @Test
    public void loginTest() {
 
        Reporter.log("Entering username");
 
        driver.findElement(By.id("username"))
                .sendKeys("testuser");
 
        Reporter.log("Entering password");
 
        driver.findElement(By.id("password"))
                .sendKeys("testpassword");
 
        Reporter.log("Clicking login button");
 
        driver.findElement(By.id("loginButton"))
                .click();
 
        Reporter.log("Validating dashboard");
 
        Assert.assertTrue(
            driver.getTitle().contains("Dashboard")
        );
 
        Reporter.log("Login test completed");
    }
 
    @AfterMethod
    public void tearDown() {
 
        Reporter.log("Closing browser");
 
        if (driver != null) {
            driver.quit();
        }
    }
}

62. Complete Listener-Based Reporting Example

import org.testng.ITestListener;

import org.testng.ITestResult;
 
public class TestListener implements ITestListener {
 
    @Override
    public void onTestStart(ITestResult result) {
 
        System.out.println(
            "STARTED: " + result.getName()
        );
    }
 
    @Override
    public void onTestSuccess(ITestResult result) {
 
        System.out.println(
            "PASSED: " + result.getName()
        );
    }
 
    @Override
    public void onTestFailure(ITestResult result) {
 
        System.out.println(
            "FAILED: " + result.getName()
        );
 
        System.out.println(
            "Reason: " + result.getThrowable()
        );
    }
 
    @Override
    public void onTestSkipped(ITestResult result) {
 
        System.out.println(
            "SKIPPED: " + result.getName()
        );
    }
}

63. Registering the Reporting Listener

import org.testng.annotations.Listeners;

import org.testng.annotations.Test;
 
@Listeners(TestListener.class)
public class ReportTest {
 
    @Test
    public void sampleTest() {
        System.out.println("Executing test");
    }
}

64. Complete Reporting Architecture

                 Selenium Test Framework

                           |
                           v
                    TestNG Execution
                           |
             +-------------+-------------+
             |             |             |
             v             v             v
          Passed         Failed       Skipped
             |             |             |
             +-------------+-------------+
                           |
                           v
                     Test Listener
                           |
             +-------------+-------------+
             |             |             |
             v             v             v
           Logs       Screenshots    Exceptions
             |             |             |
             +-------------+-------------+
                           |
                           v
                    Reporting Engine
                           |
             +-------------+-------------+
             |                           |
             v                           v
         HTML Report                XML Report
             |                           |
             +-------------+-------------+
                           |
                           v
                    CI/CD Dashboard

65. Common Mistakes in Test Reporting

• Not generating reports after test execution.
• Not capturing useful failure information.
• Not attaching screenshots for important failures.
• Logging sensitive passwords or tokens.
• Using unclear test names.
• Not recording browser and environment information.
• Sharing incorrect or incomplete reports.
• Ignoring skipped tests.
• Not separating test logs from application logs.
• Creating reports that contain too much unnecessary information.
• Not retaining reports from CI/CD executions.
• Using non-thread-safe reporting code during parallel execution.

66. Best Practices for Test Reports

• Use meaningful test names.
• Include total, passed, failed, and skipped counts.
• Capture screenshots for failed Selenium tests.
• Include exception information.
• Include browser and environment details.
• Include execution duration.
• Use structured logs.
• Do not expose passwords or security tokens.
• Keep reports easy to read.
• Store reports as CI/CD artifacts when appropriate.
• Use listeners for reusable reporting behavior.
• Use IReporter when generating suite-level custom reports.
• Use external reporting libraries when richer visualization is required.
• Make reports suitable for both technical and non-technical stakeholders.

67. Practical Project Structure for Reporting

src

|-- main
|   |-- java
|       |-- pages
|       |-- utilities
|       |-- drivers
|
|-- test
|   |-- java
|       |-- tests
|       |-- listeners
|       |-- data
|
|-- screenshots
|
|-- reports
|
|-- test-output
|
|-- testng.xml
|
|-- pom.xml

68. Reporting Utility Class

A reporting utility can centralize common reporting operations.

public class ReportUtil {

 
    public static void logStep(String message) {
        Reporter.log(message);
    }
 
    public static void logPass(String message) {
        Reporter.log("PASS: " + message);
    }
 
    public static void logFail(String message) {
        Reporter.log("FAIL: " + message);
    }
}

Test classes can then use:

ReportUtil.logStep("Opening application");

 
ReportUtil.logPass("Login successful");
 
ReportUtil.logFail("Dashboard verification failed");

69. Test Report Example

==================================================

             SELENIUM TEST EXECUTION REPORT
==================================================
 
Project       : E-Commerce Application
Environment   : QA
Browser       : Chrome
Execution Date: 30-09-2026
 
--------------------------------------------------
TEST SUMMARY
--------------------------------------------------
 
Total Tests   : 20
Passed        : 16
Failed        : 3
Skipped       : 1
 
Pass Rate     : 80%
 
--------------------------------------------------
TEST RESULTS
--------------------------------------------------
 
LoginTest        : PASS
SearchTest       : PASS
ProductTest      : PASS
CartTest         : PASS
CheckoutTest     : FAIL
PaymentTest      : FAIL
LogoutTest       : PASS
 
--------------------------------------------------
FAILURE DETAILS
--------------------------------------------------
 
CheckoutTest
Exception: NoSuchElementException
Element: checkoutButton
Screenshot: CheckoutTest.png
 
PaymentTest
Exception: TimeoutException
Element: paymentFrame
Screenshot: PaymentTest.png
 
--------------------------------------------------
EXECUTION COMPLETE
--------------------------------------------------

70. Test Reports in CI/CD Architecture

Developer

    |
    v
Git Repository
    |
    v
CI/CD Pipeline
    |
    v
Build Application
    |
    v
Execute Selenium Tests
    |
    v
TestNG
    |
    +----------------+
    |                |
    v                v
HTML Report       XML Report
    |                |
    +--------+-------+
             |
             v
       CI/CD Results
             |
             v
       Team Notification

71. Test Reports and Email Notifications

Automation teams can configure CI/CD systems to notify relevant team members when test execution finishes.

A notification can contain:

• Build number
• Environment
• Total tests
• Passed tests
• Failed tests
• Skipped tests
• Report location
• Build status

72. Test Reports and Defect Tracking

When a test fails, the report can provide information useful for creating or updating a defect.

Failed Test

    |
    v
Analyze Report
    |
    +-- Screenshot
    +-- Exception
    +-- URL
    +-- Browser
    +-- Test Data
    +-- Logs
    |
    v
Create / Update Defect
    |
    v
Developer Investigation

73. Test Reports and Traceability

Test reports can provide traceability between test execution and application functionality. A well-organized report can help determine which functionality was tested and which areas failed.

74. Test Reports and Quality Metrics

Test reports can be used to calculate useful automation metrics.

75. Example Pass Rate Calculation

Suppose a test suite contains 100 tests:

Total Tests  = 100

Passed       = 85
Failed       = 10
Skipped      = 5
 
Pass Rate = 85 / 100 × 100
          = 85%

Such metrics can provide a quick summary of the execution, while detailed reports are still needed to understand individual failures.

76. Test Reports for Daily Automation

Daily automation suites can generate reports automatically.

Daily Schedule

      |
      v
Automation Execution
      |
      v
Selenium + TestNG
      |
      v
Report Generation
      |
      v
Archive Report
      |
      v
Team Notification

77. Test Reports for Nightly Regression

Nightly regression testing can execute a large suite outside normal working hours and generate a report for the QA team to review the next day.

Nightly Trigger

      |
      v
Regression Suite
      |
      v
Selenium Execution
      |
      v
TestNG Report
      |
      v
Failure Analysis
      |
      v
Morning QA Review

78. Difference Between Logs and Reports

79. Difference Between Screenshot and Report

80. Interview Questions on Test Reports

1. What is a test report?

A test report is a structured output containing the results and details of test execution.

2. Does Selenium generate test reports by itself?

Selenium WebDriver focuses on browser automation. Test execution and reporting are generally handled by frameworks such as TestNG or JUnit and reporting tools.

3. What reports does TestNG generate?

TestNG generates HTML and XML-based result information, including an HTML index report and other reporting outputs.

4. What is an emailable report?

An emailable report is a compact HTML report designed to make test results easy to share.

5. What is ITestListener?

ITestListener is a TestNG listener interface used to respond to test execution events such as start, success, failure, and skip.

6. What is IReporter?

IReporter is a TestNG interface used to generate custom reports from completed suite execution information.

7. What is Reporter.log()?

Reporter.log() is used to add log messages that can appear in TestNG-generated reports.

8. Why are screenshots useful in reports?

Screenshots provide visual evidence of the browser state when a test fails.

9. How can a screenshot be captured in Selenium?

Selenium provides the TakesScreenshot interface for capturing screenshots.

10. What is the difference between HTML and XML reports?

HTML reports are primarily designed for human-readable analysis, while XML results are commonly useful for machine processing and CI/CD integrations.

11. Can TestNG reports be customized?

Yes. TestNG provides listeners, reporters, Reporter.log(), and reporting APIs that can be used to extend reporting.

12. What is a listener?

A listener is a component that receives test execution events and can perform actions such as logging, screenshot capture, or reporting.

13. What is the purpose of a reporting listener?

It centralizes reporting-related activities so that individual test classes do not need to duplicate reporting logic.

14. Can reports be generated in CI/CD?

Yes. Selenium and TestNG tests can generate reports during CI/CD execution.

15. Why is test reporting important in regression testing?

Regression reports help identify which existing functionalities passed or failed after application changes.

16. What information should a failure report contain?

A useful failure report can contain the test name, exception, stack trace, browser, environment, URL, logs, test data, and screenshot.

17. Can TestNG reports show parameters?

Yes. TestNG can include parameters used to invoke test methods in its HTML reports.

18. What is the difference between ITestListener and IReporter?

ITestListener is event-oriented and receives test lifecycle events, while IReporter is intended to generate reports from completed suite information.

19. Why should passwords not be printed in reports?

Passwords and other sensitive information should not be exposed because reports may be stored, shared, or accessed by multiple users.

20. What is the purpose of a test summary?

A test summary provides a quick overview of total, passed, failed, and skipped tests and other key execution information.

81. Quick Reference Table

82. Learning Roadmap for Test Reports

1. Understand TestNG test execution.
2. Learn TestNG default HTML reports.
3. Understand passed, failed, and skipped statuses.
4. Learn Reporter.log().
5. Learn ITestListener.
6. Learn IReporter.
7. Implement failure screenshots.
8. Add browser and environment information.
9. Add execution logs.
10. Understand HTML and XML reports.
11. Integrate reports with Maven.
12. Integrate reports with CI/CD.
13. Learn third-party reporting solutions.
14. Build a reusable reporting utility.
15. Create a complete Selenium reporting framework.

83. Practical Exercises

1. Create a Selenium login test and generate a TestNG report.
2. Add Reporter.log() messages to every major test step.
3. Create one passing and one failing test and compare the report.
4. Create a disabled test and observe the skipped result.
5. Implement an ITestListener.
6. Capture a screenshot when a test fails.
7. Add browser and environment information to the report.
8. Use a Data Provider and identify each test data set in the report.
9. Create a custom reporting utility class.
10. Integrate the test suite with Maven.
11. Execute the tests through a CI/CD pipeline.
12. Store generated reports as build artifacts.

84. Real-World Reporting Architecture

                    Test Data

                        |
                        v
                Selenium Test Cases
                        |
                        v
                   TestNG
                        |
          +-------------+-------------+
          |             |             |
          v             v             v
        PASS          FAIL          SKIP
          |             |             |
          |             v             |
          |       Screenshot         |
          |       Exception          |
          |       Logs               |
          |             |             |
          +-------------+-------------+
                        |
                        v
                 Reporting Layer
                        |
          +-------------+-------------+
          |                           |
          v                           v
      HTML Report                XML Results
          |                           |
          +-------------+-------------+
                        |
                        v
                    CI/CD
                        |
                        v
                 Team Notification

85. Summary

Test Reports are an essential component of a professional Selenium automation framework. They transform raw test execution results into structured information that can be analyzed by testers, developers, and other stakeholders.

TestNG provides built-in reporting capabilities, including HTML and XML result generation, and supports extensibility through listeners and reporters. The framework also provides Reporter.log() for adding useful messages to generated reports. :contentReference[oaicite:12]{index=12}

In Selenium automation, effective reporting can include test status, execution time, browser details, environment information, logs, exceptions, screenshots, test data, and other evidence required for failure analysis.

For larger automation frameworks, reporting can be integrated with Page Object Model, Data Providers, Maven, CI/CD pipelines, screenshots, custom listeners, and third-party reporting solutions.

86. Course Resources

Learn more about Selenium automation and professional testing:

• Selenium Training
• Register for Selenium Course Demo

Final Takeaway: A good test report should not only show whether a test passed or failed; it should provide enough useful evidence to understand the execution, identify failures, reproduce problems, and communicate the automation results clearly.