Test Reports

Screenshots & Reporting

<h1>Test Reports in Selenium and TestNG</h1>

<p><strong>Test Reports</strong> are an essential part of a Selenium automation framework because they provide a clear summary of automated test execution. A test report shows which test cases passed, failed, were skipped, or encountered errors during execution.</p>

<p>In Selenium automation, test reports help QA engineers, developers, team leads, and stakeholders understand the health of an application without manually reviewing console output or source code. Reports can contain test names, execution status, execution duration, error messages, screenshots, logs, environment information, and other useful test evidence.</p>

<p>Test reports become especially important when a Selenium framework contains hundreds or thousands of automated test cases. Instead of checking every test manually, the team can review a centralized report and quickly identify failures and execution trends.</p>

<p><strong>Course Resource:</strong> <a href="https://www.justacademy.co/course-detail/selenium-training" target="_blank" rel="noopener noreferrer">Selenium Training</a> | <a href="https://www.justacademy.co/register-for-course-demo" target="_blank" rel="noopener noreferrer">Register for Course Demo</a></p>

<hr>

<h2>1. What is a Test Report?</h2>

<p>A <strong>Test Report</strong> is a document or web-based result generated after test execution that summarizes the outcome of test cases.</p>

<p>A typical report provides information such as:</p>

<ul>

<li>Total number of tests executed.</li>

<li>Number of passed tests.</li>

<li>Number of failed tests.</li>

<li>Number of skipped tests.</li>

<li>Execution time.</li>

<li>Test case names.</li>

<li>Failure reasons.</li>

<li>Exception details.</li>

<li>Execution logs.</li>

<li>Screenshots or other test evidence.</li>

<li>Browser and environment information.</li>

</ul>

<hr>

<h2>2. Why are Test Reports Important?</h2>

<p>Test reports convert raw automation execution results into information that can be easily understood and analyzed.</p>

<ul>

<li><strong>Quick Result Analysis:</strong> Teams can immediately identify passed and failed tests.</li>

<li><strong>Failure Identification:</strong> Reports help identify which test cases failed.</li>

<li><strong>Debugging:</strong> Failure messages, stack traces, screenshots, and logs can help developers investigate problems.</li>

<li><strong>Test Evidence:</strong> Reports provide evidence that automated tests were executed.</li>

<li><strong>Communication:</strong> Test results can be shared with QA engineers, developers, managers, and stakeholders.</li>

<li><strong>CI/CD Integration:</strong> Reports can be generated automatically during pipeline execution.</li>

<li><strong>Historical Analysis:</strong> Teams can compare results across multiple builds.</li>

<li><strong>Professional Automation Framework:</strong> Reporting is an important component of a maintainable automation framework.</li>

</ul>

<hr>

<h2>3. Test Reporting Flow</h2>

<pre><code>Test Cases

Selenium WebDriver

TestNG / PyTest

Test Execution

+----------------+

| |

v v

Passed Failed

| |

+-------+--------+

Test Report Generator

HTML / XML / JSON

Report Dashboard

QA / Developer / Stakeholder</code></pre>

<hr>

<h2>4. Test Reports in Selenium</h2>

<p>Selenium WebDriver itself performs browser automation and does not provide a complete reporting framework. Selenium is generally combined with a test framework and reporting library to create detailed execution reports.</p>

<p>A common Selenium reporting architecture is:</p>

<pre><code>Selenium WebDriver

TestNG

Reporting Library

Detailed Test Report</code></pre>

<p>For example, a Java Selenium framework may use Selenium WebDriver for browser automation, TestNG for test execution, Maven for project management, and ExtentReports or Allure for reporting.</p>

<hr>

<h2>5. TestNG Reports</h2>

<p><strong>TestNG</strong> provides built-in reporting capabilities after test execution. When a TestNG suite is executed, TestNG generates result files that summarize test execution.</p>

<p>TestNG reports can provide information about:</p>

<ul>

<li>Passed tests.</li>

<li>Failed tests.</li>

<li>Skipped tests.</li>

<li>Test methods.</li>

<li>Test classes.</li>

<li>Execution time.</li>

<li>Exceptions and stack traces.</li>

</ul>

<p>TestNG reports are useful for basic execution analysis, while external reporting libraries can be used when richer dashboards and customization are required.</p>

<hr>

<h2>6. TestNG Report Structure</h2>

<pre><code>TestNG Suite

+-- Test

+-- Test Class

+-- Test Method 1

| |

| +-- PASS

+-- Test Method 2

| |

| +-- FAIL

+-- Test Method 3

+-- SKIP</code></pre>

<hr>

<h2>7. Basic TestNG Test Example</h2>

<pre><code>import org.testng.Assert;

import org.testng.annotations.Test;

public class LoginTest {

@Test

public void validLoginTest() {

System.out.println("Valid login test");

Assert.assertTrue(true);

}

@Test

public void invalidLoginTest() {

System.out.println("Invalid login test");

Assert.assertTrue(true);

}

}</code></pre>

<p>After execution, TestNG records the result of each test method and includes it in the generated execution reports.</p>

<hr>

<h2>8. Test Statuses</h2>

<p>Test reports normally categorize test executions into different statuses.</p>

<tr><th>Status</th><th>Description</th></tr>

<tr><td>PASS</td><td>The test executed successfully and all required validations passed.</td></tr>

<tr><td>FAIL</td><td>The test execution completed with a failed assertion or error.</td></tr>

<tr><td>SKIP</td><td>The test was not executed because it was skipped or depended on another failed test.</td></tr>

<tr><td>ERROR</td><td>An unexpected execution problem occurred outside the normal assertion flow.</td></tr>

</table>

<hr>

<h2>9. Passed Test Report</h2>

<p>A passed test indicates that the test completed successfully according to its defined validations.</p>

<pre><code>LoginTest

+-- validLoginTest

+-- PASS</code></pre>

<p>A report can display the test name, execution time, status, and other available information.</p>

<hr>

<h2>10. Failed Test Report</h2>

<p>A failed test indicates that an expected condition was not satisfied or an exception occurred during execution.</p>

<pre><code>LoginTest

+-- invalidLoginTest

+-- FAIL

+-- AssertionError

+-- Screenshot

+-- Stack Trace</code></pre>

<p>A detailed failure report helps the team understand where and why the automation test failed.</p>

<hr>

<h2>11. Skipped Test Report</h2>

<p>A skipped test is a test that was not executed. TestNG can skip tests because of explicit configuration, dependencies, or other execution conditions.</p>

<pre><code>LoginTest

+-- loginTest

+-- SKIPPED

+-- Reason</code></pre>

<hr>

<h2>12. HTML Test Reports</h2>

<p><strong>HTML reports</strong> are widely used because they can be opened in a browser and presented in a visually understandable format.</p>

<p>An HTML report may contain:</p>

<ul>

<li>Test execution summary.</li>

<li>Pass/fail statistics.</li>

<li>Test names.</li>

<li>Execution duration.</li>

<li>Failure details.</li>

<li>Stack traces.</li>

<li>Screenshots.</li>

<li>Environment information.</li>

</ul>

<hr>

<h2>13. Example HTML Report</h2>

<pre><code>Automation Test Report

--------------------------------

Total Tests : 20

Passed : 16

Failed : 3

Skipped : 1

Execution Time: 04:35

Test Results

--------------------------------

Search Test PASS

Cart Test PASS

Checkout Test FAIL

Profile Test PASS</code></pre>

<hr>

<h2>14. ExtentReports</h2>

<p><strong>ExtentReports</strong> is a popular reporting library used with Java automation frameworks. It can generate detailed HTML reports containing test statuses, logs, screenshots, and execution information.</p>

<p>ExtentReports can be integrated with Selenium and TestNG to produce customized and visually rich reports.</p>

<p>A typical architecture is:</p>

<pre><code>TestNG

Selenium Test

ExtentReports

+-- Test Status

+-- Logs

+-- Screenshots

+-- Exceptions

+-- Environment

HTML Report</code></pre>

<hr>

<h2>15. ExtentReports Dependency</h2>

<p>In a Maven-based Java project, a reporting library can be added through the project's dependency configuration. The exact dependency version should be selected according to the framework version and project requirements.</p>

<groupId>com.aventstack</groupId>

<artifactId>extentreports</artifactId>

<version>YOUR_VERSION</version>

</dependency></code></pre>

<p>Using a project-specific version is recommended rather than blindly copying an outdated dependency version.</p>

<hr>

<h2>16. Creating an ExtentReports Object</h2>

<pre><code>ExtentReports extent = new ExtentReports();

ExtentSparkReporter reporter =

new ExtentSparkReporter("test-output/report.html");

extent.attachReporter(reporter);</code></pre>

<p>The reporter is configured with the location where the generated HTML report will be stored.</p>

<hr>

<h2>17. Creating a Test in ExtentReports</h2>

<pre><code>ExtentTest test = extent.createTest("Login Test");

test.info("Starting login test");

test.pass("Login test completed successfully");</code></pre>

<p>The <strong>ExtentTest</strong> object is used to add information, status messages, warnings, failures, and other execution details.</p>

<hr>

<h2>18. Flushing the Report</h2>

<p>After test execution, the report should be flushed so that the collected information is written to the report file.</p>

<pre><code>extent.flush();</code></pre>

<p>Without properly flushing the report, the generated report may not contain all collected execution information.</p>

<hr>

<h2>19. Basic ExtentReports Example</h2>

<pre><code>import com.aventstack.extentreports.ExtentReports;

import com.aventstack.extentreports.ExtentTest;

import com.aventstack.extentreports.reporter.ExtentSparkReporter;

public class ReportExample {

public static void main(String[] args) {

ExtentReports extent = new ExtentReports();

ExtentSparkReporter reporter =

new ExtentSparkReporter(

"test-output/automation-report.html");

extent.attachReporter(reporter);

ExtentTest test =

extent.createTest("Login Test");

test.info("Opening application");

test.info("Entering username");

test.info("Entering password");

test.pass("Login successful");

extent.flush();

}

}</code></pre>

<hr>

<h2>20. Logging Test Steps in Reports</h2>

<p>Detailed test-step logging makes reports easier to understand.</p>

<pre><code>test.info("Opening login page");

test.info("Entering username");

test.info("Entering password");

test.info("Clicking login button");

test.pass("User successfully logged in");</code></pre>

<p>When a failure occurs, logging helps identify the last successfully executed step.</p>

<hr>

<h2>21. Reporting Test Failures</h2>

<p>A reporting framework can record failure information along with the failed test status.</p>

<pre><code>try {

test.info("Executing login test");

// Selenium test steps

test.pass("Login successful");

} catch (Exception e) {

test.fail("Login test failed");

test.fail(e.getMessage());

throw e;

}</code></pre>

<p>The exact reporting implementation should be designed so that the original test failure is still propagated to the test framework.</p>

<hr>

<h2>22. Screenshots in Test Reports</h2>

<p>Screenshots are extremely useful when a Selenium test fails. A screenshot can show the state of the browser at the time of failure.</p>

<p>Typical flow:</p>

<pre><code>Test Failure

Capture Screenshot

Save Screenshot

Attach to Report

Review Failure Evidence</code></pre>

<hr>

<h2>23. Selenium Screenshot Example</h2>

<pre><code>File source =

((TakesScreenshot) driver)

.getScreenshotAs(OutputType.FILE);

Files.copy(

source.toPath(),

Paths.get("test-output/login-failure.png"),

StandardCopyOption.REPLACE_EXISTING

);</code></pre>

<p>The screenshot can then be attached to a reporting framework using the reporting library's supported media or attachment mechanism.</p>

<hr>

<h2>24. Screenshot on Test Failure</h2>

<p>A common framework design is to capture a screenshot automatically whenever a test fails.</p>

<pre><code>Test Execution

Test Passed? ---- Yes ----> PASS Report

Capture Screenshot

Capture Error

Attach Evidence

FAIL Report</code></pre>

<hr>

<h2>25. TestNG ITestListener</h2>

<p><strong>ITestListener</strong> is a TestNG listener interface that can be used to respond to test execution events.</p>

<p>It is commonly used for framework-level reporting, logging, screenshots, and execution monitoring.</p>

<p>Important listener methods include:</p>

<ul>

<li><strong>onTestStart()</strong></li>

<li><strong>onTestSuccess()</strong></li>

<li><strong>onTestFailure()</strong></li>

<li><strong>onTestSkipped()</strong></li>

<li><strong>onStart()</strong></li>

<li><strong>onFinish()</strong></li>

</ul>

<hr>

<h2>26. ITestListener Example</h2>

<pre><code>import org.testng.ITestListener;

import org.testng.ITestResult;

public class TestListener implements ITestListener {

@Override

public void onTestStart(ITestResult result) {

System.out.println(

"Test Started: " +

result.getName());

}

@Override

public void onTestSuccess(ITestResult result) {

System.out.println(

"Test Passed: " +

result.getName());

}

@Override

public void onTestFailure(ITestResult result) {

System.out.println(

"Test Failed: " +

result.getName());

}

@Override

public void onTestSkipped(ITestResult result) {

System.out.println(

"Test Skipped: " +

result.getName());

}

}</code></pre>

<hr>

<h2>27. Registering a TestNG Listener</h2>

<p>A TestNG listener can be registered in different ways depending on the framework architecture. One common approach is using the <strong>@Listeners</strong> annotation.</p>

<pre><code>import org.testng.annotations.Listeners;

@Listeners(TestListener.class)

public class LoginTest {

@Test

public void loginTest() {

System.out.println("Login Test");

}

}</code></pre>

<p>Listeners can also be configured through TestNG XML in larger automation frameworks.</p>

<hr>

<h2>28. onTestSuccess()</h2>

<p>The <strong>onTestSuccess()</strong> method is called when a TestNG test method completes successfully.</p>

<pre><code>@Override

public void onTestSuccess(ITestResult result) {

System.out.println(

"PASS: " + result.getName());

}</code></pre>

<p>This event can be used to add a pass status to a custom report.</p>

<hr>

<h2>29. onTestFailure()</h2>

<p>The <strong>onTestFailure()</strong> method is useful for failure handling.</p>

<pre><code>@Override

public void onTestFailure(ITestResult result) {

System.out.println(

"FAIL: " + result.getName());

}</code></pre>

<p>A Selenium framework can use this event to capture screenshots, browser information, exception details, and logs.</p>

<hr>

<h2>30. onTestSkipped()</h2>

<p>The <strong>onTestSkipped()</strong> method can be used to record skipped test executions.</p>

<pre><code>@Override

public void onTestSkipped(ITestResult result) {

System.out.println(

"SKIPPED: " + result.getName());

}</code></pre>

<hr>

<h2>31. Allure Reports</h2>

<p><strong>Allure</strong> is another reporting solution commonly used with automated testing frameworks. It provides a web-based interface for analyzing test execution results.</p>

<p>Allure reports can present:</p>

<ul>

<li>Test status.</li>

<li>Test duration.</li>

<li>Steps.</li>

<li>Attachments.</li>

<li>Exceptions.</li>

<li>Screenshots.</li>

<li>Categories.</li>

<li>Environment information.</li>

<li>Test history when configured in the reporting workflow.</li>

</ul>

<hr>

<h2>32. Allure Reporting Architecture</h2>

<pre><code>TestNG / Selenium

Test Execution

Allure Results

Allure Report Generation

Web-Based Report</code></pre>

<hr>

<h2>33. Report Attachments</h2>

<p>Attachments provide additional evidence for a test execution.</p>

<p>Common report attachments include:</p>

<ul>

<li>Screenshots.</li>

<li>Browser logs.</li>

<li>Application logs.</li>

<li>Request and response data.</li>

<li>HTML source.</li>

<li>Videos where the framework supports video recording.</li>

</ul>

<p>Attachments should be selected carefully so reports remain useful without exposing sensitive information.</p>

<hr>

<h2>34. Test Execution Summary</h2>

<p>A professional report should provide a high-level summary before detailed test information.</p>

<tr><th>Metric</th><th>Example</th></tr>

<tr><td>Total Tests</td><td>100</td></tr>

<tr><td>Passed</td><td>85</td></tr>

<tr><td>Failed</td><td>10</td></tr>

<tr><td>Skipped</td><td>5</td></tr>

<tr><td>Pass Percentage</td><td>85%</td></tr>

<tr><td>Execution Duration</td><td>25 minutes</td></tr>

</table>

<hr>

<h2>35. Calculating Pass Percentage</h2>

<p>A simple pass percentage can be calculated using:</p>

<pre><code>Pass Percentage =

(Passed Tests / Total Executed Tests) × 100</code></pre>

<p>For example:</p>

<pre><code>Passed Tests = 85

Total Tests = 100

Pass Percentage =

(85 / 100) × 100

= 85%</code></pre>

<p>When reporting metrics, teams should define clearly whether skipped tests are included in the denominator.</p>

<hr>

<h2>36. Test Duration</h2>

<p>Test duration indicates how long an individual test or complete suite took to execute.</p>

<p>Duration information can help identify:</p>

<ul>

<li>Slow test cases.</li>

<li>Long-running browser operations.</li>

<li>Synchronization problems.</li>

<li>Performance bottlenecks in the automation suite.</li>

<li>Opportunities for parallel execution.</li>

</ul>

<hr>

<h2>37. Test Report with Browser Information</h2>

<p>Cross-browser Selenium frameworks can include browser information in reports.</p>

<tr><th>Test</th><th>Browser</th><th>Status</th></tr>

<tr><td>Login Test</td><td>Chrome</td><td>PASS</td></tr>

<tr><td>Login Test</td><td>Firefox</td><td>PASS</td></tr>

<tr><td>Login Test</td><td>Edge</td><td>FAIL</td></tr>

</table>

<p>This makes browser-specific failures easier to identify.</p>

<hr>

<h2>38. Test Report with Environment Information</h2>

<p>Reports can include environment details such as:</p>

<ul>

<li>Environment name.</li>

<li>Application URL.</li>

<li>Browser.</li>

<li>Browser version.</li>

<li>Operating system.</li>

<li>Java version.</li>

<li>Test framework version.</li>

<li>Build number.</li>

<li>Execution date and time.</li>

</ul>

<p>Example:</p>

<pre><code>Environment : QA

Browser : Chrome

OS : Windows

Java : 17

Framework : TestNG

Build : 1024

URL : https://qa.example.com</code></pre>

<hr>

<h2>39. Test Reports with Data Providers</h2>

<p>Data-driven tests can generate multiple test invocations. Reports should make it possible to identify which data set was used for each execution.</p>

<pre><code>Login Test

|-- admin / valid password

| +-- PASS

|-- manager / valid password

| +-- PASS

|-- invalid / wrong password

+-- FAIL</code></pre>

<p>Sensitive values such as passwords should not be displayed in plain text in reports.</p>

<hr>

<h2>40. Test Reports with Page Object Model</h2>

<p>Test reports work well with the <strong>Page Object Model</strong> because the framework can log high-level test actions while page classes handle Selenium interactions.</p>

<pre><code>Test Method

Report Log

Page Object

Selenium WebDriver

Application</code></pre>

<p>For example, a test can report "User login started" while the LoginPage class handles the actual element interactions.</p>

<hr>

<h2>41. Reporting Login Test Execution</h2>

<pre><code>test.info("Opening login page");

loginPage.enterUsername(username);

test.info("Username entered");

loginPage.enterPassword(password);

test.info("Password entered");

loginPage.clickLogin();

test.info("Login button clicked");

test.pass("Login workflow completed");</code></pre>

<p>This creates a readable sequence of test activities in the report.</p>

<hr>

<h2>42. Reporting Assertions</h2>

<p>Assertions verify whether the actual application behavior matches the expected behavior.</p>

<pre><code>String actualTitle = driver.getTitle();

String expectedTitle = "Dashboard";

Assert.assertEquals(actualTitle, expectedTitle);</code></pre>

<p>A reporting framework can record the result of this assertion and, when appropriate, capture failure evidence.</p>

<hr>

<h2>43. Test Reports and Maven</h2>

<p>Maven is commonly used to build Selenium TestNG projects and execute automated tests.</p>

<p>A typical execution flow is:</p>

<pre><code>Maven

Compile Project

Run TestNG

Execute Selenium Tests

Generate Test Results

Generate Reports</code></pre>

<hr>

<h2>44. Test Reports in CI/CD</h2>

<p>Automated reports are particularly useful in CI/CD pipelines because tests can execute automatically after code changes.</p>

<pre><code>Developer Commit

Git Repository

Jenkins / CI Pipeline

Maven Build

TestNG

Selenium Tests

Test Results

HTML Report

Team Notification</code></pre>

<hr>

<h2>45. Jenkins and Test Reports</h2>

<p>Jenkins can execute automated Selenium tests and publish or archive test results depending on the reporting and pipeline configuration.</p>

<p>A typical Jenkins flow is:</p>

<ol>

<li>Developer pushes code.</li>

<li>Jenkins starts a build.</li>

<li>Maven dependencies are resolved.</li>

<li>Selenium tests are executed.</li>

<li>TestNG produces execution results.</li>

<li>Reporting tools generate detailed reports.</li>

<li>Reports are made available to the team.</li>

</ol>

<hr>

<h2>46. Test Reports and Screenshots in CI</h2>

<p>When tests run in CI environments, screenshots become particularly useful because the tester may not be watching the browser directly.</p>

<pre><code>CI Test Failure

Capture Screenshot

Save Artifact

Attach to Report

Review Failure Remotely</code></pre>

<hr>

<h2>47. Logging vs Reporting</h2>

<p>Logging and reporting are related but serve different purposes.</p>

<tr><th>Logging</th><th>Reporting</th></tr>

<tr><td>Records application/test events</td><td>Summarizes test execution results</td></tr>

<tr><td>Useful for debugging</td><td>Useful for result analysis</td></tr>

<tr><td>Can contain detailed technical messages</td><td>Usually presents structured test information</td></tr>

<tr><td>Often generated continuously</td><td>Usually reviewed after or during execution</td></tr>

<tr><td>Examples include Log4j logs</td><td>Examples include TestNG, ExtentReports, Allure reports</td></tr>

</table>

<hr>

<h2>48. Test Reports vs Console Output</h2>

<tr><th>Console Output</th><th>Test Report</th></tr>

<tr><td>Mostly text based</td><td>Can provide structured visual information</td></tr>

<tr><td>Harder to review after long execution</td><td>Easier to analyze</td></tr>

<tr><td>Limited organization</td><td>Can include dashboards and sections</td></tr>

<tr><td>Temporary unless captured</td><td>Can be stored as build artifacts</td></tr>

<tr><td>Limited screenshots</td><td>Can include screenshots and attachments</td></tr>

</table>

<hr>

<h2>49. Custom Test Report</h2>

<p>A custom reporting framework can be created to standardize how automation results are presented.</p>

<p>A custom report may contain:</p>

<ul>

<li>Project name.</li>

<li>Build number.</li>

<li>Environment.</li>

<li>Browser.</li>

<li>Execution date.</li>

<li>Total tests.</li>

<li>Passed tests.</li>

<li>Failed tests.</li>

<li>Skipped tests.</li>

<li>Execution duration.</li>

<li>Failure screenshots.</li>

<li>Exception details.</li>

</ul>

<hr>

<h2>50. Report Folder Structure</h2>

<pre><code>project

|-- src

| |-- main

| |-- test

|-- test-output

| |-- reports

| | |-- automation-report.html

| |

| |-- screenshots

| | |-- login-failure.png

| | |-- checkout-failure.png

| |

| |-- logs

| |-- automation.log

|-- pom.xml

|-- testng.xml</code></pre>

<hr>

<h2>51. Test Report Naming</h2>

<p>Report filenames should be meaningful and consistent.</p>

<p>Examples:</p>

<pre><code>selenium-test-report.html

regression-report.html

smoke-test-report.html

build-1024-report.html</code></pre>

<p>For CI environments, build numbers or execution timestamps can be incorporated into report filenames when appropriate.</p>

<hr>

<h2>52. Regression Test Reports</h2>

<p>Regression testing can generate large numbers of test results. A regression report helps the team understand whether previously working functionality continues to behave as expected.</p>

<tr><th>Module</th><th>Total</th><th>Passed</th><th>Failed</th><th>Skipped</th></tr>

<tr><td>Login</td><td>20</td><td>19</td><td>1</td><td>0</td></tr>

<tr><td>Search</td><td>15</td><td>14</td><td>1</td><td>0</td></tr>

<tr><td>Checkout</td><td>20</td><td>17</td><td>2</td><td>1</td></tr>

</table>

<hr>

<h2>53. Smoke Test Reports</h2>

<p>Smoke tests verify whether the major application functionality is stable enough for further testing.</p>

<p>A smoke report can provide a quick overview:</p>

<pre><code>Smoke Suite

--------------------------------

Search PASS

Cart PASS

Checkout PASS

Logout PASS

--------------------------------

Result: PASS</code></pre>

<hr>

<h2>54. Failed Test Analysis</h2>

<p>A good report should make failure investigation easier.</p>

<p>A failure analysis section can include:</p>

<ol>

<li>Test method.</li>

<li>Execution time.</li>

<li>Browser.</li>

<li>Environment.</li>

<li>Failure message.</li>

<li>Exception type.</li>

<li>Stack trace.</li>

<li>Screenshot.</li>

<li>Relevant logs.</li>

</ol>

<hr>

<h2>55. Handling Screenshots Securely</h2>

<p>Screenshots may contain confidential information such as customer information, account numbers, internal URLs, tokens, or other sensitive application data.</p>

<p>Before attaching screenshots to reports, consider:</p>

<ul>

<li>Masking sensitive information.</li>

<li>Using test accounts instead of real customer accounts.</li>

<li>Restricting report access.</li>

<li>Avoiding screenshots of passwords or secret tokens.</li>

<li>Cleaning old reports according to project retention policies.</li>

</ul>

<hr>

<h2>56. Test Report Best Practices</h2>

<ul>

<li>Use meaningful test names.</li>

<li>Include clear test execution statuses.</li>

<li>Capture screenshots for important failures.</li>

<li>Include useful logs without excessive noise.</li>

<li>Record browser and environment information.</li>

<li>Keep reports easy to navigate.</li>

<li>Store reports as CI build artifacts when required.</li>

<li>Avoid exposing passwords, API keys, tokens, and other secrets.</li>

<li>Use unique report filenames for important builds.</li>

<li>Clean obsolete reports regularly.</li>

<li>Use consistent reporting standards across the framework.</li>

<li>Separate test evidence from sensitive production information.</li>

</ul>

<hr>

<h2>57. Common Mistakes in Test Reporting</h2>

<ul>

<li>Not generating reports after test execution.</li>

<li>Not flushing the reporting object.</li>

<li>Using unclear test names.</li>

<li>Generating reports without failure details.</li>

<li>Not capturing screenshots for Selenium failures.</li>

<li>Creating excessively large reports.</li>

<li>Logging sensitive credentials.</li>

<li>Overwriting reports from parallel builds.</li>

<li>Not attaching useful exception information.</li>

<li>Ignoring report maintenance in CI environments.</li>

<li>Sharing WebDriver or reporting objects unsafely during parallel execution.</li>

</ul>

<hr>

<h2>58. Test Reports with Parallel Execution</h2>

<p>Parallel Selenium execution can improve execution time, but the reporting framework must also be designed for concurrent test execution.</p>

<pre><code>Test Suite

+---- Thread 1 ----> Test A ----> Report

+---- Thread 2 ----> Test B ----> Report

+---- Thread 3 ----> Test C ----> Report

+---- Thread 4 ----> Test D ----> Report

Combined Results</code></pre>

<p>Thread-safe framework design is important when multiple tests write to the same reporting infrastructure.</p>

<hr>

<h2>59. Report Categories</h2>

<p>Reports can categorize failures to make analysis easier.</p>

<p>Possible categories include:</p>

<ul>

<li>Application defect.</li>

<li>Automation defect.</li>

<li>Environment issue.</li>

<li>Data issue.</li>

<li>Synchronization issue.</li>

<li>Browser compatibility issue.</li>

<li>Infrastructure failure.</li>

</ul>

<p>The classification should be based on the actual investigation rather than automatically assuming that every failed automation test represents an application defect.</p>

<hr>

<h2>60. Test Reports and Build History</h2>

<p>Keeping historical test reports can help teams compare test execution results across builds.</p>

<pre><code>Build 100

+-- 92% Passed

Build 101

+-- 94% Passed

Build 102

+-- 89% Passed

Build 103

+-- 96% Passed</code></pre>

<p>Historical data can help teams investigate changes in test stability and identify recurring failures.</p>

<hr>

<h2>61. Complete Selenium Reporting Architecture</h2>

<pre><code> Selenium Test Automation

TestNG Framework

+-------------+-------------+

| |

v v

Test Execution ITestListener

| |

+-------------+-------------+

Reporting Manager

+-------------+-------------+

| | |

v v v

Logs Screenshots Test Status

| | |

+-------------+-------------+

Extent / Allure

HTML Report

CI/CD Build Artifact

Team / Stakeholders</code></pre>

<hr>

<h2>62. Practical Reporting Manager</h2>

<p>Large Selenium frameworks can centralize report creation in a dedicated reporting utility.</p>

<pre><code>public class ReportManager {

private static ExtentReports extent;

public static ExtentReports getReportInstance() {

if (extent == null) {

extent = new ExtentReports();

ExtentSparkReporter reporter =

new ExtentSparkReporter(

"test-output/report.html");

extent.attachReporter(reporter);

}

return extent;

}

}</code></pre>

<p>In production frameworks, thread safety, lifecycle management, report naming, and parallel execution should be handled according to the framework architecture.</p>

<hr>

<h2>63. Complete Practical Selenium Test Report Example</h2>

<pre><code>import com.aventstack.extentreports.ExtentReports;

import com.aventstack.extentreports.ExtentTest;

import com.aventstack.extentreports.reporter.ExtentSparkReporter;

import org.openqa.selenium.By;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.chrome.ChromeDriver;

import org.testng.Assert;

import org.testng.annotations.AfterMethod;

import org.testng.annotations.BeforeMethod;

import org.testng.annotations.Test;

public class LoginTest {

WebDriver driver;

static ExtentReports extent;

ExtentTest test;

@BeforeMethod

public void setup() {

driver = new ChromeDriver();

driver.manage().window().maximize();

if (extent == null) {

extent = new ExtentReports();

ExtentSparkReporter reporter =

new ExtentSparkReporter(

"test-output/login-report.html");

extent.attachReporter(reporter);

}

@Test

public void loginTest() {

test = extent.createTest("Login Test");

test.info("Opening application");

driver.get("https://example.com/login");

test.info("Entering username");

driver.findElement(By.id("username"))

.sendKeys("testuser");

test.info("Entering password");

driver.findElement(By.id("password"))

.sendKeys("testpassword");

test.info("Clicking login button");

driver.findElement(By.id("loginButton"))

.click();

String title = driver.getTitle();

test.info("Validating page title");

Assert.assertTrue(

title.contains("Dashboard"));

test.pass("Login test passed");

}

@AfterMethod

public void tearDown() {

if (driver != null) {

driver.quit();

}

if (extent != null) {

extent.flush();

}

}</code></pre>

<p>This example demonstrates the basic relationship between Selenium, TestNG, assertions, and an HTML reporting library.</p>

<hr>

<h2>64. Test Reports with Data Provider</h2>

<p>When a Data Provider is used, each data set can represent a separate test invocation.</p>

<pre><code>@DataProvider(name = "users")

public Object[][] users() {

return new Object[][] {

{"admin"},

{"manager"},

{"employee"}

};

}

@Test(dataProvider = "users")

public void loginTest(String username) {

System.out.println(

"Testing user: " + username);

}</code></pre>

<p>A detailed reporting framework can record the test name and relevant non-sensitive data for each invocation.</p>

<hr>

<h2>65. Test Reports for E-Commerce Testing</h2>

<p>An e-commerce automation framework may generate reports for:</p>

<ul>

<li>Login.</li>

<li>Product search.</li>

<li>Product filtering.</li>

<li>Product details.</li>

<li>Cart validation.</li>

<li>Checkout.</li>

<li>Payment workflow.</li>

<li>Order confirmation.</li>

<li>Logout.</li>

</ul>

<p>Example:</p>

<pre><code>E-Commerce Regression Report

Product Search PASS

Product Filter PASS

Product Details PASS

Add to Cart PASS

Cart Validation FAIL

Checkout PASS

Order Confirmation PASS

Logout PASS</code></pre>

<hr>

<h2>66. Test Report and Defect Management</h2>

<p>Test reports can help QA teams identify failed scenarios that may require further investigation. A report should provide enough evidence to reproduce or analyze a failure.</p>

<p>A typical defect investigation may use:</p>

<pre><code>Failed Test

+-- Test Name

+-- Build Number

+-- Environment

+-- Browser

+-- Error Message

+-- Stack Trace

+-- Screenshot

+-- Logs

Defect Investigation</code></pre>

<hr>

<h2>67. Report Retention</h2>

<p>Automation reports can accumulate quickly in CI environments. A project should define how long reports, screenshots, logs, and other artifacts should be retained.</p>

<p>Good practices include:</p>

<ul>

<li>Keep important release reports.</li>

<li>Remove unnecessary old reports.</li>

<li>Archive reports according to project requirements.</li>

<li>Control access to sensitive test evidence.</li>

<li>Prevent unlimited growth of the report directory.</li>

</ul>

<hr>

<h2>68. Test Report Quality Checklist</h2>

<ul>

<li>Does the report show total test count?</li>

<li>Does it show passed tests?</li>

<li>Does it show failed tests?</li>

<li>Does it show skipped tests?</li>

<li>Can failed tests be identified easily?</li>

<li>Are exception details available?</li>

<li>Are screenshots available for relevant failures?</li>

<li>Are test steps understandable?</li>

<li>Is browser information available?</li>

<li>Is environment information available?</li>

<li>Are sensitive credentials excluded?</li>

<li>Can the report be accessed from CI/CD?</li>

</ul>

<hr>

<h2>69. Interview Questions on Test Reports</h2>

<h3>1. What is a test report?</h3>

<p>A test report summarizes the results and relevant evidence of automated or manual test execution.</p>

<h3>2. Why are test reports important in Selenium?</h3>

<p>They provide a structured view of automation results and help teams analyze failures, execution status, and test evidence.</p>

<h3>3. Does Selenium itself generate detailed test reports?</h3>

<p>Selenium WebDriver focuses on browser automation. Detailed reporting is generally provided by the test framework and reporting libraries integrated with the Selenium framework.</p>

<h3>4. What is the role of TestNG in reporting?</h3>

<p>TestNG manages test execution and provides execution results that can be consumed by reporting systems.</p>

<h3>5. What is ExtentReports?</h3>

<p>ExtentReports is a reporting library used to create detailed and customizable HTML reports for automated tests.</p>

<h3>6. What is Allure?</h3>

<p>Allure is a reporting solution that provides a web-based interface for analyzing automated test results, steps, attachments, and failures.</p>

<h3>7. What is ITestListener?</h3>

<p>ITestListener is a TestNG listener interface that allows automation frameworks to react to test execution events such as start, success, failure, and skip.</p>

<h3>8. How can screenshots be added to reports?</h3>

<p>A Selenium screenshot can be captured using TakesScreenshot and then attached to the reporting framework using its supported attachment mechanism.</p>

<h3>9. Why are screenshots useful in reports?</h3>

<p>They provide visual evidence of the browser state when a test fails.</p>

<h3>10. What is the difference between logging and reporting?</h3>

<p>Logging records technical events and messages, while reporting presents structured test execution results and evidence.</p>

<h3>11. What information should a good test report contain?</h3>

<p>It should normally contain test status, test names, execution duration, failure details, environment information, and useful evidence such as screenshots and logs.</p>

<h3>12. Can reports be generated in CI/CD?</h3>

<p>Yes. Selenium TestNG reports can be generated during CI/CD execution and stored or published as build artifacts according to the pipeline configuration.</p>

<h3>13. What is test report aggregation?</h3>

<p>Report aggregation combines results from multiple tests, classes, suites, browsers, environments, or parallel executions into a consolidated report.</p>

<h3>14. Why should passwords not be displayed in reports?</h3>

<p>Passwords and other secrets are sensitive information and should not be unnecessarily exposed in logs or reports.</p>

<h3>15. What is a failed test report?</h3>

<p>It is a report entry showing that a test did not satisfy its expected condition or encountered an execution error.</p>

<h3>16. What is a skipped test?</h3>

<p>A skipped test is a test that was not executed because of an execution condition, dependency, configuration, or explicit skip behavior.</p>

<h3>17. Why should test reports include browser information?</h3>

<p>Browser information helps identify browser-specific failures in cross-browser automation.</p>

<h3>18. What is report flushing?</h3>

<p>Flushing writes the collected reporting information to the configured report output.</p>

<h3>19. Can Data Provider executions appear separately in reports?</h3>

<p>Yes. TestNG can represent individual Data Provider invocations as separate test executions, depending on the reporting integration.</p>

<h3>20. What makes a test report useful?</h3>

<p>A useful report is clear, searchable, sufficiently detailed for failure investigation, and free from unnecessary sensitive information.</p>

<hr>

<h2>70. Quick Reference Table</h2>

<tr><th>Concept</th><th>Description</th></tr>

<tr><td>Test Report</td><td>Summary of test execution results and evidence.</td></tr>

<tr><td>TestNG</td><td>Java testing framework used to execute and manage tests.</td></tr>

<tr><td>ExtentReports</td><td>Reporting library for detailed HTML test reports.</td></tr>

<tr><td>Allure</td><td>Web-based test reporting solution.</td></tr>

<tr><td>ITestListener</td><td>TestNG listener for test execution events.</td></tr>

<tr><td>Screenshot</td><td>Visual evidence of browser state.</td></tr>

<tr><td>Log</td><td>Technical information recorded during execution.</td></tr>

<tr><td>PASS</td><td>Test completed successfully.</td></tr>

<tr><td>FAIL</td><td>Test did not meet the expected result or encountered an error.</td></tr>

<tr><td>SKIP</td><td>Test was not executed.</td></tr>

<tr><td>CI/CD</td><td>Automated build and test execution environment.</td></tr>

</table>

<hr>

<h2>71. Learning Roadmap for Test Reports</h2>

<ol>

<li>Understand Selenium and TestNG test execution.</li>

<li>Learn TestNG result generation.</li>

<li>Understand PASS, FAIL, and SKIP statuses.</li>

<li>Learn HTML test reporting.</li>

<li>Learn ExtentReports concepts.</li>

<li>Learn Allure reporting concepts.</li>

<li>Understand ITestListener.</li>

<li>Implement failure screenshots.</li>

<li>Add logs to reports.</li>

<li>Add environment and browser information.</li>

<li>Integrate reports with Page Object Model.</li>

<li>Integrate reports with Data Providers.</li>

<li>Integrate reports with Maven.</li>

<li>Publish reports through CI/CD pipelines.</li>

<li>Build a reusable reporting component for a Selenium framework.</li>

</ol>

<hr>

<h2>72. Practical Exercises</h2>

<ol>

<li>Create a basic TestNG test and inspect the generated test results.</li>

<li>Create an HTML report for a Selenium login test.</li>

<li>Generate separate pass and fail test results.</li>

<li>Create a custom report using a reporting library.</li>

<li>Add test-step logging to the report.</li>

<li>Capture screenshots when Selenium tests fail.</li>

<li>Implement an ITestListener.</li>

<li>Add browser information to the report.</li>

<li>Add environment information to the report.</li>

<li>Create a report for a Data Provider-based login test.</li>

<li>Integrate reporting with Page Object Model.</li>

<li>Integrate test reports into a Maven project.</li>

<li>Generate reports during CI/CD execution.</li>

<li>Create a complete Selenium reporting framework.</li>

</ol>

<hr>

<h2>73. Real-World Selenium Reporting Project</h2>

<p>Consider an e-commerce application where a QA automation team executes login, search, product, cart, checkout, and logout tests.</p>

<pre><code>E-Commerce Automation

TestNG Suite

+-- Login Tests

+-- Search Tests

+-- Product Tests

+-- Cart Tests

+-- Checkout Tests

+-- Logout Tests

Selenium WebDriver

Assertions

ITestListener

+-- Pass

+-- Fail

+-- Skip

+-- Screenshot

+-- Logs

ExtentReports / Allure

HTML Report

CI/CD Pipeline</code></pre>

<hr>

<h2>74. Summary</h2>

<p><strong>Test Reports</strong> are a critical component of professional Selenium automation frameworks. They convert test execution information into a structured format that can be easily reviewed by QA engineers, developers, automation engineers, and stakeholders.</p>

<p>Selenium WebDriver performs browser automation, while TestNG manages test execution and reporting integrations can provide detailed HTML-based results, logs, screenshots, and other evidence.</p>

<p>Popular reporting approaches include TestNG's built-in results, ExtentReports, Allure, and custom reporting solutions. TestNG listeners such as <strong>ITestListener</strong> can be used to automatically handle test-start, success, failure, and skip events.</p>

<p>For a production-quality Selenium framework, reporting can be integrated with Page Object Model, Data Providers, Maven, screenshots, logging, cross-browser execution, parallel execution, and CI/CD pipelines.</p>

<p>A well-designed report should be clear, useful for debugging, easy to access, and careful not to expose sensitive information such as passwords, tokens, or confidential application data.</p>

<hr>

<h2>75. Course Resources</h2>

<p>Learn more about Selenium automation, TestNG, reporting, frameworks, and practical testing:</p>

<ul>

<li><a href="https://www.justacademy.co/course-detail/selenium-training" target="_blank" rel="noopener noreferrer">Selenium Training</a></li>

<li><a href="https://www.justacademy.co/register-for-course-demo" target="_blank" rel="noopener noreferrer">Register for Selenium Course Demo</a></li>

</ul>

<p><strong>Final Takeaway:</strong> Test Reports make Selenium automation results understandable, traceable, and useful for debugging and decision-making. By combining TestNG execution, listeners, screenshots, logs, and reporting tools such as ExtentReports or Allure, automation teams can create a professional reporting system that supports local testing as well as CI/CD-based execution.</p>

</div>

Popular Searches

Popular Course Categories

Popular Courses

Curriculum

Introduction to Software Testing

Introduction to Selenium

Selenium Environment Setup

WebDriver Fundamentals

HTML & Web Elements

Selenium Locators

XPath in Selenium

Working with Web Elements

Waits in Selenium

Handling Browser Components

Dropdowns & Advanced Elements

Mouse & Keyboard Actions

JavaScript with Selenium

Selenium with Python

Selenium with Java

Test Automation Framework

TestNG / PyTest

Page Object Model

Data-Driven Testing

Selenium Grid

Advanced Selenium

Screenshots & Reporting

API & Database Integration

Selenium with CI/CD

Real-World Selenium Projects

Job Preparation

Test Reports

+91-9987184296