Tracking Changes in Reports

Smart provides the functionality to run Reports and compare their outputs on re-execution.

This allows you to verify that a specific initiator can run a Report in a given Workday tenant as well as confirm that Report executes successfully and produces the consistent output on re-execution.

In addition, it will allow performing quick tenant health check by executing and comparing audit and exception Reports.

'Report' module permissions are required to tun this type of test.

Please refer to User Management section for more information.

Creating Tracking Changes in Reports Test Runs

Users with the report tester or manager role can access report tests under the menu item CREATE A TEST.

You can create a 'Track Changes in Report' test by navigating to CREATE A TEST > Reports > Track Changes in Reports.

Smart supports launching all types of Workday Reports, which Output Type is set to Table.

Smart uses the Schedule a Report Task in Workday to run the Reports. The 'Don't share report output' option is selected to stop the Report outputs from being viewed by everyone.

On the initial Report execution, Smart will retrieve its output and stored as a baseline. On re-execution, Smart compares the current output and with the previous output (baseline). Smart stores the outputs for 14 days, after this time a re-execution will not return a comparison (due to the missing baseline). Instead, the re-execution will generate a new baseline to compare against going forward.

Not all reports are available for execution. You can view the list of approved reports on the Create Report Comparison Test page.

If you wish to execute additional Reports please contact the Kainos Smart Onboarding or Kainos Smart Support team for more information.

The process of downloading/populating/uploading the data template is similar to creating a regular Business Process Test Run. Users have the option to upload a template for a "Report Comparison" test run or download a blank one. 

Clicking the "Add a test pack" link will add another Report test entry. This can be useful if you prefer to keep your tests in logical groupings rather than a larger single pack.  

The Report data template must be populated with data from the Workday tenant. This data will consist of key fields like:
- Report name
- Report initiator
- Report launch criteria required to run a Report.

Like Business Process templates, each column represents a test case. You can add test case data from column F onward.

Key Report data template fields/sections to populate are:
Login field - Report initiator's username
Report Name field - name of the Report to run (Report WID or Reference ID can be provided as well)
Report Criteria section - Report launch criteria are composed of two parameters each:

  • Field - name of the Report parameter
  • Value - value for a given Report parameter field

Report Criteria are repeatable sections to allow for multiple report launch parameters. For every launch parameter a report has there must be a matching report criteria in the template.  

To select more than one value for a single launch parameter you can add additional Value rows per Report Criteria.

To complete the Report Comparison test run creation, upload the completed data template by selecting the Upload Files option, set the Report Comparison test run name and save the test run. Optionally, Report test runs can also be tagged.

Executing Report Comparison Test Runs

You can execute Report Comparison test runs the same way as other test run types in Smart.  Please refer to Test Run Execution section for more details.

Viewing Report Comparison Test Runs Results

Once executed, users can start to the review the results as the test cases complete. Clicking on a Report Comparison test run, users will go to the Test Run Results page with the following sections: 

  • Executions Details - provides test run execution details
  • Overall Results Summary - provides test run results summary
  • Test Cases - provides Reports execution and comparison details

Viewing Report Comparison Test Runs Results for baseline execution

The Test Cases section provides execution and comparison overview for all the Report test cases. There are following columns available:

  • Test Case ID - Report test case ID
  • Process - Report Comparison Process name
  • Name - name of the executed Report
  • Expected Result
  • Execution Status -  providing information whether the Report executed successfully or not
  • Result Status - Test case Result Status

In the baseline Report execution, Result Status is automatically set basing on Execution Status:

Execution Status Result Status
Completed PASSED
Failed to Complete FAILED

Test Result can be overridden on the Test Run Results or Test Case Results page. 


Clicking the Test Case ID link will take you to the Test Case Results page.

The following information is displayed:

  • Test Case Details section:
    • Result
    • Test Case ID
    • Test Case Name
    • Report Name
    • Duration
    • Tenant
    • Expected Result
    • Comment

Clicking on the Change Results link allows for changing the Test Case Result.


There are 3 tabs available below Test Case Details section:

  • Steps - providing Report execution information with a View in Workday link to Schedule a Report background process. You need to be logged in as the Report initiator to view the Report output in Workday.
  • Test Input Data - Report launch parameters defined for a test case
  • Debug - detailed execution information

Re-Executing Report Comparison Test Runs

You can re-execute existing test runs against the same Workday tenant or you can choose a different one. 

Report Comparison test runs re-execution is similar to other test types re-execution covered in Test Run Re-execution section.

Viewing Report Comparison Test Runs Results for re-executions

The Test Cases section provides execution and comparison overview for all the Report test cases. There are following columns available:

  • Test Case ID - Report test case ID
  • Process - Report Comparison Process name
  • Name - name of the executed Report
  • Expected Result
  • Execution Status - providing information whether the Report executed successfully or not
  • Comparison Result - comparison result of Report outputs from current and previous execution
  • Result Status - Test case Result Status

In the Report re-execution, Result Status is automatically set basing on Comparison Result:

Comparison Result Result Status
MATCH PASSED
NON MATCH FAILED

Test Result can be overridden on the Test Run Results or Test Case Results page. 


Clicking the Test Case ID link will take you to the Test Case Results page.

The following information is displayed:

  • Test Case Details section:
    • Result
    • Comparison Result
    • Test Case ID
    • Test Case Name
    • Report Name
    • Duration
    • Tenant
    • Expected Result
    • Previous Execution - link to previous execution of this test case
    • Comment

Clicking on the Change Results link allows for changing the Test Case Result.


There are 4 tabs available below Test Case Details section:

  • Comparison - providing Report outputs comparison details
  • Steps - providing Report execution information with a View in Workday link to Schedule a Report background process. You need to be logged in as the Report initiator to view the Report output in Workday.
  • Test Input Data - Report launch parameters defined for a test case
  • Debug - detailed execution information


In case there are no differences between the Reports outputs the following information would be displayed:

If there are differences between the Reports outputs they will be displayed in Smart: