Comparing Reports Between Workday Tenants

Smart provides the functionality to compare a Report between Workday tenants, which extends to allow you to run a set of reports on two different Workday tenants and compare their outputs. This can be used during Workday Update periods to verify the output of reports will not change on the new version of Workday. In addition, it will allow verification of the reports when migrating between tenants. 

'Report' module permissions are required to tun this type of test.

Please refer to User Management section for more information.

You need to have two Workday tenants configured in your Smart tenant to run this comparison test. Smart will prompt you to configure a second Workday tenant if there is only one present. 

Creating Report Tenant to Tenant Comparison Test Runs

Once 'Report' permissions are granted, Smart users will see the new Reports option in the CREATE A TEST menu.

"Compare Reports between Tenants" test run can be created by navigating to CREATE A TEST > Reports > Compare Reports Between Tenants.

Smart supports launching all types of Workday Reports, which Output Type is set to Table.

The process of downloading/populating/uploading the data template is similar to creating a regular Business Process Test Run. Users have the option to upload a template for a "Report Tenant to Tenant Comparison" test run or download a blank one. 

Clicking the "Add a test pack" link will add another Report test entry. This can be useful if you prefer to keep your tests in logical groupings rather than a larger single pack. 

The Report data template must be populated with data from the Workday tenant. This data will consist of key fields like:
- Report name
- Report initiator
- Report launch criteria required to run a Report.

Like Business Process templates, each column represents a test case. You can add test case data from column F onward.

Key Report data template fields/sections to populate are:
- Login field - Report initiator's username
- Report Name field - name of the Report to run (Report WID or Reference ID can be provided as well)
- Report Criteria section - Report launch criteria are composed of two parameters each:

  • Field - name of the Report parameter
  • Value - value for a given Report parameter field

Report Criteria are repeatable sections to allow for multiple report launch parameters. For every launch parameter a report has there must be a matching report criteria in the template.  

To select more than one value for a single launch parameter you can add additional Value rows per Report Criteria.

To complete the Report Tenant to Tenant Comparison test run creation, upload the completed data template by selecting the Upload Files option, set the Report Comparison Tenant to Tenant test run name and save the test run. Optionally, Report test runs can also be tagged.

Executing Report Tenant to Tenant Comparison Test Runs

You can execute Report Tenant to Tenant Comparison test runs the same way as other test run types in Smart.  Please refer to Test Run Execution section for more details.

The only difference is that you will need to provide two Workday tenants instead of one.

Report Tenant to Tenant Comparison test runs require two configured Workday tenants to execute against. Smart will prompt you to configure a second Workday tenant if there is only one found.  

Viewing Report Tenant to Tenant Comparison Test Runs Results

Once executed, users can start to the review the results as the test cases complete. Clicking on a Report Tenant to Tenant Comparison test run, users will go to the Test Run Results page with the following sections: 

  • Executions Details - provides test run execution details
  • Overall Results Summary - provides test run results summary
  • Test Cases - provides Reports execution and comparison details

The Test Cases section provides execution and comparison overview for all the Report test cases. There are following columns available:

  • Test Case ID - Report test case ID
  • Report Name - name of the executed and compared Report
  • First Workday tenant Status - Report execution status in first Workday tenant
  • Second Workday tenant Status - Report execution status in second Workday tenant
  • Comparison Status - Reports comparison status 
  • Test Result - test case result


Smart sets Comparison status and Test Case Results for you based on the following: 

1st Tenant Execution Status 2nd Tenant Execution Status
Comparison Status
Completed Completed
MATCH / NON MATCH
Completed
Failed to Complete
NON MATCH
Failed to Complete
Completed
NON MATCH
Failed to Complete
Failed to Complete
COULD NOT TEST
Comparison Status Test Case Result
MATCH
Passed
NON MATCH Failed
COULD NOT TEST Failed

Test results can be overridden on the Test Case Results page. 


Clicking the Test Case ID link will take you to the Test Case Results page.

The following information is displayed:

  • Test Case Details section:
    • Result
    • Comparison Result
    • Test Case ID
    • Report Name
    • Report Initiator
    • Expected Result
    • Comment

Clicking on the Change Results link allows for changing the Test Case Result.


There are 4 tabs available below Test Case Details section:

  • Comparison
  • First Workday tenant
  • Second Workday tenant
  • Test Data


Comparison Tab


Comparison is the default tab, showing potential differences between Report outputs from both Workday tenants.

Smart only shows the differences found along with surrounding lines for context. Each difference shows: 

  • 10 lines above the difference
  • line containing difference
  • 10 lines below the difference

This is dependent on the number of differences and their location in the file.

In addition, Smart will display up to 20 lines containing differences. If this threshold is exceeded user will be advised to review the differences in the Workday tenant(s).


First/Second Workday tenant tab


The Workday Tenants tabs will show you details of how each report performed on a given Workday tenant. If there was a problem running the Integration then investigation should be conducted here.

The following information is displayed:

  • Steps tab - Reports execution information
  • Debug tab - Detailed execution information

Clicking on View in Workday link will take you to Report Background Process in the Workday tenant.

Re-Executing Report Tenant to Tenant Comparison Test Runs

You can re-execute existing test runs against the same tenants or you can choose a different set. 

Report Tenant to Tenant Comparison test runs re-execution is similar to other test types re-execution covered in Test Run Re-execution section, with two exceptions:

  1. You need to specify two Workday tenants  
  2. You cannot select test cases for re-execution, all test cases are re-executed