View Test Runs
The TEST RUNS screen displays high level information on your Smart Test Runs. More detailed information on the results of tests can be displayed by clicking on the test run name which will take the user to the Test Results screen (see the Viewing Test Results section).
The TEST RUNS screen will initially display the most recent ten test runs available as per the users permissions. This section of the user guide will focus on the options available to you from this screen.
Introduction to the Test Runs screen
The Tests Runs screen has four key areas. The following four sections highlight the features within each of these areas:
- Test Runs Status
- Test Run Related Actions
- Enhanced Features
Test Run Status
The Test Run Status section of the screen provides a snapshot of the current state of the Test. The potential states that the Test Run can be in is as follows:
- New - The Test Run has been recently generated, has not been executed previously and currently is not been scheduled to be run at a future point. Note when copying an existing test that has been run previously, the default status of the new test is 'New' until it is run or scheduled.
- Scheduled - The Test Run has been scheduled to run automatically at a future point in time. Note that you can stipulate the date and time for execution of the test at the test execution screen.
- In Progress - The Test Cases within the Test Run are currently being executed by Smart,
- Cancelled - The Test Run was cancelled while Scheduled or In Progress. If a Test Run is cancelled, any running tests will complete, while tests not yet started will not begin execution.
- Completed - The Test Run has been executed to completion. If critical errors occurred while the test was executed, these will be displayed after the status.
Test Run Related Actions
Clicking the Related Actions "..." button in the Test Run row will present a list of the available actions that can be carried out against the test. The actions displayed change depending on the status of the Test Run. The Test Run States & Actions section of this guide list out the Action combinations displayed on the Test status.
The available actions that can be executed against a test are as follows. Please see the Test Run States & Actions section to better understand which of these options will display depending on your test's status.
- Execute - Execute the Test Run.
- Reschedule - Reschedule a Test Run that has been scheduled to run at a future point.
- Unschedule - Unschedule a future scheduled test.
- Cancel - Stop the execution of a test that is currently 'In Progress'.
- Download Test Data - Download the templates from the test run in a zipped file.
- Duplicate - Create a copy of your test run (the copy will be amended with the '- copy' suffix and appear at the top of your list of tests)
- Delete - Delete the Test Run (you will be prompted to confirm deletion)
- Export to CSV - Exports the test results in CSV format.
- Execution History - View the Execution History for the chosen Test Run. For more information on Execution History see Consolidated Test Runs
The name of the Test Run is a hyperlink. When selecting the hyperlink, the user is taken to the a screen which displays additional information on the Business Process/Integration Test Case. This allows the user to modify the contents of the Test Run before execution as required, if the test run status is "New".
Test Run States & Actions
The related actions that are available for each Test Run depend on the status of the Test Run. The table below outlines the options available depending on the Test Run status. Note that Related Actions marked with a tick symbol will be available against the Test Run at the specified status.
Download Test Data & Export to CSV
Smart offers the ability to download the Test Data used for each Test Case. The purpose of this functionality is to allow the user to easily obtain all the templates used in a test run, which can then be updated and used in future test runs. Additionally the user can select Export to CSV to produce a CSV file containing a record of the test results.
In order to access the Test Data or CSV files downloaded from Smart, firstly select the Related Action from the Test Run. A pop-up will be presented showing progress of the download to the user in the left hand side of the browser window. Upon completion, clicking the appropriate 'Download...' hyperlink will download the selected file to your local machine.
Note that the 'Download Test Data' option produces a .zip file which will need to be extracted.
Filtering Test Runs
The objective of Filtering Test Runs in Smart is to increase or decrease the number of Test Runs that are displayed to a user allowing them to focus on a specific task.
Smart offers three default Filters options to the user that allow the most common Test Run display options.
- Created by you - Displays all of the users Test Runs.
- Executed by you - Displays all of the Test Runs that have been executed by the user.
- New - Displays all of the Test Runs that have not yet been executed.
Additionally the user has the option to define a new enhanced filter tailored to their needs by selecting the + More Filters hyperlink. The enhanced filtering options available to the user are split into two sections, Test run details and Execution details. Any combination of filters can be used at the same time.
Test Run Details - Filters
- Test run name contains - Filters to display Test Runs that contain a specific value in the name of the Test (case-sensitive).
- Tags - Filters to only show Test Runs containing a specific tag.
- Created by - Filters based on the Smart user who created the Test. More than one user can be selected in this field.
- Test run type - Filters based on the type of Workday functionality that is being tested.
- Business Process - Filters based on a specific Workday business process that a test has been defined for e.g. Hire.
Execution Details - Filters
- Date range - Allows filtering on a specific date range when a test has been executed.
- Executed by - Allows filtering on who executed the test.
- Status - Allows filtering on the Test execution status.
- Tenant - Allows filtering based on the Workday Tenant that the test was executed in.
- Workday version - Allows filtering based on the Workday Tenant Version that the test was executed in.
When the user is finished selecting their filters, they can then select the 'APPLY' button and Smart will filter the test runs appropriately. 'APPLY AND HIDE' will filter the test runs and automatically close the Filter menu. 'RESET' can be used to clear all currently populated filters.
Filtered Test Runs
The filtering options set on the Test Runs page will remain in place until the user clicks the 'clear filters' hyperlink highlighted above.
Smart provides users the ability to add tags to Test Runs. These tags can be used to easily group together Test Runs, allowing you to filter to only show Test Runs that match the specified tags. Tags can only contain characters ranging from A to Z (upper and lower case), numeric characters from 0 to 9 and ' " or -.
Tags can be created or added in three ways:
- When Creating Test Runs - When creating a Test Run, there is a 'Tags' field available under the 'Test run' name field. Entering a tag(s) into this field and hitting the Enter key will add the tag to the Test Run. Upon clicking 'Save test run' this tag will then be saved to the Smart tenant, with it then being made available to use again when creating further Test Runs via a drop down list.
- When Executing Test Runs - When executing a Test Run, there is a 'Tags' field available at the bottom of the Execute Test Run screen into which tags can be added/created.
- After Executing Test Runs - After a Test Run has been executed, tags can be added by entering the results screen and clicking the tag icon under the Test Run name. This will open up a field where you can add/create tags.
If a tag is added incorrectly, users can click the 'x' icon on the tag to remove it.
Adding Tags When Creating Test Runs
Adding Tags When Executing Test Runs
Adding Tags After Executing Test Runs
Test Run Enhanced Features
In addition to providing functionality to filter on Test Runs as well as view states and perform related actions against a specific Test Run, Smart offers additional features which can be applied against more than one Test Runs.
- Delete Test Runs
- Download Results
- Generate Summary
Deleting Test Runs
Selecting the Delete Test Runs button allows the Smart user to Delete existing Test Runs that are no longer required. Note that the user can select a number of Test Runs to delete by selecting the corresponding check-box(es).
The 'Delete Selected' button confirms the deletion of the Test Run. The 'Cancel' button allows the user to cancel the delete operation.
Selecting the DOWNLOAD RESULTS button allows users to download multiple sets of test results for the selected Test Runs. Whilst the individual related actions button can be used to download individual results as a CSV file, using the DOWNLOAD RESULTS button will generate a compressed file containing all of the results (as individual CSV files) for each selected Test Run.
The DOWNLOAD RESULTS button confirms downloading of the Test Run results. The CANCEL button allows the user to cancel the delete operation.
Generating A Summary Report
Selecting the GENERATE SUMMARY button allows the user to produce a Summary Report for the selected Test Runs. Note that the user can select a number of Test Runs to report on by selecting the corresponding check-box(es). Note that only completed test runs can be selected for inclusion in the report.
The 'Generate Report' button produces the report against the corresponding Test Runs. The 'Cancel' button allows the user to cancel the generate report operation. For more information on this functionality, see the Additional Reporting section of the user guide.