Frequently Asked Questions

Due to the nature of how Smart tests your business processes, from time to time you may see tests passing/failing when you were not expecting this to be the case, or you may find that you need to update your data templates. This article will attempt to address some of the common questions asked by our customers.

My tests ran successfully last week, but this week they are now failing. Has something gone wrong?

Smart tests a Workday tenant as if it were a real-time user, this means that any potential configuration changes which have been made in the tenant week on week can cause Smart tests to fail. This is due your templates being populated with configuration which is no longer valid. Smart offers the functionality to do test-comparison when re-executing a test pack which will highlight changes in your business process flow or security configuration. Here are some examples of issues which would cause failures:-

  • Terminated Employees
    One of the most common causes of a test failing is that the employee, either the initiating employee or the subject employee, is no longer available on your tenant. This could be because they were terminated, or are pending termination and can no longer be proxied into. This can generally be resolved by utilising the "Dynamic Initiator" feature in Smart. This will pick a worker who is able to perform the required task by querying the initiating grops in the BP policy. 
  • Security Permissions
    Smart functions according to the security configuration of a tenant. This means that if a worker is not able to carry out an action on another worker, this is because they are unable to do it manually.
  • Proxy Policy
    Smart can be configured to run tests by proxying as workers. In this case it is subject to the rules dictated in a tenant's proxy access policy. If this policy has been set up or modified to exclude certain groups from allowing proxy, this may cause issues with Smart tests. In order to resolve this the proxy policy would need updated with the Kainos User. 
  • Workday Performance
    Often times if there are performance issues on the Workday tenant itself, this can affect Smart tests from executing successfully. For instance if something takes a long time in Workday (e.g. searching for a worker or a specific task name) this may cause Smart to time-out as it waits for a response from the tenant. Smart includes the funtionality to re-execute failed tests without re-executing the whole pack, this would be useful in theis scenario. 
  • Workday Configuration
    On the other side of Workday, if there are gaps in your configuration (e.g. an unassigned role due to a termination) this will also cause tests to fail in Smart if it cannot find another worker that is able to handle the step. Smart will highlight the issue and this will need addressed using your internal change control process. 

My tests seem to take a long time to run. Why is this?

Smart tests your Workday business processes in real-time. As this is the case, depending on the complexity of your business process, as well as how many steps in the business process need to be handled, the timing of your tests can vary. As Smart is an automated testing tool, it allows a certain amount of time for each step to be considered complete to account for issues beyond its control (e.g. Workday tenant performance). As well as this, Smart runs reports between business process steps to assist in the automation of the testing. Whilst these reports themselves take minimal time to execute, they can also contribute to the total time taken for a test to run.

With Smart Security testing, it can often run hundred of checks on the tenant per worker, to ensure it is getting the correct results for your security configuration.

A new step has been added to a Business Process in the Workday tenant. What do I need to do in Smart?

If a new step has appeared in a business process, the first step is to check the blank template in Smart.

In Smart, we are constantly updating our templates to allow support for new sub bps. If a new step in your business process is not already in the template, you can check the current blank template and, if the tab is available, simply copy the tab into the existing data template and update it with appropriate data.

In the case that the blank template does not contain a tab for the required sub bp, users can request support for a new step in a business process by logging a KIM ticket at

An error has occurred in my test. Where can I find out more information about this?

When a test experiences an error, either a non-critical 'Warning' or a fatal 'Error', there are a number of places to view more detailed information on the error.

The quickest way to view more information is by clicking on the status for the warning or error, or by clicking the screenshot taken from Workday:

Alternatively, users can view granular information on the entire test by using the 'Debug View' tab. This will show each individual step within each part of the business process, as well as the errors encountered, alongside the screenshots. The Debug View will highlight all warnings and errors encountered, with detailed information as to why they occurred.

In some instances users will notice that no screenshot has been presented alongside an error. Usually this indicates that there is an issue with the Workday tenant, as Smart is having trouble connecting to it to take the screenshot.

In addition, no screen shots will be provided for Field Permission tests as the Smart end user may not have permission to view the information which is being displayed in the screenshot. 

Why do my values need to be case-sensitive?

In order to reference items in Workday, Smart needs to find an exact match. So, if a value has changed since the last run (e.g. a Cost Center name), the test run will fail. An easy way to get around this is to reference values using their unique Workday ID or Reference ID, which are values that do not change on the tenant. This will ensure that tests are re-usable and there is minimal maintenance required. For more information on this please see the section on using Integration IDs in test templates.

Why are my tests failing with an error screenshot showing the Workday login page?

Depending on the template parameters, Smart will use an integration system user account  to proxy in on behalf of workers to run all tests. When this error message is displayed it means that the account is not able to access the tenant. This can be for a number of reasons, such as the account being locked out due to the password being incorrect, or the account being disabled or expired. In the event that this happens, the relevant steps will need to be taken to restore the functionality of the account:

  • Account is not locked/expired, but unable to access the Workday tenant
    The account's password will need changed in the Workday tenant, and then updated in the tenant configuration screen for the relevant tenant in Smart.
  • Account is locked/expired
    Providing the password has not been changed, unlocking or unexpiring the account will restore its access to Smart and allow tests to run correctly.