WD2024R1: 12 Steps to Successful Workday Update Testing
As Workday's next major update window grows ever closer, our senior Workday Test management team have put together this handy guide to help you navigate the challenging but rewarding landscape of testing your configuration in the run up to WD2024R1.
From data analysis and building a testing team to regression testing and beyond, our in-depth guide will help your organisation to breeze through the next major update to focus on the exciting new features and functionallity that Workday has to offer.
1. Understand why Workday update testing matters
Because Workday is a workflow-based solution and not a transactional one, processes often intersect (as do security groups/settings). This means sometimes a change made in one area can have a knock-on effect elsewhere in your configuration. While Workday thoroughly tests its software before it releases it, every customer has a unique configuration and surprises can sometimes arise.
In the context of change during the update window, your objective is to test your configuration as thoroughly as you can so that on the first day that these changes appear in production, you feel confident that your configuration will continue to perform as expected for your end-users.
2. Plan thoroughly
In the four-to-six weeks before the preview period begins, your team should be developing its update test strategy and test plans, working out exactly what you need to test, how (the strategy), and the logistics (the plan) around testing so that when the preview window arrives, you can hit the ground running.
3. Develop your update test strategy
Your test strategy defines what you plan to test and why. When deciding what to test, we recommend a risk-based approach that prioritises:
1. testing of your high-risk or high impact business-as-usual functionality. Ask yourself what processes would pose significant issues if they suddenly didn't work as expected. Often these include highly customised and complex BPs and security configurations, as well as those integrations that are tied to financial risk like third-party payroll;
2. testing of updates and enhancements that are likely to affect the functionality you currently have switched on; and
3. brand new Workday features or functionality that are switched on automatically prioritising those based on their impact to your business and configuration.
Download this eBook
4. Build your test plan
A good test plan maps out in detail the “what” (scope), “when” (schedule), “where” (tenants) and “who” (staff). It documents the following and should be agreed by your key stakeholders:
- the risks you hope to mitigate by testing (this will come from your test strategy);
- what, specifically, you will test to mitigate those risks. (For example, you might have identified in your strategy that the benefits eligibility during the hiring is a high-risk area that must work when the update goes live. But what specific processes will you test to mitigate that risk? Create position? Hire? Termination? What about Change Personal Information? Or Change Job?);
- the specific scenarios and number of individual tests you will execute in each area of focus to get the depth of coverage you need;
- which staff you will need for the various tests; and
- your test schedule, specifying which tests are taking place on which dates in which order by which staff.
It's rare to complete a test cycle with zero test failures. So remember to include time to run the initial tests PLUS additional time to investigate issues, log and resolve these, and to retest those fixes. Build this contingency in to every test cycle in your schedule.
5. Write your tests and prep your data
For every test, you’ll need to document some rudimentary details about the test such as:
- all prerequisites required to start the test, for example an “Open position”;
- what the test is (it’s objective, for example, “Hire a new Full-Time Employee into an Open Position”;
- what test data is required (for example, a job profile relevant to the open position;
- who to initiate the test on behalf of (what initiating role/security is required);
- what the other parties in the test are (for example other workers or objects such as supervisory organisations);
- a clear list of steps on how to run the test (this maybe using a script approach or, if you are taking a tenant-to-tenant comparison approach, it may be the general execution process required for each test subject to this approach;
- the expected result(s) of the test.
For example, a test of what security permissions team members have on each other might specify the initiator as Logan McNeil, the other party as Alain Dubois and an expected outcome that Logan cannot change Alain's Benefits. If you are reusing previous tests, make sure the step-by-step instructions from the past are in line with your current configuration and UI. Otherwise, when test execution begins, testers will get confused and valuable time will be lost updating the scripts.
Data prep is one of the most overlooked elements of planning. Don't underestimate the time it takes to identify workers that fulfil your test criteria. It is substantial. Begin this process well before the preview window opens. If you're reusing old test scripts that contain real worker data, review that data carefully. Data on real workers changes constantly. They change jobs and supervisory orgs, go on leave or leave the company, their salaries change, their eligibility for benefits change, etc.
Without a careful review to confirm that the test data used in previous regression packs is still suitable, tests during the update may fail not because there is a genuine configuration issue, but because the worker data used in the tests were no longer fit for purpose for the given scenarios.
6. Rally the troops
Once your test plan activities are mapped out, you'll have a clear view of how testing will impact other areas of your organisation. The testing required throughout the five-week preview window is typically a multi-departmental effort, so it’s essential to make sure everyone involved knows exactly:
- why this testing is so important for safeguarding your Workday configuration;
- what their role and responsibilities in the process are;
- what the testing, reporting, and documentation processes are and key deadlines they must meet; and
- what the lines of communication are.
7. Test your Sandbox and Sandbox Preview Tenants simultaneously
Our clients have experienced the greatest success when they thoroughly test both their sandbox and sandbox preview tenants side by side throughout the five-week period. We highly recommend that you ensure both tenants are refreshed from production at the same time.
Workday arranges this by default for Week 1 of the preview window. This gives you a simple A/B comparison that allows you to clearly spot which processes have been impacted by the Workday update and need attention. This approach is advantageous in that it increases your pool of test executors and reduces the level of detail required to document your tests. It can be done as a fully manual testing approach or with test automation assistance (and, of course, a mixture of both).
This is also when the efficiencies of automation are most pronounced. With Kainos Smart, you can execute a set of tests on one tenant and repeat the exact same set of tests on your other tenant with a click of a button. The resulting comparison is reported so you can quickly see any differences between the two tenants and take remedial action.
8. Request refreshes of your Preview Tenant
Workday releases software updates to your Sandbox Preview tenant every week during the five-week preview period, not just on day one, so it s a good idea to plan further Preview data refreshes from Production (Workday will arrange this on request after Week 1) so that Sandbox and Sandbox Preview are in sync during the update window.
Workday will however arrange the first refresh of Sandbox Preview at the start of the update window; Sandbox will be refreshed as per the usual weekly refresh from Production and therefore the correct starting point required for the simultaneous Sandbox to Sandbox Preview test approach will be established for you.
If you plan to divide your test schedule with a test cycle approach, schedule a refresh for the beginning of each cycle. We highly recommend that you request a refresh of your production data and configuration to your Sandbox Preview tenant in Week 5 and at least cover some portion of a test cycle with it. This gives you a final chance to test and confirm that your most up-to-date configuration and data are performing as expected on the new release and take action to prevent any issues from appearing in production where end-users could be impacted.
9. Start with Security, Reporting and Integrations testing in Week 1
Assuming a Sandbox to Sandbox Preview comparison approach, always start with Security, Reporting and Integrations testing (given that these functionalities are often business critical and are more sensitive to data changes within the tenants than Business Processes); we recommend that you consider a tenant “freeze” for both tenants whereby the freeze will limit the risk of data or configuration contamination (and thus ensure confidence in the comparison of results between tenants, i.e., if the results match in both tenants, you can be confident that all is well in this tested area).
This can be implemented, on one hand, as a simple instruction to all tenant users to refrain from accessing or running events within the tenant or, on the other hand, perhaps de-activating the accounts of those not involved in testing (to effectively enforce the freeze).
As recommended, for Security, Reporting and Integration testing, the best approach is to run your tests on both your Sandbox and Sandbox preview tenants and to do so before any other changes have been made to the tenants. This will allow you to do an exact comparison of your actual results running on the current version of Workday versus the preview version of Workday and immediately pinpoint any discrepancies (which are then likely to be caused by regression of your configuration due to Workday’s changes as opposed to data contamination causes).
Starting these tests each Monday morning will limit the risk of data and configuration compromise as a result of staff making changes following tenant refreshes. If you have a significant number of security groups, custom reports and integrations, we recommend using test automation.
10. Add BP testing and regression testing in Weeks 2-5
As you'd expect with any major software update, Workday releases fixes throughout every week of the preview window. So, it's best to conserve your efforts and hold off on BP testing until Week 2 of the window when their first batch of fixes is released. Also, BP testing will create and change data so, to avoid contamination of your Integration and Report testing, you’ll want to hold off on BP testing until that is complete (again, assuming the use of a Sandbox to Sandbox Preview comparison approach).
Remember to continue running your Security, Reporting and Integration tests alongside your BPs throughout the five-week window as each weekly release from Workday will include new functionality. This is more difficult to do on a manual testing basis so if you don’t have access to test automation then consider breaking your testing up in to two test cycles where the 2nd cycle is a mirror of the first and executed during weeks 4-5.
11. Track and document
Take advantage of the tracking and documentation tools out there. Manual test teams often lack traceability around tests and their results. This leads to uncertainty and risk in the test process, as there's no reliable and consistent view on progress or potential blockers. Without a means of tracking progress and incidents (be they change requests or defects), you miss out on valuable insights ahead of the next update window. And when auditors arrive, there's no reliable record of the testing you 've completed.
12. Introduce automation
Test automation can be a key differentiator when it comes to update window testing. The ability to run comprehensive packs with automatically set results at the click of a button helps you reduce testing effort to mere hours instead of days and weeks, improves accuracy, and frees up your SMEs to spend less time on testing and more time exploring and adopting Workday's useful new features.
Our Smart Test automated testing platform can reduce hands-on effort during Workday update testing by 90%, increase test coverage by over 400%, and catch 200% more defects than manual testing.