Self-assessments are great for helping you evaluate your position in a given situation. Even though they’re most commonly associated with helping with your tax returns, you can apply the same premise to evaluate how well you test your enterprise SaaS system, such as SAP, Oracle, and Workday.
When testing of your SaaS ERP configuration and its integrations is weak or incomplete, important errors can go undetected—for example problems with processes, broken integrations, sudden data visibility, and unexpected changes to who can do what in the system. These can not only interrupt business but also put your company at risk for data breaches, compliance failures, and even fraud.
A simple self-assessment can help you evaluate how thorough your current testing is and if you’re doing enough to mitigate risks. As you review your situation, you should seek to understand:
- how much time and resource your test teams are spending on software due diligence, and what is the opportunity cost of this work;
- the quality, depth and coverage of your current testing methods; and
- the frequency of testing.
Here are three key questions to ask yourself or your system owners to help you on you way (along with answers that reflect best practice).
“What kind of tests do we run on our system?”
Ideally, these five types of testing should be taking place:
This testing confirms that all business processes execute in the software as expected, route through their steps correctly, accurately trigger approvals, and so on. It also checks that the software handles invalid inputs appropriately.
This confirms that data is being shared between connected software correctly. Is data the enterprise SaaS depends on from a separate system being called in correctly, and is the enterprise SaaS also pushing out its data to systems downstream? This can be quite complex and challenging for non-specialist teams; in large enterprises it’s common for a SaaS system to be integrated with dozens of other software systems.
This verifies that users only have access to the appropriate related actions, tasks, reports and data fields. This includes testing segregation of duties.
This tests a sequence of related individual processes to ensure the chain executes seamlessly.
Frequently overlooked (and at your peril), this testing verifies that everything that was working correctly before is still working correctly after a change in the software. Configurations changes made by your software administrators, software updates by the vendor, and new integrations are just some of the changes that can affect functionality.
“How often are we testing?”
These systems are not static. Your vendor doesn’t stand still and nor do you as a business. Your platform constantly evolves as your business changes. Most teams know to test the major alterations, for example for the introduction of new processes, rolling out functionality to new geographies, etc. These are the types of changes that require all five types of testing.
But it’s the small stuff that teams skip, and this is where the majority of errors and risks creep in. As a rule, no matter how seemingly small or minor the change, teams should regression test afterwards. If internal changes are infrequent, then they should still be regression testing at least once a week to minimise risk and catch issues early that might be caused by external changes, like software updates or changes in connected systems that live up and downstream from your enterprise SaaS.
“Have we explored an automated testing solution?”
While continuous testing is best practice, teams that test manually rarely have the manpower to carry these out to the level of quality, frequency or depth of coverage that is required to keep the system running smoothly and minimise risk. Automated testing of your enterprise SaaS systems can reduce these risks, cutting operational costs and strengthening security and compliance.
For example, it helped Magellan Heath’s HR team reduce costs considerably and reallocate staff back to higher value work. “After introducing Kainos Smart to automate our testing, we were able to reduce our testing costs by 70%,” says Lawrence Berra, Magellan’s Manager of HR Systems. “We also reduced the number of employees involved in hands-on testing by 80%, freeing up staff to concentrate on their other HR responsibilities.”
Its compliance-related advantages are also noteworthy. “Automation helps you from a compliance perspective because it removes some of the human element,” says Coreen Campbell, Director of HR Systems at International Rescue Committee in New York City. “Unlike manual testing, it’s objective. You don’t have one person marking a test as a pass and another marking that same test as a fail because they’ve interpreted things differently.”
So what’s your diagnosis? Do your answers and current approach stack up against best practice? If so, well done. However, if not, don’t worry. You can use the information above to take remedial action. Look at what you can do to shore up knowledge and skills in your testing team, invest in additional resources to get the coverage needed to carry out comprehensive testing and reduce your risk, or consider introducing automated testing to lift some of the burden off of test teams while simultaneously improving test quality and reducing risk.