Introduction

UAT or User Acceptance testing is widely used when introducing new software systems in large and complex ERP (Enterprise Resource Planning) projects and in CRM (Customer Relationship Management) projects.

This post reflects on why in my view, particularly in CRM of all IT projects, this can be a crucial if not a critical factor in a projects success and in the subsequent roll-out and engagement of Users. Many authors over the years have noted a high failure rate in CRM projects and again, my view is that inadequate UAT amongst other factors, will often be a root cause, but why is this?

What is it?

First let’s just clarify what UAT is.

User Acceptance Testing is usually towards the end of a project and will pre-date User roll-out and full training delivery depending on your project methodology, for example ‘waterfall’ or agile.

As a general rule, after any configuration or customisation changes to your system and internal supplier testing, time is then allocated to a select number of customer users to test what has been changed in context and ensure that this works or behaves in the manner which is acceptable and meets stated and documented requirements.

Usually scripts or usage cases are already prepared by the client or consultant to test the key usage scenarios effected by the change.

Any deviations are then fed back in some form of log, often called a Findings and Issues list showing items needing to be addressed/discussed by the Project sponsors. These findings may be rated by priority or impact and urgency.

How these are addressed will then be decided. But once these changes are made, these will then be tested again in another round of UAT until passed and accepted.

A simple enough process in theory and practice!

Why do it?

The core reason for UAT is to ensure prior to the new system or changes being handed over, there is a sign-off of the system to ensure it meets client specifications as shown and agreed in any Functional System Design documentation or Blueprint specification. Think of this UAT as the final check-list prior used and walked through when agreeing to accept a new car. In much the same way, often there will be a milestone payment associated with sign-off to focus all minds attention to the detail!

The focus on UAT is on how the system behaves. However, often during this process, a subset of real data may be added as Test Data prior to final migration and this may be tested too, but, I will touch on this aspect again later.

My experience shows that always there will be a Findings and Issues log that comes back from the Customer after their UAT. Often these findings may be minor and easy to fix, but sometimes major changes may be identified. These major changes are of big concern and a plan needs to be agreed and the impact understood in terms of the project plan.

Dangers and Risks

In many projects under tight time deadlines, the risk is this Testing and UAT period can be rushed or too short or that not enough trained personnel or not enough End Users are given the necessary time and tools to test and report thoroughly. How the feedback is documented can also cause an issue.

However, with good Project Management, these Risks should be clearly identified and early on in a RAID Log. This is simply a project planning tool, usually in Excel for identifying key Risks, Assumptions, Issues, and Dependencies. At start of any project, the Project Manager should identify any events, activities, resources that could impact on the successful completion of the project. This should be a weekly or monthly Update to project sponsors as the project evolves.

What to do what not to do

Any good testing will have a Finding and Issues log from UAT and don’t forget you may have more than one round of UAT. Some suggestions now on what to do and what not to do:

To do

• Selected Users need to undergo some form of system training before they are involved in UAT, so they must understand the system as if they were new Users, not come to this fresh with only limited exposure.

• UAT must be built-in with enough time for testing and a fixing allowance of time before Go-Live. Having it too close to this date runs risk of having to move it out the go live date which will not be welcome and impacts the business and the projects reputation.

• Have good script or usage scenarios available, easy to read and answer. Again, this requires adequate preparation time, don’t leave this to the last minute

Perhaps the most important point, but also one of the hardest to meet is:

• Allow adequate time for fixes. Most of the time, if there is a good cleat blueprint, good project management and enough resource allocation then many issues tended to be minor such as screen form design issues where users may want certain fields moved around or drop downs added too.

Also, it can be useful to split your UAT Testing into two work streams: User UAT and a separate Data Migration UAT team. They may be the same team but ensure different time is allocated or a different room, for instance swapping over. Trying to combine the two may result in missed testing since Data Migration UAT needs careful diligence and is not exciting, so allocate separate time to this. The quality of data can impact the project and I’ve covered this in other posts in the past.

What not to do!

• Don’t rush this process

• Don’t do a last minute allocation of resources with no adequate testing time in their diary

• Don’t forget these User need some Training first

• Try not to have your Testing by different people spread out. Ideally aim to have a minimum time in a “locked room” scenario. So there are no email or other distractions to enable people to focus 100% on their testing. Reflect on how best to achieve this in your project.

• Try not to do fixing “on the fly” before all your first round of testing is completed by all Users. This can just create complications…”Finish the testing, log it, fix it” is a good mantra.

• Don’t skimp on the Findings log detail. Ensure you are providing a good log and detailed description for everyone to be clear on the issue and for other to be able to replicate your findings. This will save time tooing and froing on clarification.

This is a short list, but no doubt experienced practitioners can add to this immensely, but for now let’s focus on the big picture still.

Resource Allocation

Choosing your Project Team and your internal Testing Team should involve a mix of Users and ideally at all levels and all impacted departments. If you can, you want champion End Users and sometimes, whilst it may be tough for ourselves, those Users who are the most vocal and even resistant to change can be your most effective champions if they can be turned around. In my experience oftentimes, they will also appreciate the fact they have been asked to contribute in the first place. More importantly, you want them onside.

As mentioned above, for data migration testing you want people who are very detailed. Data migration issues can really deflate a new project rollout, so attention to detail when reviewing is a vital skill. Hence I think this should be considered a separate work-stream.

A final note here, which is if you are struggling for resources, don’t be afraid to ask others including your consultants for help.

How to recover?

But what happens if for whatever reason UAT has not identified all the key issues. You then either go to User training and find out problems during the training, not the nicest experience for a Trainer or later only after Go Live.

This can still happen. Often the root cause is traceable down in my view down to three core reasons:

• Usage scripts not having right information or enough scenarios or being well written and easy to follow, or no scripts at all!

• Not enough adequate Testing time. Rushing the test time or just not allocating enough time or people not being disciplined to use this time. Again, having a whole dedicated day and ‘locked room’ will help here.

• Right mix or level of Tester. Possibly the Testers were not familiar with all the processes they were testing, it can happen, especially if time or resource is tight.

To recover the answer is pretty simple. You many need lots of bodies and take action early and fast to get back to a recovery position. I’ve said it many times in post before, but I believe the first 90 days after roll-out are critical.

But you need to see the signs, so if your users begin to say things like:

“It takes longer”

and more importantly can prove it, take this as a good sign! It is feedback at least and concern for the business, or:

“I can’t easily find what I need “

This last is often easier to fix, where often it is a question of improving the screen design and layout.

As in all things it, prevention is easier than cure

But act fast and recognise you have a problem and don’t blame the Users.

They are simply trying to do their job. Your job is to listen and validate first, not moan at the Users, whatever they say there will be kernel of truth there, the key is in then finding how big the impact actually is. Ignoring these warning signs will lead to a system with poor User adoption or worse a reluctant adoption and can lead to lost data, wasted time and eventually even employee de-motivation and churn.

Benefits of getting it right

Rather than me, I think, it is worth noting this comment from a leading CRM Project Manager with 20 years’ experience.

“Good UAT and the resultant follow-up and fixing is one of the key ingredients to successful adoption. Avoid ‘rushing’ it, but remember this also needs a strong Project Management focus on both sides here…”

Let’s not forget, this process is all to do with User Adoption and User Engagement. And the key to any CRM new project is this User Engagement in my view. This means that not only do your Users embrace the new system and can use it well, but they develop a strong a sense of ownership. Any ideas that come up then will be for positive improvements rather than just negatives.

Failure in that first 30 - 90 days can kill a system and is probably single biggest source of system change and failure in CRM projects.

And finally

So, don’t underestimate power and importance of good UAT.

This post is really all about appreciating just how important this stage can be to your User Adoption and Engagement success.

Don’t rush it!

Ensure as a client you do give this process the necessary resources, ‘time to test’ and allow for ‘time to fix’.

In essence, this all comes back to good project planning and choosing a Partner with the skills and project experience to deliver your CRM Project on time and on-budget.