As we come to the end of another CRM project, in this instance by far our biggest challenge has been the data migration and importing. This project has had more issues than usual, but the difference here was that I was very closely involved with the whole data migration project and so have re-learnt some invaluable lessons!

What are the lessons learnt from these last few weeks, when I was back in the data migration trenches so to speak!!! Here are some invaluable lessons, worth repeating!

Agree ALL data sources in advance

Never accept or confirm your quotation until you have all the data available and have seen this. Anything that is added after an agreed list of Sources should be costed separately, this protects both you and the client. As an example in this project, we have 3 separate core sources, but in each source, were myriad individual spreadsheets

Lesson learnt: Data migration costing is Always “subject to sight of data”, now we are adding the caveats (up to 2k records in one source and “ALL data Must be seen in advance.” Data sent after the cut-off date will be treated as NEW and costed accordingly!

Insist on common format

Again, whilst in this instance a common format template was issued, individuals then decided to re-label columns and add new columns to their own data, so almost creating their own individual data format!. So we had to check and ensure every excel column matched before import, despite a common template being issued to prevent this!

Lesson learnt here is Client need to be aware of time this can take and again this is chargeable.

Press for unique ID if possible.

If not create one as invariably you may be asked to add more data at some point and this can be time well spent.

Lesson Learnt: This can save a lot of time in re-work, there shouldn’t be any of course, but in the real world….

Ensure Client reviews and looks at their own data first.

Much time was spent in trying to put data into some sort of shape. So, for instance, in telephone field there were email addresses, in postcode there were Towns and in towns Zip codes. This was due in part to various source exports.

Lesson learnt. This is for the client, since they will better understand issues and avoid paying for re-work which they could have done before submitting.

Ensure a good import utility tool

The lesson here is to cost this in initially, invariably data once starting to be looked at gets more complex especially in a new system where data sources are often excel with multiple contacts. Most CRM system struggle with this, especially as it is often the case there are no matching ID or unique fields created, so deduplication is needed.

Lesson learnt. Include this as an allowance always since at some point, you may need as you see the data arrive and requirements grow from simple to the ‘is it possible or can you just add this?”

Within the ebook download request, we include more Lessons together with an additional list of Data Import ‘Gotchas’ that should be reviewed as well.

A final lesson is:-

Allow for adequate Client Testing time

The client importantly must have their project team or designated testers available for a ‘first pass’ testing for EACH data source and these do need to be signed off. Yes, this sound very bureaucratic, but time and again, clients give data imports only a cursory look, glad to see their data moving across from the old system, but then don’t look at the detail. For the consultant importing, this is just ‘data’ and they don’t always see the minute differences or nuances that can be in there in placing information in slightly differently. Invariably it is a new look and format for the new CRM system, so time must be paid and checklists created!

Lesson Learnt: UAT team need to have a conference call and test and review data closely.

Conclusion

When speaking with clients, the term ‘best endeavours’ is really the most apt, since we are so reliant on the quality of data and more importantly data consistency. But, here Is the good result, as discussed with this client, they now have a single centralised database in their Customer Relationship Management software. This now means that the new system data is consistent, accurate and de-duplicated data, thus preventing the myriad ‘silos’ of information building up again.

Yes, Data Migration is time consuming and can be expensive and this is do due to the complexity of source data and mapping data across to new system. IT project suffer from this, and it is one reason why the Big Banks and Government have so many big failures and delays, since Data Migration can be a nightmare and you can lose sleep, so take these lessons to heart!

More lessons and some important data gotcha’s are contained in the downloadable ebook Data Nightmare Lessons in CRM.