Data Migration Best Practice and Guidelines

This blog covers the process of Salesforce data migration, highlighting its complexities and the importance of accuracy to avoid business disruptions. It outlines the steps involved—planning, preparation, and execution—and provides best practices and guidelines for a successful migration

Abstract

Salesforce data migration is the process of moving data from one system to Salesforce. It can be a complicated and time-consuming process, but it's important to get it right so we can avoid business disruptions.

The process of salesforce data migration can differ depending on the followings:

• The tools that perform data extraction, transformation, and loading.
• Size, format and accuracy of the source data.

Salesforce also provides data migration tool (Salesforce data loader) to

• Enable convenient extraction of data from Salesforce org, and
• Import data into salesforce system. It imports data only from .csv file format.

It doesn’t provide data transformation capability so while using salesforce data loader data transformation must be done manually.

Problem Statement

• Have starter-kit to migrate data to salesforce.
• Define best practices and guidelines.
• Avoid and Solve issues during data migration.

Background

• Data Migration to salesforce could be periodic activity. OR
• It could be part of green field implementation. OR
• It could be part of salesforce org merge.

Solution

Data migration to salesforce needs to be executed in three steps:

• Planning
• Preparation
• Execution

Planning: Data migration planning is a crucial step in successful data migration. In this phase, we define followings:

• Objective of data migration
• Scope of data migration
• Source system of data
• Source and target data model.
• Volume of data
• Tools and approaches for data extraction and migration.
• Approach to clean and transform data.
• Onboarding of resources as per tools (Data Loader, Jitterbit, Mulesoft, etc) to be used for data migration.

Preparation: Data migration preparation is the step where we prepare and finalize data to be migrated to the salesforce. Following are the activities which needs to be performed in this step:

• Define sequence of data objects to be migrated.
• Define data extraction criteria for each object.
• Create mapping files for each object.
• Creating a custom field in target org to store ID (Primary Key) on each object, this needs to be set as an external Id so that it can be used to upset records and importing related records with having correct parent IDs in it.
• Define cut off dates for full migration and delta migration.

Execution: Execution of data migration has three key activities:

• Data extraction:
• Data Transformation: Here data needs to be transformed as per data model of each object in target org. We need to pay special attention to fields of type date, datetime, number.
• Data migration: At the time of data migration please save mapping file used in appropriate folder with its meaningful name. Establish parent-child relationship between two objects using external if field and avoid doing this manually unless this cannot be done automatically. Keep data files open during migration otherwise transformation done may be lost.

Other than business objects there are technical objects like Roles, Profiles, Group, GroupMember, Permission Set, Permission Set Group and Permission Set Assignment which needs to be migrated before migration of master and transaction objects.

Following is the recommended sequence of data load for different entity related with data migration on sales cloud and service cloud:

Here please check if some of the metadata components like role and profiles have not been migrated as part of metadata deployment. If community users need to be migrated, then it must be done after account and contact migration.

Guidelines to be followed:

• Extract all fields from all objects.
• Maintain only one version of mapping file.
• Maintain three folders Extraction, Transformation and Migration for each of data entity to me migrated.
• Create external id field in target org on each of the object to store primary key of record from source system.
  • Establish parent-child relationship between two objects using this field and avoid doing this manually.
• Pay extra attention to followings:
  • Issues like special character,
  • Leading zero in data transformation file.
• Whenever you transform manually in .csv file
• Deactivate all automations (to be triggered after record creation / update) during data load and make a list of it to be referred to switch it on after data migration:
  • Validation Rule
  • Workflow Rule
  • Process Builder
  • Flow
  • Apex Trigger
  • Case Assignment Rule
  • Case Auto-Response Rule
  • Lead Assignment Rule
• When data is getting migrated from legacy system to salesforce audit fields (CreatedByID, CreatedDate, LastModifiedbyID, LastModifiedDate) can be field with value from source system.
• Make sure that all dates must be in YYYY-MM-DD format, and date/time values must be in YYYY-MM-DDThh:mm:ss.SSSZ format, where T and Z are literal values, SSS is milliseconds (000 is fine), and hh is 00 through 23.
• Turn off following while migrating data:
  • Sharing rule calculations: to make sure that data load doesn’t take long time.
  • Org wide Email deliverability to make sure that any actual email address doesn’t get triggered emails.
• Make sure picklist values (to be migrated) are active on target object.
• Do sample migration in dev sandbox and define extract, transform and migrate procedure for each object. Maintain copy of mapping files used.
• First do full migration on pre-production box (UAT, full copy sandbox). Update procedure and mapping files to be used while migrating data on production.
• Maintain only one copy of data mapping files.
• Verify count of data after load using SOQL OR Reports.
• Perform unit testing on random data to verify missing values, bad values, incorrect characters etc.
• Using bulk API in parallel mode gives data lock error and we need to re-process failed data.
• Sometime bulk API doesn’t update few of the fields hence please validate value in each of the field during sample data migration. It will help us to avoid issues during actual migration.

Conclusion

Data can be migrated to salesforce successfully if we are following best practices and guidelines during pre-migration, migration and post-migration phases. This guidelines and best practices can be used to migrate large numbers of data records as well.

References

This content has been written based on experience from previous data migration work.