Seamless Data Migration: Navigating the Transition from Old to New B2B Systems

As a professional B2B product manager, you will inevitably encounter scenarios where you need to replace an old system with a new one or undergo a system refactor and upgrade. Data migration is a critical step in the system transition process, and its success directly impacts the stable operation of the subsequent system. Different transition scenarios present various data migration challenges, but having a comprehensive data migration strategy can guide a smooth transition process.

I. Data Migration Scenarios

In the course of our business operations, we often encounter situations where the original system cannot meet business requirements, necessitating the development of a new system to replace the old one. However, historical data from the old system or ongoing business data must be reflected in the new system. Before the new system goes live, data migration from the old system to the new one is necessary.

II. System Transition Methods

The method of transitioning the system affects the choice of data migration approach. When transitioning from old to new systems, several common operation methods are:

• After determining the go-live time of the new system, both the old and new systems run concurrently. New orders use the new system, while orders in progress continue to use the old system. Once all in-progress orders from the old system are completed, historical data is migrated to the new system in a single batch.
• After confirming the go-live time of the new system, the old system is deactivated, and all data and functionalities are switched to the new system.
• In cases where the new system undergoes significant adjustments and major changes in business processes, it may involve selecting specific years or months as settlement intervals for the go-live transition. After the transition, business data is not migrated, and historical data remains in the old system for reference.

III. Data Migration Content

1. Data

• Basic Data: Supports primary data for various modules and businesses, such as clients, assets, projects, products, vehicles, tool information, etc.
• Dictionary Data: Globally used dictionary data within the system.
• User Data: User accounts, organizations, roles, permissions, and more.
• Business Data: Data generated during the course of business operations, including order data, transaction data, billing data, operation data, and data pertaining to various stages of the business process.
• Historical Data: Data related to completed settlements and other milestones in the business.
• In-Progress Data: During system upgrades or transitions, normal business operations continue, resulting in numerous in-progress data items, such as incomplete order data and ongoing waybill data. Handling in-progress data requires special attention.

2. Data Relationships

Relational data is linked through IDs or primary keys, and during migration, the relationships between data items need to be reassigned.

3. Files

Certain attachments, images, and other data are directly associated with business data and need to be reorganized and linked. Some files stored separately can be directly transferred to the new server, along with the corresponding URLs.

IV. Migration Approaches

Several factors influence the choice of migration approach, including data volume, transfer time, and whether the business can be temporarily paused.

1. Offline Migration:

• Excel Import-Export: Data is imported into the new system by exporting and importing in Excel format. The exported data can be adjusted to match the format of the new system before importing.
• Database Synchronization: Through SQL or database synchronization tools, fields in the new and old systems are matched, and data is batch-imported into the new system.

2. Online Migration:

• Interface Transmission: By defining and writing interfaces, data synchronization between the old and new systems is achieved. In scenarios where both systems need to run concurrently, interface transmission enables bidirectional data synchronization between the old and new systems. During enterprise IT projects, there may be scenarios where external procurement transitions to in-house development. In such cases, it is advisable to perform a one-time system switch and upgrade to avoid parallel system operation.
• Database Synchronization: In projects with large data migration and longer duration, database migration tools can be employed. Data can be fully migrated to the new system, followed by daily automatic incremental migration or manual incremental migration. Before the system switch, a final incremental backup must be completed.

V. Data Validation

Data validation is essential during and after the migration process to ensure data accuracy, completeness, and consistency. This involves verifying data quantity, content, and ensuring that no incremental data is generated during the migration process. Data migration typically requires multiple rehearsals and validations to ensure a successful transition when the formal system switch occurs.

Validation requires detailed test cases, similar to testing cases:

• Low-frequency changes in basic data and data dictionary can be synchronized in advance as part of the pre-go-live environment preparation, allowing them to be transferred to the new system early and tested.
• For historical business data that has been completed, validation can be performed through queries, exports, business intelligence tools, and more to confirm data correctness and completeness.
• For ongoing business in-progress data, additional verification of business process flows is necessary. Based on the stage of the data in the process, forward and reverse business operations are conducted to ensure the smooth completion of in-progress data.

VI. At The End

In the realm of B2B systems, data migration is a pivotal process during system transitions. This article provides insights into the significance of a well-structured data migration strategy, addressing various scenarios and approaches. Whether opting for offline or online migration, factors like data volume and business continuity play a key role. Furthermore, data validation emerges as a crucial checkpoint to ensure the accuracy and integrity of information. By thoroughly understanding the intricacies of data migration, professionals can navigate the shift from old to new systems with confidence, maintaining the stability and reliability of their business operations.