Skip to main content

Migrate data in bulk

Outcome. Data from a legacy system lands in Finance and Operations through data management. Journals arrive as ready-to-post journals and are posted, and you can show that what arrived matches the source. Who takes part: Data management consultant, Integration administrator, Integration developer (only when your own system posts the journals). Stages that use standard pages (the Data management workspace, the journal pages) need your standard Finance and Operations access to them; the Lattice roles do not open them. How often: once per migration, and once per rehearsal of it.

1. Measure coverage (Integration administrator, Data management consultant)​

Start by learning which tables hold data and which entity can reach each one, in the legal entity you are migrating. The administrator runs the coverage analysis with row counts; the consultant reads the snapshot and picks one entity per table.

Done when: Table data and entity coverage shows a snapshot taken after the last deployment, with row counts, and every table in scope either has a chosen entity or is on your list of Data, no entity gaps.

2. Agree the file layouts with the source team (Data management consultant)​

Give the team that extracts from the legacy system the exact shape of each entity: its key, its mandatory fields and the fields an update may change. A template per entity is the hand-off.

Done when: the source team has an Excel template or an XSD schema for every entity in scope, and a sample file for each one matches it.

3. Prepare the target environment (Data management consultant, Integration administrator)​

Before the first load, make sure data management knows the current entities, and build the data projects with their mappings. Set up what the files refer to: journal names with their number sequences, items, accounts and dimension values, and the legacy system's keys if you bring them across. Registering the legacy system and loading its keys need the Integration administrator.

Done when: validateMappings with your migration projects' name pattern returns Success true.

4. Load the data (Data management consultant)​

Load what the journals refer to before the journals themselves, because the import refuses a line that names an item, account or dimension value that does not exist. Load journals through the set-based entities, which write large files as open journals; each distinct grouping key in the file becomes one journal.

Done when: each execution succeeded, or partially succeeded with its refused rows listed in the staging data, and the new journals are listed on the journal page of their type.

5. Resend the refused rows (Data management consultant)​

Each refused row carries a message. Fix those rows at the source and resend only them: rows that were written are never written twice, so a resend is safe.

Done when: every refused row has been sent again and its new staging row is Completed.

6. Post the journals (Data management consultant, Integration developer)​

The import creates journals open, not posted. Post them on their journal page or with the standard batch posting. Or have your own system post them by journal number through the Lattice journal posting service, many journals per call or a very large ledger journal in the background.

Done when: each journal shows as posted on its journal page (with the filter set to show posted journals), or its posting call returned Success true with a PostedDocumentId.

7. Reconcile (Data management consultant, Integration administrator)​

Read back what arrived and compare it with the source extract: balances and transactions through Lattice's read-only entities, and row counts from a new coverage snapshot, which the administrator takes.

Done when: the totals and row counts you read back match the source extract for every entity in scope, and each difference that remains is explained.

If something goes wrong​