Skip to main content

Run recurring data jobs

Outcome. A data management recurring data job moves files into or out of Finance and Operations on a schedule. Only the client applications you allow can use it, and your team hears when it fails, backs up or goes silent. Who takes part: Data management consultant, Integration administrator, Integration developer, Integration operator. Stages that use standard pages (Microsoft Entra ID applications, the Data management workspace, the standard Business events catalog) need System administrator or the access named on the role page; the Lattice roles do not open them. How often: once per job to set it up, then every day while it runs.

1. Build the data project and its mapping (Data management consultant)​

A recurring data job runs a data project that must already exist; the Lattice entities do not create one. Build the project in the Data management workspace, settle its field mapping (a delimited file from a sender with its own columns is read by position), and check it before any job points at it.

Done when: validateMappings with the project's name as the pattern returns Success true.

2. Register the client application (System administrator)​

The client application that submits or collects the files signs in like any other integration. It needs its own Microsoft Entra application, registered in Finance and Operations and mapped to a dedicated user.

Done when: the application is listed on System administration > Setup > Microsoft Entra ID applications with its own user, and you have its client ID for the next stage.

3. Create, authorize and activate the job (Data management consultant, Integration administrator, Integration developer)​

Create the job by hand in data management or through Lattice, let the client application submit to it, and activate it: activation is refused, with the reason, until the job has a schedule and an enabled client application. Through Lattice these steps need the Integration administrator role (activating through the Activate action or by setting State alike), and the client application's developer then sends or collects the files.

Done when: the job is on Recurring integration health with Job state Active, it is not Broken, and, for an import job, the first file the client application submits appears on the job's Messages.

4. Keep the job in source control (Data management consultant)​

Once the job works, capture the project and the job as a file. You can then apply the same integration to the next environment and check later whether this one has drifted.

Done when: the file is committed, and diffSpec with it returns Differences 0 against this environment.

5. Set thresholds and alerts (Integration administrator, System administrator)​

Tune the health thresholds to the job's rhythm, for example a silence period for a job that must receive a file every hour. Then switch alerts on and have a System administrator activate the alert event, so a change in health reaches your team without anyone having to look. The Integration operator can set the same values through the LatticeRecurringJobHealths entity.

Done when: the job's Health thresholds show Alerts enabled set to Yes, and Recurring data job health changed is activated in the standard Business events catalog, for all legal entities or for the company the job's batch job runs in.

6. Watch health every day (Integration operator)​

Lattice evaluates every job every 15 minutes. The three recurring job tiles on the Integration operations workspace open the jobs that need attention. A reviewer with the Integration read-only role can check the same health without changing anything.

Done when: Recurring integration health with Needs attention only selected lists no job, or only jobs someone is already working on.

7. Fix rejected rows at the source (Integration operator, Data management consultant)​

When a message fails, or is processed with failed rows, find the rows and the reason. Then have the team that owns the sending system fix the data, or have the consultant fix the mapping. Lattice never resubmits, skips or deletes a message, so the client application sends the corrected file again.

Done when: the corrected file shows as processed on the job's Messages with no row errors. The job stays Failing until its failed messages are older than the failure window (24 hours unless you set another).

8. Repair mapping drift and broken jobs (Data management consultant, Integration administrator)​

An active job shows as Broken when its data project has mapping errors (typically after an update added or removed entity fields), a batch job stopped, or no client application is enabled; mapping errors count only while a job is active, so validate an inactive job's project before you activate it again. The consultant validates the project, and the mapping refresh, Repair and the client application fix need the Integration administrator, so an operator hands the job over.

Done when: validateMappings returns Success true for the project and, after Refresh health, the job is Active and no longer Broken.

If something goes wrong​