Loading and updating data from MDM to Cloudant
The MDM and Cloudant® integration sample provides two InfoSphere® DataStage® jobs that support initial bulk data loads and continuous updates to ensure data synchronization between MDM and Cloudant.
The party_bulk_load job accomplishes the initial bulk data load from MDM to Cloudant. The job uses the DB2® Connector stage to drive iteration over a set of business entity primary keys and then uses the MDM REST API to construct JSON documents for each entity. The job then stores the JSON documents in Cloudant.
After the bulk load, the party_trickle_load job supports subsequent continuous updates (insert and update). By using the JMS Listener stage, the job triggers on MDM JMS notifications. When updates are detected, the MDM REST API reconstructs the JSON documents and refreshes the contents that are stored in Cloudant.
This illustration shows the basic flow of the jobs.
