Choosing a reader for persistence
As part of configuring batch persistence, choose an appropriate reader.
Two of the prebuilt readers included with the batch processor are particularly appropriate to persistence.
- Database reader
- Consider using the Database reader if your persistence job reads directly from the MDM database. The Database reader can read only from the MDM database using a simple SQL directive.
- Titled CSV file reader
- The Titled CSV file reader requires you to create a CSV file and save it where the batch instance can access it. To create the file, you might create the file by hand, generate it with a third-party tool, or extract data from a database. In the case of the Titled CSV file reader, you are not limited to reading only from the MDM database. You might see a performance advantage to using the Titled CSV file reader because the reader does not need to read directly from the database during persistence. You might also consider the Titled CSV file reader if you want to guarantee that errant SQL does not corrupt your data. The Database reader cannot account for errant SQL directives. For the Titled CSV file reader some job definition templates provided within $home/templates/jobs can help you to create batch jobs more quickly.
On balance, the Titled CSV file reader provides greater flexibility, but you might find that the Database reader is easier to use because it does not require you to create a CSV file as an intermediate step.
If neither reader meets your needs, consider creating a customized reader. A customize reader applies not only to the new type of batch jobs, but also applies to batch input file. This is known as pure batch and provides the most flexibility for providing the batch input.
To specify the default reader used by the batch processor, set the value for the ReaderQueue property within the Batch.properties file. For example, to specify the database reader as the default reader, set the ReaderQueue property to com.ibm.mdm.batchframework.bulkprocessing.queue.DatabaseReaderQueue. To specify the titled CSV file reader as the default reader, set the ReaderQueue property to com.ibm.mdm.batchframework.bulkprocessing.queue.TitledSingleLineCSVFileReaderQueue.