Designing process applications for Apache Kafka messages created by IBM InfoSphere Information Server

You can use Kafka messages to start your IBM® BPM process applications.

Before you begin

Make sure that Data Quality Exception sample process application is configured and that you can work with exceptions that are created by data rules or jobs in IBM Stewardship Center.

About this task

InfoSphere® Information Server has a ready-to-use installation of Kafka and a Kafka topic that provides all InfoSphere Information Server events as Kafka messages. For details, see Managing events by using Apache Kafka. When you use Kafka messages to start your IBM BPM process applications, you do not need to create callbacks and subscriptions. The Data Quality Exception sample process application demonstrates how to directly listen to the Kafka messages. You can start the Kafka listener in the sample to see how IBM Stewardship Center creates tasks for your exceptions based on Kafka messages.
Note: The sample reads messages from Kafka by using a Kafka Java Client. You can access the source code of the client to build your own version of the Kafka client by using the BPMKafkaIntegration sample.

Procedure

  1. Configure event processing by using Apache Kafka by following instructions in the Managing events by using Apache Kafka topic. Save the Zookeeper connection string for further reference. The connection string is hostname:port, where port typically has value 52181.
  2. Copy all .jar files from the <IS_install_path>/ASBServer/apps/lib/iis/shared-open-source-clients directory to the <BPM_install_path>/v8.5/lib/ext directory.
  3. Restart the IBM BPM server.
  4. In IBM Process Designer, expand IIS Exception Remediation Sample.
  5. Click All, and then click ProcessKafkaMessagesUCA to open the Undercover Agent.
  6. In the Parameter Mapping section, enter the Zookeeper connection string.
  7. In the Scheduler section, click Run.

Results

The ProcessKafkaMessagesUCA Undercover Agent triggers the Kafka message processing. As a result, you can see new tasks for your exceptions next to the existing tasks in the BPM Process Portal.

What to do next

The Kafka message processing stops when all messages on the Kafka topic are processed. You need to rerun the ProcessKafkaMessagesUCA Undercover Agent to start the processing of new messages. If you want to automatically run the Kafka message processing, you can configure the Undercover Agent for a scheduled message event. For details, see Undercover agents.
Parent topic: Configuring and managing exceptions in Data Quality Exception Console