IBM Support

Understanding the JDBC Gateway database record update rate

Technote (FAQ)


Question

How can you tell the rate at which the gateway is transferring records or writing to the database, from the information in the debug log file?

Cause

The output in the gateway.log file can be detailed, making it difficult to know what to look for.

Answer

The gateway operates as follows:

On IDUC, the gateway converts IDUC data into batch data, writes the batch to disk.

It then updates the cache with the data contained in the batch, such as the alert history as well as the new batch id, before it writes the cache to disk.  

Once the above two tasks are complete, the IDUC data has been stored to disk and is recoverable in the event of a crash. The gateway then processes the batch and writes it to the database. The batch execution time is the time to process that batch into the database.

The cache write time is the amount of time it has taken the gateway to write the cache to the disk, so for example:

Thread-2: STATS: Cache write time 1241 ms

means the cache took 1.241 seconds to be written to the disk.

To find this information, grep for 'STATS' in the log file.


So for example, in the log file, we may have:


2014-10-03T16:31:36: Debug: D-IDC-105-011: Iduc Client: Received IDUC v2 control message [c:iduc2 00000030], Payload=30.

- The gateway starts creating new batches:

2014-10-03T16:31:38: Debug: D-GJA-000-000: [ngjava]: G_JDBC: Thread-2: New batch UUID: 52273367-aef0-41c8-9b58-bd885b4d4e8d

2014-10-03T16:31:45: Debug: D-GOB-105-148: [ngobjserv]: Mapper: Pre-Mapping insert/update iduc data from 'NCOMS', table 'alerts.status' of '12971' item(s). [Inserts/Updates=58 (unfiltered)]

- Completes processing batch uuid 52273367-aef0-41c8-9b58-bd885b4d4e8d:

2014-10-03T16:31:45: Information: I-GJA-000-000: [ngjava]: G_JDBC: pool-1-thread-1: STATS: 52273367-aef0-41c8-9b58-bd885b4d4e8d Batch execution time: 102 ms (1931.3725490196077 rows/second)
2014-10-03T16:31:45: Debug: D-GJA-000-000: [ngjava]: G_JDBC: pool-1-thread-1: Removing batch: 52273367-aef0-41c8-9b58-bd885b4d4e8d

The time from batch creation to its insertion in the database is 7 seconds, with 1931 rows written per second.

New batch created:

2014-10-03T16:31:45: Debug: D-GJA-000-000: [ngjava]: G_JDBC: Thread-2: New batch UUID: 1344a015-7f47-4d97-81e2-4683d032f58d

2014-10-03T16:31:47: Debug: D-GOB-105-289: [ngobjserv]: Mapper: Post-Mapping insert/update iduc data from 'NCOMS', table 'alerts.status' of '12971' item(s). [Inserts=38][Updates=12933]

Batch uuid 1344a015-7f47-4d97-81e2-4683d032f58d completed:
 

2014-10-03T16:31:49: Information: I-GJA-000-000: [ngjava]: G_JDBC: pool-1-thread-1: STATS: 1344a015-7f47-4d97-81e2-4683d032f58d Batch execution time: 46 ms (1782.608695652174 rows/second)
2014-10-03T16:31:49: Debug: D-GJA-000-000: [ngjava]: G_JDBC: pool-1-thread-1: Removing batch: 1344a015-7f47-4d97-81e2-4683d032f58d

Total time from creation to database insertion 4 seconds, with 1782 rows written per second.

New batch created:

2014-10-03T16:31:49: Debug: D-GJA-000-000: [ngjava]: G_JDBC: Thread-2: New batch UUID: e7f42011-8edc-4b87-8e5d-97e879e80430
2014-10-03T16:32:46: Information: I-GJA-000-000: [ngjava]: G_JDBC: pool-1-thread-1: STATS: e7f42011-8edc-4b87-8e5d-97e879e80430 Batch execution time: 14601 ms (911.2389562358743 rows/second)
2014-10-03T16:32:48: Debug: D-GJA-000-000: [ngjava]: G_JDBC: pool-1-thread-1: Removing batch: e7f42011-8edc-4b87-8e5d-97e879e80430


Total time from creation to database insertion 57 seconds, with 1782 rows written per second.

From this also, you can calculate roughly how many rows there are in a batch by multiplying the rows per second with the time taken for the execution.

Document information

More support for: Tivoli Netcool/OMNIbus

Software version: 7.2.1, 7.3.0, 7.3.1, 7.4.0, 8.1.0

Operating system(s): AIX, HP-UX, Linux, Solaris, Windows

Reference #: 1688453

Modified date: 26 October 2016