OMEGAMON XE For IMS Historical Performance Analysis Options And Best Practices
from CCR2, Issue 9 - 2006
 |
By Ed Woods
Consulting IT Specialist
IBM Corporation
|
IBM Tivoli OMEGAMON XE for IMS offers powerful functions for historical monitoring and analysis, and includes valuable components such as Historical Component (EPILOG) and Transaction Reporting Facility (TRF). Ed Woods discusses these component's architecture and functionality, the performance information they provide, and best practices you can use to get the most benefit from these facilities.
This is the first installment in a two-part article.
IBM Tivoli OMEGAMON XE for IMS can monitor all the essential resources that make up an IMS subsystem, including IMS pools, regions, subsystem address spaces, transactions, databases, programs, locking and more. Tivoli OMEGAMON XE for IMS Classic, Common User Access (CUA), and Tivoli Enterprise Portal (TEP) interfaces can display resource monitoring information in real time and gather historical data as well. In addition to resource monitoring, OMEGAMON XE for IMS provides analysis collectors, such as Response Time Analyzer (RTA) and Bottleneck Analysis (DEXAN), that can look at and analyze a workload running in IMS.
RTA gathers information on transaction response time and breaks it into user-definable information groups that may be viewed in real time to reveal the performance of related transactions. These groups include transaction input queue time, processing time, output queue time, PI time (program to program switch time), R1 response time (the sum of input, output and processing time), and R0 time (the sum of input and processing time). You can define groups based upon transaction code, program, scheduling class, or a combination, and name them to clarify how the workload items are related. The KOIGBLxx macro in the hilev.RKANSAM library is used to define these groups.
Bottleneck Analysis (DEXAN), another collector component, uses the group definitions employed by RTA to further analyze the IMS workload. On an ongoing basis, Bottleneck Analysis looks at the workload and overall activity in the IMS subsystem, gathers counts and then calculates percentages that reflect where a given workload is spending its time. The bottleneck data can show the percentage of time a workload uses the CPU or waits for CPU, the percentage of time a workload is waiting for I/O, and the percentage of time a workload is waiting for an IMS resource, such as an available message region.
Store and analyze interval data with EPILOG
Resource monitoring, RTA and Bottleneck Analysis are important real time components of OMEGAMON XE for IMS. EPILOG, a powerful but often overlooked data collection tool included with the product, extends these capabilities by enabling OMEGAMON to take this real time performance data and store it for later perusal and analysis. A performance analyst can use EPILOG data to answer questions such as: When was response time for a group of transactions above a specified threshold? What was the major bottleneck wait reason for that time interval? And what were the system and subsystem resource usage statistics for the time interval?
EPILOG gathers data by running RTA and bottleneck analysis in parallel with real time OMEGAMON monitor collection and supplements the data with additional resource information such as CPU, storage and DASD information. Data is summarized and stored at intervals, usually 15 minutes, in a VSAM file called an EPILOG Data Store (EDS). There is a one-to-one correspondence of EDS files to IMS subsystems and a separate EDS VSAM file for each IMS subsystem. The quantity and cost of the stored interval information is relatively low compared to other historical data sources, so you can retain it for many days or even weeks.
You can interactively analyze the EPILOG data using commands in a TSO/ISPF interface or via JCL batch jobs. The clist to invoke the ISPF interface is member KEISPF in library hilev.RKANSAM. Example commands and usage scenarios are well documented in Historical Component (EPILOG) Reference Manual (SC32-9360).
You need to know only a small number of commands, such as the display (DIS) command, to derive benefit from EPILOG. For example, DISPLAY GROUP(1) TDAY SUMMARY shows a summary display with a line of information corresponding to each time interval for workload items defined as group 1. The TDAY parameter means show data for today. Figure 1 provides an example of the output from this command.
Figure 1 – Example DISPLAY GROUP(1) SUMMARY command
Each line represents a summary of activity for a given 15 minute interval. The tool uses the most statistically significant Bottleneck Analysis wait reason for the workload in that time interval. To see the detail for a specific time interval, enter 'd' at the left of the line for the desired time interval, and press enter. A sample result is shown in Figure 2, where you can see RTA information for a time interval, including the number of transactions and the RTA component numbers. Below the RTA data is the Bottleneck Analysis data for the workload in that interval, which reveals information such as where the workload is spending most of its time, and where to target tuning.
Figure 2 – RTA and Bottleneck data for a time interval
In EPILOG you can also use commands to display IMS resource usage for given time intervals, such as DIS RCMP for IMS pool statistics, DIS RCPU for CPU utilization by address space, DIS RDAS for IMS dataset utilization, DIS RDMP for DMB pool utilization, DIS RFPI for fast path statistics and more. Figure 3 provides an example of a pool display command.
Figure 3 – Example resource DIS command output
EPILOG can also show information based upon exception criteria. For example, the command DIS GRP(1) TDAY RIF(R1(>2)) SUMM will show group 1 summary data for those time intervals where response time is greater than two seconds.
EPILOG General Usage Recommendations
Follow these best practices to get the most benefit from EPILOG:
|
 |
Customize the EPILOG history workload groups to parallel real time collection. The setup for EPILOG workload groups is control card driven and contained in member KEIOPTxx in library hilev.RKANPAR. Customizing the workload groups makes the data more meaningful and relevant to the workload being monitored. |
|
Learn the main commands, such as DIS and its variations. Save commonly used commands as PF keys in EPILOG using the PFK command. |
|
Take advantage of the batch interface to create useful reports. EPILOG provides sample JCL. The sample members KEIJCTRS, KEIJDBAS, and KEIJSPGS are contained in library hilev.TKANSAM. |
|
Take advantage of EPILOG as a mechanism for top down performance analysis and trending. The data is powerful and easy to access once you know the commands. |
|
Get greater granularity with Transaction Reporting Facility
By design, EPILOG historical data is interval based, and the granularity of the information is limited to the collection time interval. Transaction Reporting Facility (TRF), another historical data collection tool included with OMEGAMON XE for IMS, provides a more detailed level of granularity than EPILOG. TRF provides detail at the individual transaction execution that is suitable for IMS performance analysis and chargeback purposes.
TRF also enables you to gather Database Language/I (DL/I) call statistics for each transaction, including the type of DL/I call, number of calls and elapsed call time. TRF DL/I call history records are written to the IMS log, and OMEGAMON XE for IMS can extract the IMS log records and TRF records from the IMS log files. The result is a TRF history file with message (MSG) records that provide transaction response time, resource utilization statistics, database (DL/I, FP) call detail and database call summary (DL/I, FP, or DB2) records. The record layouts are documented in IBM manual Transaction Reporting Facility (SC32-9358). The JCL to run the TRF Extractor is a member of KI2XTRC in library thilev.TKANSAM.
Give thought and analysis before activating TRF, because TRF has the potential to generate a large volume of historical data that may impact your overall IMS log data volume. DBD is the critical TRF configuration parameter that controls the level of summarization by the TRF collector and log data volume. If DBD is set to 0, then TRF will write a DL/I detail record for every DL/I call. If, as in the example shown in Figure 4, DBD is set to 5, you'll collect only DL/I database summary records for the first 5 IMS databases that a given transaction references; however, for any databases beyond the first 5, the TRF collector will write DL/I detail records.
Figure 4 – OMEGAMON classic command showing TRF status
TRF historical data is much more detailed than EPILOG historical data. The MSG record contains transaction response time, including response time break downs, transaction storage, CPU, and program executed, region executed in, and much more detail. The database detail records will show the name of the database accessed, the type of call and the duration of the call. Database summary records will show the database accessed, then the number of calls by type of call (GU, GN, etc.), and the elapsed time of the calls. Typically a transaction will have one MSG record and multiple database records, either summary or detail, as dictated by the DBD option.
You can report on the TRF extracted data using sample reports provided in library hilev.TKANSAM. Refer to member KI2$IDX for a cross reference of the available reports. Figure 5 shows a sample reports with database call counts and elapsed time correlated by transaction and program.
Figure 5 – TRF sample report output
Transaction Reporting Facility General Recommendations
Follow these best practices to get the most benefit from TRF:
|
|
If you are new to TRF, you may want to turn TRF on and off via the ITRF OMEGAMON command prior running TRF on an ongoing basis. This will allow you to assess the impact of running TRF collection. |
|
Most shops try to set the DBD option to a level that will encourage DB call summarization. Summarization will cut down the quantity of data sent to the IMS log by the TRF collector. |
|
Determine the optimal setting for the DBD parameter. This may be as simple as determining the average number of DBDs accessed by the average transaction. |
|
Since TRF data volume is larger than EPILOG, give more thought to determining you needs for storage, retention, and summarization. |
|
Take advantage of historical data capture built into TEP
A third major historical collection facility in OMEGAMON XE for IMS is the snapshot historical data capture integrated into the Tivoli Enterprise Portal (TEP), which gathers and stores history data on an interval basis (5, 15, 30, or 60 minutes). This historical data may be stored at the level of the OMEGAMON monitoring agent address space in what called the Persistent Data Store files, sent to the Tivoli Enterprise Management Server (TEMS), or sent to the Tivoli Data Warehouse for long term retention. Any information displayed real time within the Tivoli Enterprise Portal may optionally be stored as snapshot history and sent to the Tivoli Data Warehouse.
What's your historical performance analysis strategy?
Historical performance data is an important part of an overall performance analysis strategy. Epilog and TRF are powerful historical data collection facilities included with OMEGAMON XE for IMS. Each facility has its own unique information, strengths and recommended usages. Take care in developing your history strategy, because historical information collection and retention can impact the overall resource cost of monitoring. Consider the type of historical data you want, sources available, retention needs, summarization, and reporting and analysis requirements. Native capabilities of OMEGAMON XE for IMS can provide much of the historical data you need.
Stay tuned for part two – OMEGAMON XE for IMS historical considerations with the Tivoli Enterprise Portal – in next month's CCR2.
For more information
|
|
 |
 |
Subscribe to CCR2's RSS news feed today!
If you are new to RSS, we suggest you read the Introduction to RSS article. |
 Free eNewsletters!
Publications for the IBM Tivoli and System z communities |
|
|
|
|
|
|
