Managing log data in interim storage to minimize movement to DASD

You can use a very large coupling facility structure in the magnitude of gigabytes vs. megabytes to minimize movement to DASD for cases where log data deletion is actively managed by the exploiter. The use of a supersized CF structure is intended to minimize the likelihood of log stream offload delays that impact an exploiter that writes log data into the log stream. Coupled with configuration changes and taking advantage of monitoring and automation, this will allow the installation a path a greater resiliency and minimize the impact of offload inhibitors when they occur. Offload inhibitors generally fall into two categories:

DASD allocation hang: happens when logger data set allocation hang obtaining "next to be used as current" offload data set.
DASD I/O bandwidth slowdown: happens when an I/O performance delay causes the rate of a new log data written into a log stream to exceed system logger’s ability to write log data to offload data sets and subsequently make more room in the structure.

The offload inhibitors often occur for reasons outside of the control of system logger. Due to the nature of these problems, the supersized structure environment takes the approach of configuring the log stream and its exploiter in a manner so that the log stream does not need to be physically offloaded to DASD during normal activities. This will assist in avoiding potential problems with offload dataset allocation or I/O performance. This approach is already taken by CICS Transaction Server (CTS) with respect to configuring a CICS system log. The same type of approach can be used by other log stream exploiters.

The installation will need to make operational changes to the specifications on log stream and CF structure attributes to provide the installation awareness and ability to react if the log stream primary/interim resource is becoming constrained. The installation should focus on taking the necessary action to allow a log stream exploiter to logically delete the data periodically.

When sizing and setting these attributes, you should keep in mind that the goal is to allow a "warning track" for log stream offload processing in case an inhibitor is encountered. The installation will set a low percentage for the logger HIGHOFFLOAD so that normally very little of the CF structure would be used. If the offload is keeping up with the write rate, the HIGHOFFLOAD threshold should not be significantly exceeded.

The CF structure has been changed to allow for a more automated approach. System logger provides the user with programmatic data about structure usage. IMS Common Queue Services (CQS) will act on this data to determine when to checkpoint. If you use IMS CQS as an example of a log stream exploiter that is usually considered a funnel-type log stream user with the following algorithm, than you can use an active-type log stream user. These changes are only effective when the log stream and related resources are configured correctly. For specific references on CTS and IMS CQS using log streams, see Finding information for CICS log manager and Finding information for IMS common queue server log manager.

The algorithm used for determining the various CFRM specifications is as follows:

Structure Size

The structure size = 2 IMS check points of data + safety zone + 2 minutes of data + reaction time data.

2 IMS check points of data = 2 * amount of data written in one minute * the number of minutes between structure checkpoints.
Safety Zone = 20% of the 2 IMS check points of data.
Reaction time data = (the minutes of desired reaction time + 1) * the data written in one minute.

Ensures that the structure is large enough to hold 2 IMS CQS check points of data and create a reaction time or warning track to handle any offload issues if system logger needs to physically offload data.

INITSIZE and MINSIZE

Allows XES to manage the CF structure size based on the above SIZE specification.

Note: Do not code these options.

FULLTHRESHOLD

FULLTHRESHOLD = (100 - HIGHOFFLOAD) / 3 + HIGHOFFLOAD

The IXC585E is the warning message for bandwidth issues causing offload delays. Ensure the IXC585E comes out early enough in an error situation to permit reaction and ensure the message is not issued frequently during normal operation.

ALLOWAUTOALT(NO)

CF structures used for log streams generally have ALLOWAUTOALT(NO) specification.

The considerations for using this option are explained in Define the coupling facility structures attributes in the CFRM function couple data set in z/OS MVS Setting Up a Sysplex.

The algorithm used for determining the various LOGR specifications is as follows:

STG_DUPLEX(NO): Stipulated that the super-sized structure works only if the staging dataset was not being used.; Restriction stems from system logger log stream staging data set size being limited to 4GB which is the maximum size staging dataset. Log blocks written into the staging data set are rounded to the 4K (CI) boundary, which means that much of the staging data set may be white space. If you super-size the structure with staging data sets in place, they will likely offload due to the staging dataset utilization and not the newly recommend HIGHOFFLOAD and LOWOFFLOAD points for the structure. When you use staging data sets it is unlikely that you will achieve your goal in not having to physically move data to DASD and waste a fair amount of space in the structure.; Mathematically, if there are many systems writing evenly, there is an outside chance of success with the super-sized structure and staging data sets since a staging data set is allocated for each system and for each log stream. This outcome would be based on the logging rates on the different systems.
RETPD: RETPD = 0 (default); A retention period (RETPD) value forces log data to be offloaded to DASD before it can be deleted. Specifying a nonzero retention period value defeats the purpose of sizing a CF structure to large to avoid offloading during peak times.; If the installation must use the RETPD value to force data to be retained for a period of time, then you should not use this super-sized structure approach as a mechanism to avoid offloads.
HIGHOFFLOAD: HIGHOFFLOAD = (2 IMS check points of data + safety zone + 2 minutes of data) / total structure size; Ensure the LOWOFFLOAD and HIGHOFFLOAD are sufficiently close together so that the offload happens more frequently. Ensure the LOWLOFFLOAD and HIGHOFFLOAD are far enough apart so that system logger is not continuously offloading.
LOWOFFLOAD: LOWOFFLOAD = (2 IMS check points of data + safety zone) / total structure size; Ensure the LOWOFFLOAD is set so that 2 IMS CQS checkpoints of data remain in the structure.
LS_DATACLAS: LS_DATACLAS = 24 CI size and striping.; Ensure making use of recommended and higher performing attributes for offload data sets.
LS_STORCLAS: LS_STORCLAS = striping.; Ensure making use of recommended and higher performing attributes for offload data sets.

Although this approach is designed to minimize the offloading of data to DASD, it must be noted that no matter how large a log stream’s primary storage is configured, system logger will write data to offload data sets for certain events or activity’s. When the connector of the IMS CQS log stream terminates on a given system or if there is a major spike in the workload, a rebuild will occur.

This environment introduces additional concerns regarding the duplexing of log data and related resources. Staging data sets should not be used in this environment because it will constrain the size of the CF structure to the maximum supported staging data set size of 4GB. One implication of this is another method of duplexing CF structure resident log data must be used.

CF structure resident log data can be duplexed by XES structure duplexing, system logger local buffers, or some combination of these. The factors that determine this are the definition of the log stream and structure resources, as well as the environment of the system. If the intent is to use a duplexed CF structure with XES system managed duplexing), then the installation will need to be aware that if the structure falls back to simplex mode, system logger will initialize an offload to move all eligible structure resident log data to DASD. System logger will then have control and the component has to ensure log data is solidified when it occurs. If the intent is to use an environment where local buffers are the duplexing medium, the installation must ensure it has adequate page data set space available to back up the local buffers it needs.

This supersized structure raises considerations with regards to certain recovery cases. One such case is the existing system logger processing with regards to the loss of a connected CF structure. Currently, certain structure failures will trigger system logger to allocate a staging data set when a rebuild to a new structure fails. In this case, system logger will allocate a staging data set to harden the log data so it can be recovered later. This is done to preserve the data until a structure can be allocated. If the installation has not defined the staging data set parameters that control the size, the allocate will fail since the default system logger behavior is to use the CF structure size when sizing the staging data set. If the installation defines the staging data set related log stream parameters, then system logger may be able to preserve some of the data. This staging data set fallback copy is important as it is a method of preserving the log data against cases where a system logger image fails since the local buffer copies of the data would be lost when the address space goes away.