DB2 10.5 for Linux, UNIX, and Windows

Log stream merging and log file management in a DB2 pureScale environment

In a DB2® pureScale® environment, each member maintains its own set of transaction log files (that is, a log stream) on the shared disk, each set in a separate log path. The log files for a member contain a history of all data changes that occurred on that member.

Multiple applications, each accessing a different member simultaneously, might generate dependent transactions during run time. A dependency between two transactions can occur if, for example, both transactions change the same row. To effectively interpret the log records, the DB2 data server must examine the records from all log streams and order the records so that they reflect the order of the updates that occurred at run time. This ordering is known as a log stream merge operation. Several operation types in a DB2 pureScale environment require log stream merges; these include (among others) group crash recovery, database roll-forward operations, and table space roll-forward operations.

Logging configuration parameters in a DB2 pureScale environment

Table 1 shows which logging-related database configuration parameters are global in scope and which parameters are dynamically updatable.
Table 1. Logging-related database configuration parameters
Parameter Global? Dynamically updatable?
archretrydelay Yes Yes
blk_log_dsk_ful No Yes
failarchpath Yes Yes
logarchcompr1 Yes Yes
logarchcompr2 Yes Yes
logarchmeth1 Yes Yes
logarchmeth2 Yes Yes
logarchopt1 Yes Yes
logarchopt2 Yes Yes
logbufsz No Yes
logfilsiz Yes No
logprimary Yes No
logsecond Yes Yes
max_log No Yes
mirrorlogpath 1 Yes No
newlogpath 1 Yes No
num_log_span No Yes
numarchretry Yes Yes
overflowlogpath Yes Yes
page_age_trgt_grc Yes No
page_age_trgt_mrc Yes No
softmax2 Yes No
vendoropt Yes Yes
1 The first member that connects to or activates the database processes the changes to this log path parameter. The DB2 database manager verifies that the path exists and that it has both read and write access to that path. It also creates member-specific subdirectories for the log files. If any one of these operations fails, the DB2 database manager rejects the specified path and brings the database online using the old path. If the database manager accepts the specified path, the new value is propagated to each member. If a member fails while trying to switch to the new path, subsequent attempts to activate the database or to connect to it fails, and SQL5099N is returned. All members must use the same log path.
2
Important: The softmax database configuration parameter is deprecated is deprecated in Version 10.5 and might be removed in a future release. For more information, see Some database configuration parameters are deprecated.

Retrieving logs for a log stream merge operation in a DB2 pureScale environment

A subdirectory is created in the path for retrieved log files. The subdirectory has the following format: log_path/LOGSTREAMxxxx, where log_path represents the log path, overflow log path, or mirror log path, and xxxx is a 4-digit log stream identifier. (The log stream identifier is not necessarily equivalent to the associated member ID.) Within this subdirectory, if a member requires log retrieval, the DB2 database manager creates another level of subdirectories for retrieved logs from each member. For example, if you specify an overflow log path of /home/dbuser/overflow/ on a 3-member system, and an application on member 0 must retrieve logs that are owned by other members, the path for member 0 is /home/dbuser/overflow/NODE0000/LOGSTREAM0000, and subdirectories under this path contain retrieved logs that are owned by other members, as shown in the following example:
Member 0 retrieves its own logs here:
   /home/dbuser/overflow/NODE0000/LOGSTREAM0000/LOGSTREAM0000
Member 0 retrieves logs that belong to member 1 here:
   /home/dbuser/overflow/NODE0000/LOGSTREAM0000/LOGSTREAM0001
Member 0 retrieves logs that belong to member 2 here:
   /home/dbuser/overflow/NODE0000/LOGSTREAM0000/LOGSTREAM0002
Note: Do not manually insert log files in to these retrieve subdirectories. If you want to manually retrieve log files, use the overflow log path instead.

When reading archived log files that are owned by other members, a member might need to retrieve log files in to its own log path or overflow log path. In this case, the log stream merge operation creates a db2logmgr engine dispatchable unit (EDU) for each log stream, as needed.

As mentioned earlier, there are three paths that can be used to store log files that are owned by other members, as shown in the following list:
  1. If you set the overflowlogpath database configuration parameter, the overflow log path is used.
    Tip: You can use ROLLFORWARD DATABASE and RECOVER DATABASE command options to specify an alternative overflow log path; the values of these options override the database configuration for purposes of the single recovery operation.
  2. The primary log path
  3. If you set the mirrorlogpath database configuration parameter, the mirror log path is used.
If the DB2 database manager is unable to store a log file in the first path, it attempts to use the next path in the list. If none of these paths is available, the utility that invoked the log stream merge operation returns an error that is specific to that utility.

Output from the GET DATABASE CONFIGURATION command in a DB2 pureScale environment identifies each log path followed by the name of the member. For example, if the mirror log path was set to /home/dbuser/mirrorpath/, for member 2, the output displays /home/dbuser/mirrorpath/NODE0000/LOGSTREAM0002.

If you must manually retrieve log files that are owned by other members, ensure that the database manager can access the log files by using the same directory structure that is automatically created. For example, to make logs from member 2 available in the overflow log path of member 1, place the logs in the /home/dbuser/overflow/NODE0000/LOGSTREAM0001/LOGSTREAM0002 directory.

Retrieved log files are automatically deleted when they are no longer needed. Subdirectories that were created during a log stream merge operation are retained for future use.

Detection of missing logs during a log stream merge operation

If you accidentally deleted, moved, or archived and lost a log file that is required for a recovery operation, you can roll-forward recover the database to the last consistent point before the missing log file.

If, during a log stream merge operation, the DB2 database manager determines that there is a missing log file in one of the log streams, an error is returned. The roll-forward utility returns SQL1273N; the db2ReadLog API returns SQL2657N.

Figure 1 shows an example of how two members could write log records to the log files in their active log stream. Each log file is represented by a box.
Figure 1. Log files in a DB2 pureScale environment
Diagram showing log files in a DB2 pureScale environment

Consider a scenario where only log file 4 from log stream 1 is missing, a roll-forward operation to time A succeeds while roll-forward operations to time B, time C, or to the END OF LOGS fail. The ROLLFORWARD command returns SQL1273N because log file 4 is not available. Furthermore, since the log records in files 2 and 3 on log stream 0 were written during the same time period as the beginning of log file 4 on log stream 1, the roll-forward operation cannot process log files 2 and 3 until log file 4 from log stream 1 is available. The result is that the roll-forward operation stops at time A, and any subsequent roll-forward operations cannot proceed beyond time A until log 4 from stream 1 becomes available.

Consider another scenario where only log file 4 from log stream 0 is missing during a roll-forward operation. If you issue a ROLLFORWARD command with the END OF LOGS option (or anytime after time B), the operation will stop at time B and will return SQL1273N because log file 4 on stream 0 is missing. A roll-forward operation can replay log records from files 2 and 3 on log stream 0 and some logs from file 4 on stream 1 up to time B. The roll-forward operation must stop at time B even though additional logs from stream 1 are available because the log merge process requires that all the logs from all the streams be available.

If you can find the missing log file, make it available and reissue the ROLLFORWARD DATABASE command. If you cannot find the missing log file, issue the ROLLFORWARD DATABASE...STOP command to complete the roll-forward operation at the last consistent point just before the missing log file.

Although missing log detection ensures that database corruption does not occur as a result of missing log files, the presence of missing log files prevents some transactions from being replayed and, as a result, data loss could occur if the missing log files are not located.

Required resources

Log stream merge operations require additional EDUs. During database activation, one db2lfr EDU is created on each member. When a log read operation that requires a log stream merge is initiated, one db2shred EDU and one db2lfr EDU is created for each log stream. Although each db2lfr-db2shred group allocates its own set of log page and log record buffers, this is not a significant amount of additional memory or system resources; approximately 400 KB is allocated for each member that is involved in the log stream merge.

During a log stream merge operation, a member retrieves log files that are owned by other members into its overflow log path, primary log path, or mirror log path. In a DB2 pureScale environment, ensure that there is adequate free disk space in the retrieval path before starting a roll-forward operation. This allows the operation to retrieve the larger number of files from the archive, as required in a DB2 pureScale environment, without affecting performance. Use the following rule-of-thumb to calculate how much space you need to retrieve the active log files for all members: (logprimary + logsecond) * number of members.

Examples

Related concepts:
Configuration parameters that affect the DB2 pureScale Feature
Related reference:
DB2 pureScale Feature configuration parameters