Concatenating Rolled Logs For A Single Analysis

Answer

The default behavior of Maximo, TPAE Applications, and WebSphere is to roll logs to maintain a reasonable file size. The default size for Maximo and TPAE is 5MB meaning that after a log file reaches 5MB in size, it will be renamed and logging will continue into the new and empty log file since the original has been renamed. The default index size for Maximo and TPAE logs is 20. This means that a history of the most recent 20 rolled logs will be kept before the oldest one is replaced.

Sometimes it may be easier to draw conclusions and capture trends if a log file is complete rather than looking at several separate analyses.

For systems, stability, and performance related issues, it is common for IBM to request up to 24 hours of logs to view trends and analyze when problems begin to surface. In order to capture 24 hours of logs it can sometimes require a little trial and error. If high levels of logging have been enabled we have seen up to 1MB per minute of logs. This is equal to 60MB per hour or 1.4GB per day. In that case it would be recommended to set the log file size to 20MB and set the index to 72. In other cases the logging may be set to more standard sizes and it may be safe to set log file size to 10MB and the index to 24. You would then be able to write 10MB in logs (per JVM) per hour. For information on how to control WebSphere SystemOut log files see document 1640659 Setting WebSphere Logging Options

There are many patterns that can be specified in the logging of these applications. Some include the mxe.name and index number, others include a random number and the index number to maintain uniqueness, and still others may have a time and date stamp appended to the log file name. There are more patterns that just these but the key is that the oldest rolled log is first in the concatenated log, the second oldest is next etc… In this way, the time and date stamps of the log lines are kept in order.



In the case above, the logs from oldest to newest are:

SystemOut_10.05.25_15.47.13.log
SystemOut_10.05.25_16.00.10.log
SystemOut_10.05.25_16.16.00.log
SystemOut.log

To quickly concatenate these logs you would rename the oldest log as 1, the next as 2, the 3^rd as 3, and the newest as 4.



Next go to a command prompt and change to the directory where the renamed logs are. Execute the command:

Copy 1+2+3+4 NewLogFile.log



The file NewLogFile.log now contains the concatenated contents of all four logs.

The following may be a little more difficult. In this case each JVM that starts up creates it’s own ID followed by the log file name and then an index if it has rolled. In this case 1267609060271_maximo.log has rolled twice so there are 2 iterations with the oldest being .2 second oldest being .1 and then the current .log. All of these are for the same JVM. These should be concatenated together for a single analysis but other logs should not be added to this collection because they are from a different JVM or startup.



A good procedure for renaming these might be:

11, 12, 13
21, 22, 23
31, 32, 33
41, 42, 43



Then use the same procedure for concatenating the logs together.

Copy 11+12+13 JVM1.log