IBM Support

Perl script to collect IBM Spectrum Protect server monitoring data

Technote (troubleshooting)


Problem(Abstract)

Is there an easy way to collect monitoring data needed to evaluate IBM Spectrum Protect/Tivoli Storage Manager server activities (performance, processes, sessions)?
Which data to collect in a first step?

Symptom

The script, as an example, can help you to investigate server performance.

Note: Version 2 of the monitoring script is available in evaluation mode, it supports easier configuration and you might want to give the new version a try. While in evaluation mode you can find it here: servermonV2.pl


Resolving the problem

You can use the Perl script from the attachment to collect data needed. You can adjust the script according your needs, but in case you plan to send in the data for a support ticket please check with your support representative or use the defaults.

By default the script runs in the foreground and prompts you for required parameters.

The minimum variables to specify for the script to work in the background are:

my $server        = "localhost";    # server stanza for UNIX, TCPSERVERADDRESS for MSWin32
my $tcpport       = "1500";         # only used in MSWin32
my $administrator = "admin";        # the admin id
my $password      = "password";     # guess what?

my $instance      = "tsminst1"; # DB2 instance name, if your DB2INSTANCE  environment variable is set it will override this value

Then start the script with the additional noprompt parameter, e.g.:
AIX: perl64 servermon.pl noprompt
(the perl64 interpreter prevents from an out of memory symptom on AIX systems)

Other platforms: perl servermon.pl noprompt
or ./servermon.pl noprompt

To run the script in the foreground, just invoke the script without the additional noprompt parameter)

For *IX users, if you want to define a cron job to start the script unattended, you need to define a wrapper script to source the instance users profile. This is necessary for the DB2 connect to work. A sample crontab entry would look like:

45 2 * * * /se_tools/tsm/perl_script/servermon.wrapper 1>/se_tools/tsm/perl_script/servermon.wrapper.log 2>&1

The wrapper script then looks like:

[AIX]: cat /<path_to_script>/servermon.wrapper
#!/usr/bin/ksh
. /home/tsminst1/.profile
/usr/bin/perl64 /<path_to_script>/servermon.pl noprompt 1>/<path_to_script>/servermon.pl.log 2>&1

A detailed explanation of the parameters to specify can be found below.

You can easily adjust the script to collect data as you like, the default set has been found helpful in analysis of IBM Spectrum Protect/Tivoli Storage Manager server performance. Please make sure to run the script from a user with sufficient DB2 access rights (e.g. the instance user) with the DB2 environment properly initialized. Message DB21061E is thrown if the script fails to connect to the database.

In general, this script is very easy to setup and run and provides excellent information to support for diagnosis of several different kinds of IBM Spectrum Protect/Tivoli Storage Manager problems. It is recommended to run the script as the instance user from the IBM Spectrum Protect/Tivoli Storage Manager server box. If your installation paths are different to the default paths you need to adjust the path variables accordingly. Alternatively you can run the script with DSM_DIR and DSM_CONFIG environment variables specified.

If you want to use the script to collect storage agent monitoring data. the $server and $tcpport values need to point to the storage agent and the script can be run from any system with the IBM Spectrum Protect/Tivoli Storage Manager administrative client (dsmadmc) installed. Under this condition some of the docs collected, e.g. iostat, render invalid.

Script output is generated in the current directory from where the script is started from.

The script formerly was referred to as server performance script, to reflect the increased usage of the script it has been renamed to "server monitoring script".

Note: If you are running your server on a Windows box, you need to install a perl interpreter to be able to run the script. There are free and commercial interpreters available, see the related URL section for links to some free interpreters.

Run the script from a DB2 command window as instance user:
Start -> Run -> db2cmd

From there, cd to the target directory for the files generated using the script.

Notes: The following APARs should be reviewed before running the script and make sure corresponding fixes to the IBM Spectrum Protect/Tivoli Storage Manager Server have been applied.

1. APAR IC72251 documents a problem that the SHOW THREADS command can crash the server. The script uses this command frequently on the Windows platform, so please make sure the fix for this is applied. See the link section for APAR details. On platforms other than Windows, the SHOW THREADS usage has been greatly reduced, however it is still recommended to apply the referenced fix.

2. APAR IC79957 documents a problem that there is a rare chance that SHOW DEDUPDELETEINFO might crash the server. The script has been modified so that the command is commented out. If you have the fix installed , and you need to collect diagnostic data related to deduplication, you can reactivate the command again.

3. APAR IC82490 documents a problem that the SHOW SESSION command can crash the server. The script uses this command frequently all server platforms, so please make sure the fix for this is applied. See the link section for APAR details.

4. APAR IC83787 documents that the server might get hung caused by a deadlock between client dedup sessions and server show commands.
APAR IC96438 documents that the server might crash if two INSTRUMENT END commands run in parallel. Make sure not to manipulate instrumentation tracing while the script is active.

5. APAR IT21957 documents that SHOW THREADS command might hang a Linux 8.1.2.000 server. Script version 1.40 and newer does not collect SHOW THREADS command output on the Linux platform, however it is important to apply the fix as for doc analysis there is a lot of dependancy for the "show threads" to be working correctly.

If the script is run in the foreground you will be prompted for the following parameters, the same parameters are available via the configuration section of the script itself:

Enter the servername as found in dsm.opt or dsm.sys [localhost]:
On Windows boxes specify the TCP/IP address of the target server, as the script is expected to be run on the same box as the server is running usually localhost will work.
For all non-Windows environments specify the servername as found in dsm.sys/dsm.opt.

Enter the tcpport [1500]:
On Windows boxes you will be prompted for the port the server is listening for dsmadmc commands.
For all non-Windows boxes this parameter is not required as the definitions are taken from the server stanza.

Enter the TSM administrator login ID [admin]:
Specify the IBM Spectrum Protect/Tivoli Storage Manager administrator id to collect the command output with.

Enter the TSM administrator password (warning, it will echo to the screen) [password]:
Specify the current password for the Tivoli Storage Manager administrator.

Enter PMR number as PPPPP,BBB,CCC [none]:
If the doc collection is for a PMR you can specify PMR number, branch and country and the collected docs will be put together for easier upload to the PMR doc repository.

Collect SHOW THREADS output? (Y/N) [Y]:
Earlier versions of the server could show problems when SHOW THREADS was collected. Current servers do not show that symptom, so the default is to collect the docs. It is not recommended to run production Tivoli Storage Manager servers on levels that do not have the related code fixes applied.

Collect SHOW DEDUPDELETE output? (Y/N) [Y]:
Earlier versions of the server could show problems when SHOW DEDUPDELETE was collected. Please see APARs listed above for more information under the section titled "Notes". Current servers do not show that symptom, so the default is to collect the docs. It is not recommended to run production Tivoli Storage Manager servers on levels that do not have the related code fixes applied.

Collect SHOW ALLOC output? (Y/N) [N]:
SHOW ALLOC output is not collected by default, but the information is helpful if the symptom to investigate is memory usage related. Please use as advised by IBM support.

Enable collectstmt instrumentation? (N/Y) [N]:
Enabling COLLECTSTMT will result in the generation of instrumentation statistics for long running SQL statements. Starting with version 1.24 of the script the default changed from "N" to "Y". Please use as advised by IBM support.

Enable SNAPSHOT FOR DYNAMIC SQL? (N/Y) [N]:
Enabling SNAPSHOT FOR DYNAMIC SQL will result in the generation of additional DB2 statistics. As the collection could impact server performance the default is set to not collect the statistics. Usually collecting snapshot for dynamic SQL requires to enable certain DB2 monitors, please use as advised by IBM support.

Monitor log spanning? (N/Y) [Y]:
Log span monitoring is enabled by default, the information gathered can help to analyze transactions that use lots of active log space.

Enter comma separated stop events, e.g. ANR#####E [none]:
Here you can specify a comma separated list of messages as logged to the activity log. If any of the messages specified is found during the runtime of the script, the script will go through some final doc collection and then stop.

Enter whitespace separated traceflags [none]:
Here you can specify any server traceflags as asked for by IBM support. Tracing will be stopped upon stopping the script via sending SIGINT or if the number of diagnostic cycles are reached (see below).

Enter traceflags to disable [none]:
Here you can specify any server traceflags to disable, for e.g. if you specified trace aggregate AF you might want to disable AFTXN subset. Specify as instructed by IBM support.

Enter size limit for the tracefile. (1-65535MB, -1 unlimited) [-1]:
If you enabled tracing via the traceflags parameter you can specify the maxsize limit in MB for the generated tracefile.

Copy FFDCLOG file? (N/Y) [Y]:
At the end of each (documentation collection cycle (see below) add a copy of the FFDCLOG file to the collected docs. Default is "Y".

Enter number for diagnostic cycles of 1200s to run? (72 = 1 day, indefinite if -1)  [-1]:
Here you can specify how many diagnostic cycles you want to collect documentation for, e.g. 3 cycles would collect data for ~ 60 minutes. As gathering the docs can take up some time, the time until script completion might differ.

Only if you specify a positive value for diagnostic cycles, you will get prompted for collection cycles.
The following prompt example is for 72 diagnostic cycles:

Enter number of 86400s documentation collection cycles to run? (1 = 1 day, indefinite if -1) [1]:
This option can be used to configure some long term monitoring. In the above example (72 diagnostic cycles), if for example you specify 7 collection cycles, monitoring will be active for about 7 days. Again, as gathering the docs can take up some time, the time until script completion might differ. Once the number of diagnostic cycles (see above) is reached, the docs collected during this collection cycle will get zipped and a new collection will get started. (This requires the perl Archive::Zip module to be installed.) Basically with the number of collection cycles you specify a loop around diagnostic cycles.

Note also that if the script is being run with noprompt with a set number if diagnostic/collection cycles, it could also be used with some kind of a task scheduler (making sure that the DB2 environment is being established correctly first). This way, the script could be run continuously on a day to day basis to gather performance data or monitoring information for a given problem. In addition, the corresponding files created could be also be archived off (and deleted) with Tivoli Storage Manager for historical purposes using a client schedule on the Tivoli Storage Manager/Spectrum Protect server that had been created.

Related information

IC72251
IC79957
IC82490
IC96438
IT21957
ActivePerl
Cygwin
Strawberry Perl for Windows
Script out of memory problem on AIX

Server monitoring script: servermon.plservermon.pl


cksum servermon.pl
91069637 79700 servermon.pl

md5sum servermon.pl
d4f40f0fa7f1a54623a0a806a9de023b *servermon.pl

Document information

More support for: Tivoli Storage Manager
Server

Software version: All Supported Versions

Operating system(s): Platform Independent

Reference #: 1432937

Modified date: 15 December 2017


Translate this page: