IBM Support

Collecting Data for Spectrum Protect Flashcopy Manager for Unix/Linux

Troubleshooting


Problem

Collecting Data documents aid in problem determination and save time resolving Problem Cases (previously called PMRs).

Cause

Review the following Knowledgebase documents for additional troubleshooting information:

Tivoli Storage FlashCopy Manager V4.1 documentation:
IBM Tivoli Storage FlashCopy Manager for UNIX and Linux, Version 4.1, Troubleshooting
Requirements, Pre-Installation Checklist and Planning Sheet: Provides the requirements for each FlashCopy Manager version. The individual requirement documents also contain links for the Pre-Installation Checklist and the Installation Planning Sheet.

Environment

Operational Flow of the Snapshot/ Flashcopy Manager processing:

Spectrum Protect Snapshot / Flashcopy Manager (FCM) sits between the application (i.e. Oracle, DB2, SAP, custom app), the storage system and Spectrum Protect. It is optional to backup the data to the Spectrum Protect Server. Snapshot / FlashCopy Manager is a standalone product that can be used to just run snapshots (aka flashcopies) to local disk. "Local disk" is either SVC, Storwize V7000, XIV, or DS8000.

Spectrum Protect Snapshot / FlashCopy Manager has the following basic components:
· the application agent (db2acs, acsora, backint, fcmcli), the management daemon (acsd)
· the background monitoring demon and mount agent (acsgen -D [-M]),
· the offload agent (tsm4acs -D)
· the device agent (acsgen)

The production server initiates the flashcopy of source to target volumes, and the acsgen on the backup server is responsible for the mounts and unmount. All processing is started on the production server because that is where the management daemon (acsd) controls all the action.

Diagnosing The Problem

Try to identify where, in the Snapshot / FlashCopy Manager process, the problem is occurring. The summary.log is a good place to start. Most errors should be readily identifiable. Usually, these problems are syntax errors in the profile or .fct. If the summary.log does not help, try to identify the problem in the acsgen and tsm4acs log files.
If the problem cannot be identified in the log file and if it occurs at any time PRIOR to the backup to the Spectrum Protect server, then set TRACE to YES in the GLOBAL section of the Snapshot / FlashCopy Manager profile. This will generate logs and traces for each component. Make sure that the parameter TRACE is set to YES in the profile on the production and on the backup server. Be sure to restart the management daemon (acsd) and the other daemon processes (acsgen and tsm4acs)

Resolving The Problem

Collecting Troubleshooting Data early, even before opening the Case, helps IBM® Support quickly determine if:
Symptoms match known problems (rediscovery).
There is a non-defect problem that can be identified and resolved.
There is a defect that identifies a workaround to reduce severity.
Locating root cause can speed development of a code fix.

Collecting Data: Table of contents:

Gathering general information
Manually Gathering General Information
Gathering Snapshot Information
Submitting Information to IBM Support
Online Self-Help Resources
Related Information


 
Gathering General Information

Entering general information into an electronically-opened Problem Case (previously called PMRs) eliminates waiting on the phone to provide general information to support. 

 
]Manually Gathering General Information[

Gather the following files:
  • dsm.opt file specified by the variable DSMI_CONFIG
  • dsm.sys (from the /usr or opt/tivoli/tsm/client/api/bin or bin64 directory)
  • dsmerror.log
  • dsierror.log
  • dsmsched.log
  • The output from a 'set' command
  • details of operating system levels
  • details of Oracle or DB2 instance levels
  • profile file (default is /HOME/acs/profile - this would be defined by the "-p" option when running the tsm4acs)
  • .fct file (this is specified in the profile file by the VOLUMES_FILE parameter)
  • init<SID>.utl (if this is an SAP environment)
Manually Gathering Snapshot Information

1) Be sure to have TRACE YES in the GLOBAL section of the profile on production and backup server
2) Collect all logs in the ACS_DIR/logs directory on both backup and production servers as well as
  • ps -ef | grep acs
output from both servers and the acsrespository.
3) Identify whether this is an Snapshot for native DB2, native Oracle, SAP, or custom applications and collect the db2diag.log, RMAN message log and Oracle alert log, or brbackup/brrestore log accordingly.
4) Try to identify where in the process the problem is occurring. The summary.log is a good place to start. If the problem is in the flash, then you won't have a lot of traces/logs. Otherwise you should look at the application agent or device agent trace files. If the problem is in the mount, then it made it past the flashcopy. You should look at the mount agent traces (acsgen). If the problem is in the backup to the Spectrum Protect server, then collect the Data Protection for SAP configuration files and traces (if using SAP) or Backup/Archive client traces and config files.

Gather the following files from the <ACS_DIR>/logs directory(not all files will exist, depends on the specific setup):
LOGS:
summary.<timestamp>.log
monitor.<timestamp>.log
details/<function>.<longtimestamp>
rman.<SID>.<id>.log
db2diag.log

TRACES:
acsd.<id>.<trace>
client.<instance>.<db name>.<node>.<id>.<trace>
acsgen.<hostname>.<device class>.<node num>.<id>.trace
acsgen.<hostname>.<function>.<id>.trace
acsgend.<hostname>.<id>.trace
xivadapter_<id>_<function>.trace
fmcima.<hostname>.<function>.<id>.trace
fmcima.<hostname>.<device class>.<node num>.trace
tsm4acsd.<host>.<id>.trace
fcmcli fcmcli.<host>.<id>.trace

After the traces/logs are collected, set TRACE NO in the profiles to turn off tracing. Continuing tracing could fill the filesystem as cumulative traces can grow quiet large.


 
Submitting Information to IBM Support

After a Case is open, you can submit diagnostic troubleshooting data to IBM.

When data is uploaded to the Case, the status will be automatically be updated to acknowledge that data has been received. If you are still working with older Problem Management Records, you will want to update the PMR to indicate that you have sent the data.

 
Online Self-Help Resources
 
Related Information

[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSERFV","label":"IBM Spectrum Protect Snapshot"},"Component":"Not Applicable","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}},{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SS36V9","label":"Tivoli Storage FlashCopy Manager"},"Component":"","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]

Product Synonym

TSM FCM DP

Document Information

Modified date:
01 February 2022

UID

swg21502353