Collecting data for InfoSphere Federation Server Hang Issues

Answer

Provide the following output for looping or hanging problems
1. Exact name of product installed by issue this command and operating system command prompt:
db2licm -l

2. Current version of DB2 engine level. Enter the following from an operating system command prompt:
   db2level

3. The operating system type, and release, where Federation Server is installed. On UNIX/Linux, enter the following from an operating system command prompt:
   uname -a

For AIX, also run below command:
oslevel -s

For Linux, provide the vendor/flavor/release too
- On SuSE Linux, enter:
    cat /etc/SuSE-release
- On RedHat Linux, enter:  
    cat /etc/redhat-release

On Windows, right click on "My Computer", then select the Properties option.

4. What is the data source used? Provide data source client and server exact release and version.
Examples commands:

DB2 for LUW
Enter the following from an operating system command prompt:
db2level

Informix
Enter following from an operating system command prompt:
cd $INFORMIXDIR/bin
./ifx_getversion clientsdk
./ifx_getversion libasf.a
./ifx_getversion libasf.so

Oracle
Enter the following from an operating system command prompt:
cd $ORACLE_HOME/bin
./sqlplus

Microsoft SQL Server
Issue the following SQL statement (from the Microsoft Query Analyzer):
select @@version

IBM Branded DataDirect ODBC driver
Go to directory <IBM_branded_odbc_install_dir>/bin and execute below command:
ddtestlib <libname>

For example:
i) Get ODBC driver manager version:
ddtestlib <IBM_branded_odbc_install_dir>/lib/odbc.so

ii) Get Microsoft SQL Server ODBC driver version:
ddtestlib <IBM_branded_odbc_install_dir>/lib/VMmsss23.so


Sybase
sybase client version:
Go to directory $SYBASE/$SYBASE_OCS/bin and execute below command:
isql -v

Sybase server version:
Issue the following SQL statement after connecting database using isql command:
select @@version
go

Teradata
Start DWB
issue get version command

5. Have there been any recent changes to your environment (hardware, operating system, software version)?

6. Has this problem occurred before? If yes, do you have a previous PMR number?

7. How often does this problem occur?

8. Can the problem be reproduced? If so, provide exact details of problem including steps performed when problem encountered. Jobs, scripts, applications executing when problem occurred.

9. Execute "db2fodc -hang full".

10. Generate all db2fmp call stacks if the wrapper runs as fenced mode. db2fodc calls "db2pd -stack all" to collect stack trace. But it does not generate stack trace for db2fmp.

UNIX:
Get all the db2fmp process IDs with the following command:
ps -ef | grep db2fmp
Issue the following command to EACH db2fmp process with the following command
On AIX:
kill -36 <db2fmp process ID>
On other UNIX:
kill -URG <db2fmp process ID>
The command will generate stack trace files as <instance path>/sqllib/db2dump/<db2fmp process ID>.<db2fmp thread ID>.000.app_stack.txt

Linux:
Get all the db2fmp thread IDs with the following command:
ps -eLF |grep db2fmp
Note: Thread ID is LWP column of the "ps" output
Issue the following command to EACH db2fmp thread ID with the following command
db2pd -pstack <db2fmp thread ID>
Copy the output as <db2fmp process ID>.<db2fmp thread ID>.stack.txt and put it in
<instance path>/sqllib/db2dump/

Windows:
Download process explorer: http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx
It doesn't need to install, just download and run it.
Right click on the process line, choose Properties, click on Threads, choose db2fmp.exe, then, click on Stack button. For the generated files(the file with stack trace), use the following command to format file before sending to IBM Support.
db2xprt DB30882416.TRP DB30882416.FMT

11. Collect db2support
db2support . –d <dbname> -cl 0

12. Check for deadlock or hang condition on datasource.
Issue following commands from data source client to check datasource lock status

i) DB2 LUW
db2 get snapshot for locks on <dbname>

ii) MS SQL Server
Consult your MS SQL Server DBA.
"sys.dm_tran_locks" may help.
Related links:
+ http://msdn.microsoft.com/en-us/library/ms190345.aspx

iii) ORACLE
Consult your ORACLE DBA.

iv) INFORMIX
Consult your Informix DBA.
See following link:
+ http://publib.boulder.ibm.com/infocenter/idshelp/v115/topic/com.ibm.oat.doc/ids_oat_064.htm

v)SYBASE
Consult SYBASE DBA. Running the following commands with isql may help:
sp_transaction
sp_lock


vi) TERADATA
Consult TERADATA DBA
13. Is this a production, development, or test environment?

14. What is the business impact of this problem?
Examples:
a. Cannot roll out a new application
b. Cannot monitor in-store security systems
c. Losing money due to applications not talking to each other

15. Zip, or compress, all files and rename with PMR number as prefix before submitting ; example 12345.678.999.files.zip.


Submitting information to IBM Support

Once you have collected your information, you can begin Problem Determination through the product Support web page, or simply submit the diagnostic information to IBM support. Use the URL below for submitting information to IBM Support.
http://www.ecurep.ibm.com/app/upload

For a listing of all technotes, downloads, and educational materials specific to the InfoSphere Federation Server component, search the product support site at the following link : http://www-01.ibm.com/software/data/infosphere/support/federation-server/