Unexpected high number of EndTxn processing during back up

Resolving The Problem

In this scenario, the following settings have been defined:

Server->TxnGroupMax 256

Client-> TXNBYTELIMIT 25600

The following statistics have been reported by a client instrumentation trace:

  Server Version 6, Release 3, Level 3.0                                
  Data compression forced off by the server                            
  Server date/time: 29-06-2006 12:00:05 Last access: 29-06-2006 09:15:07
                                                                       
 Current command:                                                        Incremental                                                            
Total number of objects inspected:   74.819
Total number of objects backed up:   74.814
Total number of objects updated:          0
Total number of journal objects:     79.687
Total number of objects rebound:          0
Total number of objects deleted:          0
Total number of objects expired:      4.265
Total number of objects failed:           0
Total number of bytes transferred:    13,96 GB
LanFree data bytes:                       0  B
Server-Free data bytes:                   0  B
Data transfer time:                5.295,51 sec
Network data transfer rate:        2.766,04 KB/sec
Aggregate data transfer rate:        699,58 KB/sec
Total number of bytes pre-compress:  14.996.313.174
Total number of bytes post-compress: 14.996.313.174
Objects compressed by:                    0%
Elapsed processing time:           05:48:57
Average file size:                 195,74 KB
                                                                 
------------------------------------------------------------------
                                                                 
Detailed Instrumentation statistics for                          
Thread: 6328  Elapsed time 20937,580 sec                
Section      Actual (sec) Average(msec)  Frequency used  
                                                         
---------------------------------------------------------
Process Dirs         0,000           0,0              0  
Solve Tree           0,000           0,0              0  
Compute              1,657           0,0         270984  
BeginTxn Verb        0,141           0,0          11167  
Transaction         48,856           4,4          11167  
File I/O           299,004           1,0         307988  
Compression          0,000           0,0              0  
Encryption           0,000           0,0              0  
CRC                  0,000           0,0              0  
Delta                0,000           0,0              0  
Data Verb         2726,280          10,1         270984  
Confirm Verb         0,000           0,0              1  
EndTxn Verb      17755,910        1590,0          11167          
Sleep                0,000           0,0              0          
Thread Wait         85,075        2430,7             35          
Other               20,657           0,0              0          
                                                                 
------------------------------------------------------------------
                                                                 
Detailed Instrumentation statistics for                          
                                                                 
Thread: 6508  Elapsed time 20908,581 sec                          
Section      Actual (sec) Average(msec)  Frequency used          
                                                                 
------------------------------------------------------------------
Process Dirs         0,000           0,0              0          
Solve Tree           0,000           0,0              0          
Compute              1,633           0,0         250649          
BeginTxn Verb        0,076           0,0          11370          
Transaction         49,845           4,4          11370          
File I/O           304,188           1,1         288521          
Compression          0,000           0,0              0          
Encryption           0,000           0,0              0          
CRC                  0,000           0,0              0          
Delta                0,000           0,0              0          
Data Verb         2563,864          10,2         250649          
Confirm Verb         0,110         110,0              1          
EndTxn Verb      17901,403        1574,4          11370          
Sleep                0,000           0,0              0          
Thread Wait         72,014        1636,7             44          
Other               15,448           0,0              0          
                                                                 
------------------------------------------------------------------
From above client instrument:detail stats:

1) total # of files backed up: 74814
2) # of EndTxn's:
thread 6328: 11167
thread 6508: 11370
Total EndTxns: 22537 or avg # files/txn: 3.3.

With TXNBYTELIMIT=25600 KB , TxnGroupMax=256 and Average file size: 195,74 KB, it seems that the above stats show that Tivoli Storage Manager client is requesting way too many DB commits (EndTxn's) from Tivoli Storage Manager server.

Avg # of files/txn should be closer to 256, rather than 3.3 as calculated above.

Further investigation did show that different management classes with different copy destinations were configured for files and directories with this setup.

In this case, the directories on the fileservers changed just as much as the files, forcing Tivoli Storage Manager client to request EndTxn's every time it encountered files/dirs being backed up that are destined for a different MC during backup operation.

The change implemented in this case was to use the same management class for files and directories. The above example should give you a starting point to verify if you are facing this condition when running into performance problems during back up.

For more information on client instrumentation traces see 15.1.2 Tivoli Storage Manager client performance tracing in the IBM Tivoli Storage Manager Implementation Guide:

http://www.redbooks.ibm.com/redbooks/pdfs/sg245416.pdf