IC63740: BACKUP STGPOOL OR RECLAMATION FAILS WITH ANR0103E AND ERROR 115.

IBM Tivoli Storage Manager V6.1 Fix Pack 3 (6.1.3) Server Downloads

APAR status

• Closed as program error.

Error description

• On Tivoli Storage manager 6.1.2 Reclamation and Backupstgpool
  can fail and generate error as below.
  .
  ANR0103E aftxn.c(1434): Error 115 updating row in table
  "AF.Vol.Clusters".
  .
  Reclamation will fail with the following errors:
  .
  [3514][output.c][6653][PutConsoleMsg]:ANR2017I
  Administrator ADMIN issued command: RECLAIM STGPOOL COPY1 TH=60
  DUR=300 wait=yes
  [3516][output.c][6653][PutConsoleMsg]:ANR1040I
  Space reclamation st
  arted for volume A00162L3, storage pool COPY1 (process number
  30).
  [3516][output.c][6653][PutConsoleMsg]:ANR1044I
  Removable volume /path/to/00021903.BFS is required for space
  reclamation.
  [3515][output.c][6653][PutConsoleMsg]:ANR1044IRemovable volume
  A00144L3 is required for space reclamation.
  [3633][output.c][6653][PutConsoleMsg]:ANR8337I LTO volume
  A00199L3 mounted in drive DRIVE2 (/dev/rmt2).
  [3631][output.c][6653][PutConsoleMsg]:ANR1340I Scratch volume
  A00199L3 is now defined in storage pool COPY1.
  [3516][output.c][6653][PutConsoleMsg]:ANR0103E aftxn.c(1434):
  Error 115 updating row in table"AF.Vol.Clusters".
  
  [3516][output.c][6653][PutConsoleMsg]:ANR1181 Eaftxn.c(309):
  Data storage transaction 0:1035184 was aborted.
  [3516][output.c][6653][PutConsoleMsg]:ANR2183W afmove.c(6591):
  Transaction 0:1035184 was aborted.
  [3516][output.c][6653][PutConsoleMsg]:ANR1093E Space reclamation
  is ended for volume /path/to/00021903.BFS. The transaction is
  ended.
  
  .
  Backup Stgpool failure shows the following:
  .
  [12825][aftxn.c][1965][AddCluCurVolId]:Exit, rc 115 for volid
  1285.
  [12707][dsalloc.c][1912][DsWriteSeg]:Writing 262144 bytes (64
  blocks) directly to disk /dev/rvol_tsm12sa_19(23).
  [12825][output.c][6653][PutConsoleMsg]:ANR0103E aftxn.c(1434):
  Error 115 updating row in table"AF.Vol.Clusters".
  
  [12831][bfrtrv.c][2216][bfGetRtrvOrder]:Requesting order for
  606836407.
  [12825][output.c][6653][PutConsoleMsg]:ANR1181 Eaftxn.c(309):
  Data storage transaction 0:13092681 was aborted.
  [11439][pvr.c][13455][AgentThread]:PVR I/O agent(11439) finished
  WRITE request; rc=0.
  
  [12831][bfaggrut.c][1074][bfGetSubBitfileInfo]:Owner of bfId
  0.606836407 is 0.0
  [12706][bfcreate.c][4858][AddAggrBitfile]:Adding a aggregate
  
  aggregate bitfile to the list. bfId 614678246, head is 11c2a4bd8
  tail is 11a961558.
  [12825][output.c][6653][PutConsoleMsg]:ANR2183Wafcputil.c(5020):
  Transaction 0:13092681 was aborted.
  
  Another symptom is that after an upgrade to V6.1.2.1, different
  processes such as the backup stgpool process may fail with
  the following errors in the activity log :
  
  ANR0984I Process nn for BACKUP STORAGE POOL
  ...
  ANR9999D_3193291044 AddCluCurVolId(aftxn.c:2081)
   Thread<1511>: Bad AFCL row, pool=-4, srvid=0, ck1=42,
   ck2=1, error=1.
  ANR9999D Thread<nn> issued message 9999 from:
  ANR9999D Thread<nn> 0x0000000000bf08a2 OutDiagToCons+0x0x142
  ANR9999D Thread<nn> 0x0000000000bf3424 outDiagfExt+0x0x194
  ANR9999D Thread<nn> 0x00000000006a36b9 AfPrepareTxn+0x0x1be9
  ANR9999D Thread<nn> 0x000000000060893c bfPrepareTxn+0x0x28c
  ANR9999D Thread<nn> 0x0000000000bc09a4 CollectVotes+0x0xc4
  ANR9999D Thread<nn> 0x0000000000bc0da4 tmEnd+0x0xb4
  ANR9999D Thread<nn> 0x000000000067391b CopyBatch+0x0x141b
  ANR9999D Thread<nn> 0x0000000000676059 AfCopyVolume+0x0x1389
  ANR9999D Thread<nn> 0x000000000066c56a AfBackupPoolThread+0x0xba
  ANR9999D Thread<nn> 0x0000000000c607eb StartThread+0x0xcb
  ANR9999D Thread<nn> 0x000000362c406367 *UNKNOWN*
  ANR9999D Thread<nn> 0x000000362b8d30ad *UNKNOWN*
  ANR0103E aftxn.c(1458): Error 87 updating row in table
   "AF.Vol.Clusters".
  ANR1181E aftxn.c(323): Data storage transaction 0:nnn
   was aborted.
  ANR2183W afcputil.c(5024): Transaction 0:nnn was
   aborted.
  ANR1280E BACKUP STGPOOL: Process nn terminated -
   transaction aborted.
  
  A server TXN trace will show :
  
  <timestamp> [1260][aftxn.c][1890][AddCluCurVolId]:
   colSize 240 is invalid
  
  Tivoli Storage Manager Versions Affected:  6.1.2.0
  
  Customer/L2 Diagnostics (If Applicable)
  
  Initial Impact: HIGH
  
  Additional Keywords: AF.Vol.Clusters,ERROR 115, ANR0103E,
  RECLAMATION, BACKUP STGPOOL
  

Local fix

• Not available.
  

Problem summary

• ****************************************************************
  * USERS AFFECTED: All Tivoli Storage Manager server users.     *
  ****************************************************************
  * PROBLEM DESCRIPTION: See ERROR DESCRIPTION.                  *
  ****************************************************************
  * RECOMMENDATION: Apply fixing level when available.           *
  *                 This problem is currently projected          *
  *                 to be fixed in level 6.1.3.                  *
  *                 Note that this is subject                    *
  *                 to change at the discretion of IBM.          *
  ****************************************************************
  *
  

Problem conclusion

• This problem was fixed.
  Affected platforms:
  AIX, HP-UX, Sun Solaris, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels

• R61A PSY

     UP

• R61H PSY

     UP

• R61L PSY

     UP

• R61S PSY

     UP

• R61W PSY

     UP