Fixes are available
APAR status
Closed as program error.
Error description
After an upgrade of a TSM 5.5 Server to TSM 6.1 (both 32bit) and later on a migration of this TSM 6.1 Server from 32bit to 64bit (same platform), the space reclamation fails with following errors in the ACTLOG: 07/03/2009 09:43:27 ANR1040I Space reclamation started for volume 123456L3, storage pool TEST_POOL (process number 1). (PROCESS: 1) [...] 07/03/2009 09:43:27 ANR1176I Moving data for collocation set 1 of 1 on volume 123456L3. (PROCESS: 1) 07/03/2009 09:43:28 ANR9999D_1306896927 GetCluCurrVolIds(afcreate.c:4126) Thread<37>: Bad AFCL row, pool=21, srvid=0, ck1=321, ck2=6, error=1. (PROCESS: 1) 07/03/2009 09:43:28 ANR1092E Space reclamation is ended for volume 123456L3. An internal server error was detected. (PROCESS: 1) 07/03/2009 09:43:28 ANR0985I Process 1 for SPACE RECLAMATION running in the BACKGROUND completed with completion state FAILURE at 09:43:28. (PROCESS: 1) . During the upgrade process to TSM 6.1 on the 32bit system, a column in the internal AF.Clusters table got initialized with a size of 24 bytes. When the TSM 6.1 Server has then been migrated to a 64bit system (on the same platform), the size of that column did not change though a 64bit TSM 6.1 Server expects this column to have a minimum size of 32 bytes, . In this specific case, the TSM 5.5 Server has been upgraded to TSM 6.1 both running on Windows 32bit and later on this TSM 6.1 Server has been migrated to a Windows 64bit system. . Customer/L2 Diagnostics: A TSM Server trace with traceflags AF BF DS shows the following during space relamation: [...] 09:43:27.587 [35][bfspctrg.c][1745][CreateMarkTrgForCheck]:CreateMarkTrgForCh eck: Storage Pool has no trigger. No default stgpool space trigger. Not marking stgpool for check. Exiting CreateMarkTrgForCheck 09:43:27.587 [35][bfspctrg.c][890][BfCheckTrigger]:BfCheckTrigger: Exiting trigger check on poolid 21 poolName TEST_POOL stTrgP returned is 0000000000000000 09:43:27.619 [35][afcreate.c][4069][GetCluCurrVolIds]: colSize 24 is invalid 09:43:27.619 [35][output.c][6653][PutConsoleMsg]:ANR9999D_1306896927 GetCluCurrVolIds(afcreate.c:4126) Thread<35>: Bad AFCL row, pool=21, srvid=0, ck1=321, ck2=6, error=1.~ 09:43:27.619 [35][afcreate.c][2033][AfAllocSpace]:GetIncludeVolList returned rc=87 for destPoolId=21, ck1=321, ck2=6. 09:43:27.619 [35][afmove.c][7043][RelocateBitfile]:AfAllocSpace returned rc=87 for poolId=21, ck1=321, ck2=6, size=5.1320737397. 09:43:27.619 [35][afmove.c][6142][MoveBatch]:RelocateBitfile returned rc=87 for bfId=0.274217301, ck1=321, ck2=6, size=5.1320737397. 09:43:27.619 [35][afmove.c][5826][MoveCluster]:MoveBatch returned rc=87. 09:43:27.619 [35][output.c][6653][PutConsoleMsg]:ANR1092E Space reclamation is ended for volume 123456L3. An internal server error was detected.~ . The issue is that in the server table AF.Clusters TSM keeps track of current output volumes used by this particular NODE/FILESPACE. This list of current output volumes is used as a hint during processing to determine the best/most likely next volume(s) to use for output for a new operation. The specific issue is that the data for this column is not correct. It is expected to be of a minimum length of 32 bytes but in the cases where this fails it is only a length of 24 bytes. So, the error is raised and the operation fails because TSM was not able to use the existing current output volume hint that is supposed to be in place. . Similar trace message can be seen in a trace on a target server when running a backup db of the source server to a server deviceclass: [...] 14:41:40.488 [33][aftxn.c][1283][PrepareClusters]:Adding volid 27721 into cluster for stg pool 27. 14:41:40.488 [33][aftxn.c][1746][AddCluCurVolId]:colValue is 000000002748B0DC, colSize is 24 size of entry: 32 14:41:40.488 [33][aftxn.c][1756][AddCluCurVolId]:colSize 24 is invalid 14:41:40.488 [33][output.c][6653][PutConsoleMsg]:ANR9999D_3193291044 AddCluCurVolId(aftxn.c:1947) Thread<33>: Bad AFCL row, pool=27, srvid=0, ck1=17, ck2=1073741825, error=1. 14:41:40.488 [33][aftxn.c][1954][AddCluCurVolId]:Exit, rc 87 for volId 27721 14:41:40.488 [33][output.c][6653][PutConsoleMsg]:ANR0103E aftxn.c(1423): Error 87 updating row in table "AF.Vol.Clusters".~ 14:41:40.488 [33][output.c][6653][PutConsoleMsg]:ANR1181E aftxn.c(306): Data storage transaction 0:140420 was aborted.~ 14:41:40.488 [33][output.c][6653][PutConsoleMsg]:ANR0532W smnode.c(3357): Transaction 0:140420 was aborted for session 60 for node TEST-NODE (Windows).~ 14:41:40.535 [33][output.c][6653][PutConsoleMsg]:ANR0514I Session 60 closed volume 456789L3.~ . . TSM Versions Affected: TSM 6.1 Server . . Initial Impact: High . . Additional Keywords: upgrade dsmupgrd dsmupgrade bit bits 32 64 migrate reclaim afcreate.c aftxn.c afmove.c GetCluCurrVolIds AddCluCurVolId AF.Clusters AF.Vol.Clusters ANR1092E ANR1093E intListT
Local fix
In case tapes are needed, the MOVE DATA command can be used to empty (some of) the tapes.
Problem summary
**************************************************************** * USERS AFFECTED: Tivoli Storage Manager V6.1 servers that * * have been upgraded from V5.x and have * * been changed to run using the * * 64-bit version of the product while * * having been upgraded using * * the 32-bit version. * **************************************************************** * PROBLEM DESCRIPTION: See ERROR DESCRIPTION. * **************************************************************** * RECOMMENDATION: Apply fixing level when available. This * * problem is currently projected to be fixed * * in levels 6.1.2.1 and 6.1.3.0. Note that * * this is subject to change at the * * discretion of IBM. * **************************************************************** *
Problem conclusion
This problem was fixed. Affected platforms: AIX, HP-UX, Sun Solaris, Linux, and Windows.
Temporary fix
Comments
APAR Information
APAR number
IC62065
Reported component name
TSM SERVER
Reported component ID
5698ISMSV
Reported release
61W
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2009-07-18
Closed date
2009-09-14
Last modified date
2009-09-14
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
TSM SERVER
Fixed component ID
5698ISMSV
Applicable component levels
R61A PSY
UP
R61H PSY
UP
R61L PSY
UP
R61S PSY
UP
R61W PSY
UP
[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSGSG7","label":"Tivoli Storage Manager"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"61W","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]
Document Information
Modified date:
14 September 2009