IBM Support

IC62065: SPACE RECLAMATION FAILS WITH ANR9999D_XXXXX GETCLUCURRVOLIDS() AFTER MIGRATION OF TSM 6.1 SERVER FROM 32BIT TO 64BIT

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • After an upgrade of a TSM 5.5 Server to TSM 6.1 (both 32bit) and
    later on a migration of this TSM 6.1 Server from 32bit to 64bit
    (same platform),
    the space reclamation fails with following errors in the ACTLOG:
    07/03/2009 09:43:27      ANR1040I Space reclamation started for
                             volume 123456L3,  storage pool
                             TEST_POOL (process number 1). (PROCESS:
                             1)
    [...]
    07/03/2009 09:43:27      ANR1176I Moving data for collocation
                             set 1 of 1 on volume 123456L3.
                             (PROCESS: 1)
    07/03/2009 09:43:28      ANR9999D_1306896927
                             GetCluCurrVolIds(afcreate.c:4126)
                             Thread<37>: Bad AFCL row, pool=21,
                             srvid=0, ck1=321, ck2=6, error=1.
                             (PROCESS: 1)
    07/03/2009 09:43:28      ANR1092E Space reclamation is ended for
                             volume 123456L3. An internal server
                             error was detected. (PROCESS: 1)
    07/03/2009 09:43:28      ANR0985I Process 1 for SPACE
                             RECLAMATION running in the BACKGROUND
                             completed with completion state FAILURE
                             at 09:43:28. (PROCESS: 1)
    .
    During the upgrade process to TSM 6.1 on the 32bit system, a
    column in the internal AF.Clusters table got initialized with a
    size of 24 bytes. When the TSM 6.1 Server has then been migrated
    to a 64bit system (on the same platform), the size of that
    column did not change though a 64bit TSM 6.1 Server expects this
    column to have a minimum size of 32 bytes,
    .
    In this specific case, the TSM 5.5 Server has been upgraded to
    TSM 6.1 both running on Windows 32bit and later on this TSM 6.1
    Server has been migrated to a Windows 64bit system.
    .
    Customer/L2 Diagnostics:
    A TSM Server trace with traceflags AF BF DS shows the following
    during space relamation:
    [...]
    09:43:27.587
    [35][bfspctrg.c][1745][CreateMarkTrgForCheck]:CreateMarkTrgForCh
    eck: Storage Pool has no trigger. No default stgpool space
    trigger. Not marking stgpool for check. Exiting
    CreateMarkTrgForCheck
    09:43:27.587
    [35][bfspctrg.c][890][BfCheckTrigger]:BfCheckTrigger: Exiting
    trigger check on poolid 21 poolName TEST_POOL stTrgP returned is
    0000000000000000
    09:43:27.619 [35][afcreate.c][4069][GetCluCurrVolIds]: colSize
    24 is invalid
    09:43:27.619
    [35][output.c][6653][PutConsoleMsg]:ANR9999D_1306896927
    GetCluCurrVolIds(afcreate.c:4126) Thread<35>: Bad AFCL row,
    pool=21, srvid=0, ck1=321, ck2=6, error=1.~
    09:43:27.619
    [35][afcreate.c][2033][AfAllocSpace]:GetIncludeVolList returned
    rc=87 for destPoolId=21, ck1=321, ck2=6.
    09:43:27.619 [35][afmove.c][7043][RelocateBitfile]:AfAllocSpace
    returned rc=87 for poolId=21, ck1=321, ck2=6, size=5.1320737397.
    09:43:27.619 [35][afmove.c][6142][MoveBatch]:RelocateBitfile
    returned rc=87 for bfId=0.274217301, ck1=321, ck2=6,
    size=5.1320737397.
    09:43:27.619 [35][afmove.c][5826][MoveCluster]:MoveBatch
    returned rc=87.
    09:43:27.619 [35][output.c][6653][PutConsoleMsg]:ANR1092E Space
    reclamation is ended for volume 123456L3. An internal server
    error was detected.~
    .
    The issue is that in the server table AF.Clusters TSM keeps
    track of current output volumes used by this particular
    NODE/FILESPACE.  This list of current output volumes is used as
    a hint during processing to determine the best/most likely next
    volume(s) to use for output for a new operation.
    The specific issue is that the data for this column is not
    correct.  It is expected to be of a minimum length of 32 bytes
    but in the cases where this fails it is only a length of 24
    bytes.  So, the error is raised and the operation fails because
    TSM was not able to use the existing current output volume hint
    that is supposed to be in place.
    .
    Similar trace message can be seen in a trace on a target server
    when running a backup db of the source server to a server
    deviceclass:
    [...]
    14:41:40.488 [33][aftxn.c][1283][PrepareClusters]:Adding volid
    27721 into cluster for stg pool 27.
    14:41:40.488 [33][aftxn.c][1746][AddCluCurVolId]:colValue is
    000000002748B0DC, colSize is 24 size of entry: 32
    14:41:40.488 [33][aftxn.c][1756][AddCluCurVolId]:colSize 24 is
    invalid
    14:41:40.488
    [33][output.c][6653][PutConsoleMsg]:ANR9999D_3193291044
    AddCluCurVolId(aftxn.c:1947) Thread<33>: Bad AFCL row, pool=27,
    srvid=0, ck1=17, ck2=1073741825, error=1.
    14:41:40.488 [33][aftxn.c][1954][AddCluCurVolId]:Exit, rc 87 for
    volId 27721
    14:41:40.488 [33][output.c][6653][PutConsoleMsg]:ANR0103E
    aftxn.c(1423): Error 87 updating row in table
    "AF.Vol.Clusters".~
    14:41:40.488 [33][output.c][6653][PutConsoleMsg]:ANR1181E
    aftxn.c(306): Data storage transaction 0:140420 was aborted.~
    14:41:40.488 [33][output.c][6653][PutConsoleMsg]:ANR0532W
    smnode.c(3357): Transaction 0:140420 was aborted for session 60
    for node TEST-NODE (Windows).~
    14:41:40.535 [33][output.c][6653][PutConsoleMsg]:ANR0514I
    Session 60 closed volume 456789L3.~
    .
    .
    TSM Versions Affected:
    TSM 6.1 Server
    .
    .
    Initial Impact:
    High
    .
    .
    Additional Keywords:
    upgrade dsmupgrd dsmupgrade bit bits 32 64 migrate reclaim
    afcreate.c aftxn.c afmove.c GetCluCurrVolIds AddCluCurVolId
    AF.Clusters  AF.Vol.Clusters ANR1092E ANR1093E intListT
    

Local fix

  • In case tapes are needed, the MOVE DATA command can be used to
    empty (some of) the tapes.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED: Tivoli Storage Manager V6.1 servers that     *
    *                 have been upgraded from V5.x and have        *
    *                 been changed to run using the                *
    *                 64-bit version of the product while          *
    *                 having been upgraded using                   *
    *                 the 32-bit version.                          *
    ****************************************************************
    * PROBLEM DESCRIPTION: See ERROR DESCRIPTION.                  *
    ****************************************************************
    * RECOMMENDATION: Apply fixing level when available. This      *
    *                 problem is currently projected to be fixed   *
    *                 in levels 6.1.2.1 and 6.1.3.0. Note that     *
    *                 this is subject to change at the             *
    *                 discretion of IBM.                           *
    ****************************************************************
    *
    

Problem conclusion

  • This problem was fixed.
    
    Affected platforms:  AIX, HP-UX, Sun Solaris,
    Linux, and Windows.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IC62065

  • Reported component name

    TSM SERVER

  • Reported component ID

    5698ISMSV

  • Reported release

    61W

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2009-07-18

  • Closed date

    2009-09-14

  • Last modified date

    2009-09-14

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    TSM SERVER

  • Fixed component ID

    5698ISMSV

Applicable component levels

  • R61A PSY

       UP

  • R61H PSY

       UP

  • R61L PSY

       UP

  • R61S PSY

       UP

  • R61W PSY

       UP

[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSGSG7","label":"Tivoli Storage Manager"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"61W","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]

Document Information

Modified date:
14 September 2009