IBM Support

IT14146: REPLICATED DATA FAILS TO RESTORE WITH ANS1028S/ANS0361I OR ANS4020E.

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • Restore of replicated data from the target server fails with
    ANS1028S An internal program error occurred. In the error log
    prior to this message an ANS0361I DIAG: fileio.cpp (3173):
    fioWrite(): Unrecoverable error: NNNNNN Bytes left in receive
    buffer with a BLOCK_EOF sdb type can be found. Or in other cases
    the restore of replicated data from the target server fails with
    ANS4020E The data format for object '<failing path and file
    name>' is unknown. The unknown format usually occurs when the
    file was backed up or archived by a later version of IBM Tivoli
    Storage Manager.
    Restore of the original source objects from the replicating
    server complete without error
    For the ANS1028S/ANS0361I situation a service trace of the
    failing restore is needed to determine the problem object and
    it's object id. Searching the trace for the "BLOCK_EOF" entry
    will identify the failing object name. Searching the trace for
    the versid that follows will provide the object id. A show invo
    using this object id on the target server will show that the
    bitfile size of the object in bitfile object entry the is larger
    than expected (larger than the size in the inventory object
    entry):
    tsm: ADSM1>show invo 12345678
    Session established with server TSM2: AIX
      Server Version 7, Release 1, Level 3.0
      Server date/time: 03/01/16   15:18:54  Last access: 03/01/16
    07:38:02
    Inventory object 12345678 of copy type Backup has attributes:
      NodeName: MYNODE, Filespace(1): /fs1,
      ObjName: /dir1/dir2/dir3/dir4/file.
      hlID: 5BE345DE3087385770A97C40BF3901E10F4F1E33
      llID: CE70A8855B26694305BCAC218573BFBA4E7538D5
      Type: 2 (File)  MC: 1 (DEFAULT) CG: 1  Size: 1073741824
    HeaderSize: 338
      Inactive, Inserted 02/24/16 01:07:40 (UTC 02/24/16 07:07:40),
    Deactivated 02/25/16 01:08:52
      GroupMap   00000000, bypassRecogToken NULL, flags 0000
    Replicas:
      Server: TSM1, ObjId: 98765432
    Bitfile Object: 12345678
      Inactive
    **Archival Bitfile Entry
      Bitfile Type: PRIMARY  Storage Format: 22
      Bitfile Size: 1127864103  Number of Segments: 1, flags: 0
      Storage Pool ID: 17  Volume ID: 57681  Volume Name: VOL000L6
    The replica id in the show invo output for the problem object on
    the target can be used on the source to get the show invo output
    for the original object and the bitfile size will be a smaller
    correct size.
    The problem with the replicated object is being created at
    replication time. A trace of the replication on the source
    server shows that there is a preemption of the replication of
    the object occurring. In the reporting case the object was
    originally being replicated from an active data pool. During the
    replication processing from this pool a preemption occurred, rc
    3008 (GRC_SEGMENT_PREEMPTED). The replication continues
    processing the object using another pool as the source but
    begins from the start of the object. This results in the
    replicated object on the target being larger (having more data)
    than it should. And subsequent restores fail.
    A BFRTRV trace of the replication introducing the problem will
    show the rc 3008 for one device class and then the attempt to
    retrieve the bitfile from a different device class:
    02:08:24.767 [381473][bfrtrv.c][4474][RtrvOne]:afRtrv returned
    3008 for FILE ADP.
    02:08:24.768 [381473][bfrtrv.c][4505][RtrvOne]:Attempting
    retrieval of bitfile 98765432 from DISK.
    With this level of preemption there is no external messaging
    that it has occurred.
    Tivoli Storage Manager Versions Affected: 6.3.5 and 7.1.x.x on
    all platforms
    Initial Impact: High
    Additional Keywords: TSM IBM Spectrum Protect
    | MDVREGR 6.3.5.0-TIV_5698MSV |
    

Local fix

  • To avoid the problem use the NOPREEMPT server option or
    determine source of the preemption and configure the server to
    keep that from occurring.
    To correct any problem objects once replicated with this issue:
    - Backup the problem object(s) on the source and re-replicate
    them to the target.
    Or
    - Delete the problem objects from the target replication sever
    and re-replicate the node.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * All Tivoli Storage Manager server users.                     *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See Error Description.                                       *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Apply fixing level when available.  This problem is          *
    * currently projected to be fixed in levels 6.3.6, 7.1.5.100,  *
    * and 7.1.6.  Note that this is subject to change at the       *
    * discretion of IBM.                                           *
    ****************************************************************
    

Problem conclusion

  • The problem was fixed.
    Affected Platforms: AIX, HP-UX, Solaris, Linux, and Windows.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT14146

  • Reported component name

    TSM SERVER

  • Reported component ID

    5698ISMSV

  • Reported release

    71A

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2016-03-04

  • Closed date

    2016-03-22

  • Last modified date

    2016-03-22

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    TSM SERVER

  • Fixed component ID

    5698ISMSV

Applicable component levels

  • R63A PSY

       UP

  • R63H PSY

       UP

  • R63L PSY

       UP

  • R63S PSY

       UP

  • R63W PSY

       UP

  • R71A PSY

       UP

  • R71H PSY

       UP

  • R71L PSY

       UP

  • R71S PSY

       UP

  • R71W PSY

       UP



Document information

More support for: Tivoli Storage Manager

Software version: 7.1.3

Reference #: IT14146

Modified date: 22 March 2016