IBM Support

IT05084: TIVOLI STORAGE MANAGER SERVER REPORTS ANR9999D_0376207520 WHEN RUNNING NODE REPLICATION.

Direct links to fixes

6.3.6.000-TIV-TSMRPT-AGENT-Linux
6.3.6.000-TIV-TSMRPT-AGENT-AIX
6.3.6.000-TIV-TSMRPT-AIX
6.3.6.000-TIV-TSMRPT-AGENT-Windows
6.3.6.000-TIV-TSMRPT-Linuxx86
6.3.6.000-TIV-TSMRPT-Linuxx86_64
6.3.6.000-TIV-TSMRPT-WindowsI32
6.3.6.000-TIV-TSMRPT-WindowsX64
6.3.6.000-TIV-TSMAC-WindowsX64
6.3.6.000-TIV-TSMAC-WindowsI32
6.3.6.000-TIV-TSMAC-SolarisSPARC
6.3.6.000-TIV-TSMAC-Linuxx86
6.3.6.000-TIV-TSMAC-Linuxs390x
6.3.6.000-TIV-TSMAC-AIX
6.3.6.000-TIV-TSMALL-SolarisSPARC
6.3.6.000-TIV-TSMALL-WindowsX64
6.3.6.000-TIV-TSMSTA-WindowsI32
6.3.6.000-TIV-TSMALL-HP-UX
6.3.6.000-TIV-TSMALL-Linuxppc64
6.3.6.000-TIV-TSMALL-AIX
6.3.6.000-TIV-TSMALL-Linuxs390x
6.3.6.000-TIV-TSMALL-Linuxx86_64
7.1.3.000-TIV-TSMSRV-WIN
7.1.3.000-TIV-TSMSRV-SolarisSPARC
7.1.3.000-TIV-TSMSRV-Linuxx86_64
7.1.3.000-TIV-TSMSRV-Linuxs390x
7.1.3.000-TIV-TSMSRV-Linuxppc64
7.1.3.000-TIV-TSMSRV-HP-UX
7.1.3.000-TIV-TSMSRV-AIX
IBM Tivoli Storage Manager V6.3 Fix Pack 6 (6.3.6.000) Server Downloads

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • When node replication is run between the source and target
servers, client data will be sent but may end unexpectedly with
the following messages seen in the target server actlog.

10/06/14   17:07:29   ANR9999D_0376207520
SmEncryptKey(smutil.c:20167) Thread<3282>: Error decrypting key.
Magic not 0xA5A5A5A5 (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282> issued message 9999
from: (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000ec7a43
OutDiagToCons  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000ecaa31
outDiagfExt  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000cffc17
SmEncryptKey  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000c64dd2
SmParseBackInsNormE  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000c6a0e8
SmDoBackInsNormEnhanced  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000d7a8a8
SmReplServerSession  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000c4bbdf
DoReplServer  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000c4f1f0
smExecuteSession  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000f7f067
psSessionThread  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000f6addf
StartThread  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x007f91c896d9d1
*UNKNOWN*  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x007f91c545986d
*UNKNOWN*  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D_2077889187
SmReplServerSession(smrepl.c:1695) Thread<3282>: Replication
protocol error, unknown verbType=0400 (SESSION: 429, PROCESS:
12)
10/06/14   17:07:29   ANR9999D Thread<3282> issued message 9999
from: (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000ec7a43
OutDiagToCons  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000ecaa31
outDiagfExt  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000d7847a
SmReplServerSession  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000c4bbdf
DoReplServer  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000c4f1f0
smExecuteSession  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000f7f067
psSessionThread  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x00000000f6addf
StartThread  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x007f91c896d9d1
*UNKNOWN*  (SESSION: 429, PROCESS: 12)
10/06/14   17:07:29   ANR9999D Thread<3282>  0x007f91c545986d
*UNKNOWN*  (SESSION: 429, PROCESS: 12)


Customer/L2 Diagnostics:
A trace can be run on the target server during the replication
time using the flags addmsg and repl.

The messages found in the trace file will be very similar to the
stack in the actlog but will also show that the 9999 error was
returned from Sm Insert Verb.

17:07:30.621
[3284][output.c][7474][PutConsoleMsg]:ANR9999D_0376207520
SmEncryptKey(smutil.c:20167) Thread<3284>: Error decrypting key.
Magic not 0xA5A5A5A5~
17:07:30.621 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284> issued message 9999 from:~
17:07:30.621 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000ec7a43 OutDiagToCons ~
17:07:30.621 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000ecaa31 outDiagfExt ~
17:07:30.621 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000cffc17 SmEncryptKey ~
17:07:30.621 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000c64dd2 SmParseBackInsNormE ~
17:07:30.621 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000c6a0e8 SmDoBackInsNormEnhanced ~
17:07:30.622 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000d7a8a8 SmReplServerSession ~
17:07:30.622 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000c4bbdf DoReplServer ~
17:07:30.622 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000c4f1f0 smExecuteSession ~
17:07:30.622 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000f7f067 psSessionThread ~
17:07:30.622 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000f6addf StartThread ~
17:07:30.622 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x007f91c896d9d1 *UNKNOWN* ~
17:07:30.622 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x007f91c545986d *UNKNOWN* ~
17:07:30.622 [3284][smrepl.c][13587][TranslateToGRC]:abort
reason code = 1
17:07:30.623 [3284][smrepl.c][13681][TranslateToGRC]:rc=0,
*objRcP=9999, confirmState=NO
17:07:30.623 [3284][smrepl.c][1620][SmReplServerSession]:Error
9999 from Sm Insert Verb
17:07:30.623
[3284][smrepl.c][1098][SmReplServerSession]:Received verb
StreamMarker (400) on session 430
17:07:30.623
[3284][output.c][7474][PutConsoleMsg]:ANR9999D_2077889187
SmReplServerSession(smrepl.c:1695) Thread<3284>: Replication
protocol error, unknown verbType=0400~
17:07:30.623 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284> issued message 9999 from:~
17:07:30.623 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000ec7a43 OutDiagToCons ~
17:07:30.623 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000ecaa31 outDiagfExt ~
17:07:30.623 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000d7847a SmReplServerSession ~
17:07:30.624 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000c4bbdf DoReplServer ~
17:07:30.624 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000c4f1f0 smExecuteSession ~
17:07:30.624 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000f7f067 psSessionThread ~
17:07:30.624 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x00000000f6addf StartThread ~
17:07:30.624 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x007f91c896d9d1 *UNKNOWN* ~
17:07:30.624 [3284][output.c][7474][PutConsoleMsg]:ANR9999D
Thread<3284>  0x007f91c545986d *UNKNOWN* ~
17:07:30.721 [3284][output.c][7474][PutConsoleMsg]:ANR0409I
Session 430 ended for server SERVER1 (Windows).~


Tivoli Storage Manager Versions Affected:
7.1, 6.3


Initial Impact: (high/med/low)
Med

Additional Keywords:
node, replication, replicate, fail, abend, tsm

Local fix

  • The define server command on the target replication sever should
 be run with a password in all caps to ensure a successful
literal match of the source server password. The source sever
password is used in part of the server to server communications
during node replication and is passed to the target server in
all caps which may cause this failure to occur on the target
server.

Problem summary

  • ****************************************************************
* USERS AFFECTED:                                              *
* All Tivoli Storage Manager server users.                     *
****************************************************************
* PROBLEM DESCRIPTION:                                         *
* See error description.                                       *
****************************************************************
* RECOMMENDATION:                                              *
* Apply fixing level when available. This problem is currently *
* projected to be fixed in levels 6.3.6 and 7.1.3. Note that   *
* this is subject to change at the discretion of IBM.          *
****************************************************************

Problem conclusion

  • This problem was fixed.
Affected platforms:  AIX, HP-UX, Solaris, Linux, and Windows.

Temporary fix

  • 



Comments


    

  • 



APAR Information




    

  • APAR number
    IT05084
    • 

  • Reported component name
    TSM SERVER
    • 

  • Reported component ID
    5698ISMSV
    • 

  • Reported release
    71W
    • 

  • Status
    CLOSED  PER
    • 

  • PE
    NoPE
    • 

  • HIPER
    NoHIPER
    • 

  • Special Attention
    NoSpecatt / Xsystem
    • 

  • Submitted date
    2014-10-23
    • 

  • Closed date
    2014-12-14
    • 

  • Last modified date
    2015-01-29
    • 





    

  • APAR is sysrouted FROM one or more of the following:
    

    • 

  • APAR is sysrouted TO one or more of the following:
    

    • 



Fix information


    

  • Fixed component name
    TSM SERVER
    • 

  • Fixed component ID
    5698ISMSV
    • 



Applicable component levels


    

  • R63A  PSY
       UP
    • 

  • R63H  PSY
       UP
    • 

  • R63L  PSY
       UP
    • 

  • R63S  PSY
       UP
    • 

  • R63W  PSY
       UP
    • 

  • R71A  PSY
       UP
    • 

  • R71H  PSY
       UP
    • 

  • R71L  PSY
       UP
    • 

  • R71S  PSY
       UP
    • 

  • R71W  PSY
       UP
    • 








      
              
                            

              

              [{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSGSG7","label":"Tivoli Storage Manager"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"7.1.3","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]
              

                          

          
 
        

      

    

      

        

          

            
Document Information


            

            


                        

            Modified date:
            

            29 January 2015
            

            
                      

        


        

        


      

    

  



    

  

  
    
            

          
Page Feedback