II06310: SVCDUMP / SDUMP / DUMPSRV PROCESSING HANG INFORMATION

APAR status

• Closed as canceled.

Error description

• ================================================================
  SVCDUMP / SDUMP / DUMPSRV HANG
  ================================================================
  A. Confirmation of an svcdump / sdump /dumpsrv hang.
  B. Documentation needed by IBM to diagnose.
  C. What might be done to relieve the hang.
  D. Recommended maintenance.
  
  ===============================================================
  A. WHAT INDICATES AN SVCDUMP PROBLEM?
  ================================================================
  For problems involving the sdump data capture phase:
  1. CVT+x'24C' (CVTSDBF) high order bit MUST be on (x'80......')
     CVT+x'23C'-->RTCT+x'9C' (RTCTSDPL) is non-zero
                  (in a few instances RTCTSDPL may be zero).
  2. No dumps have been received for a long time and
     any of the following have occurred:
   - If requesting a console dump and MSGIEE711I is received
     even though there is no other dump in progress.
   - If slip traps requesting ACTION=SVCD match and produce
     MSGIEA412I '1 sdumps not scheduled' even though there is
     no other dump in progress.
     (msgiea412i with rc08, rsn02 from SDUMP macro)
   - If issuing the SDUMP (SDUMPX) macro and return code 8 (rc8,
     rc08) is received even though no other dump is in progress.
   - If the SYSIEA01.SDUMPENQ resource is held exclusively
     over a very long time and not being released
     and a SYS1.DUMPxx dataset cannot be cleared because
     SDUMP is still serialized on it even though no other dump
     is in progress.
  
  It is also possible to have problems when SDUMP is not
  serialized as described above. These problems generally
  involve sdump related commands or the svcdump writing phase.
  Sdump related commands (eg. DISPLAY DUMP, DUMPDS) may
  not be processed and/or svcdumps may not be written
  to the dump data sets (no msgiea911e or msgiea611i issued
  even though msgiea794i for the dumpid has been issued.
  Or a DISPLAY DUMP may show captured dump dataspaces which
  are not being written.
  Other DUMPSRV held ENQ's may remain in conflict or outstanding,
  eg. SYSIEA01 DMPDSENQ or SDDSQ or SYSZTIOT.
  
  Another type of problem may be the DUMPSRV address space
  (or one of its tasks) has terminated and is not able
  to restart.
  ================================================================
  B. INITIAL DOCUMENTATION NEEDED FOR LEVEL2 DIAGNOSIS
  ================================================================
  1. Take a standalone dump if/when the system is to be re-ipl'd.
  2. Scan back in syslog to find the last time a dump was taken.
     This is indicated by MSGIEA794I and MSGIEA911E
     (or MSGIEA611I if dynamic allocation is used).
     Note the time and text for each message especially any
     SDRSN bits on msgiea911e/611i.
  .
   2a. You can check the SDUMP 4K buffer in the last dump.  If it
       hasn't been reused, it might have some information about
       what started to serialize SDUMP, but never got to issue
       the SDUMP(x) macro to take one.  The SDUMP 4K buffer is
       pointed to by CVTSDBF = CVT+x'24C'.
  .
  3. Run EREP against SYS1.LOGREC from this last dump time
     (or IPL if no message) to get software detail records.
     Scan for keywords such as component SC1CM or SCDMP or
     modules IGC0005A or IEAVTSD-- or IEAVAD00 or DUMPSRV's asid
     or anything for the jobname/asid(s) being dumped.
  4. Get the rmids for IEAVTSDT, IEAVTSDS, IEAVAD00,
                       IEAVTSDX, IEAVTSDR, IEAVTSRR,
                       and IEAVTDWT and IEECB926.
  5. If it is possible to look at active storage,
     RTM may be able to do some online diagnosis while
     DUMPSRV is hung prior to obtaining a sadump.
     Or if the above is not possible, Level 2 support
     support has a program available that may collect much
     of the initial information needed.
     The program name is 'DUMPLING', the object deck and sample
     jcl are available on data link.
     DUMPLING does not replace a sadump but will collect
     control block information as described below.
     Alternately the info. can be screen-printed using an
     oem monitor.
     Contact SDUMP level2 for any assistance with it.
  6. From the sadump or active storage/dumpling:
     start with the following basic RTM control blocks:
         RTCT (get address from CVT +23C) for x'1C0' bytes.
              RTCT+10C contains a table of the asids involved,
              2 bytes for each asid + 2 bytes flags.
         SDPL (get address from RTCT +9C) for x'80'  bytes.
         SDWK (get address from RTCT +DC) for x'160' bytes.
               SDWK/SDWORK/SDW1 contains timestamps at
               +C,+14, and +118 (at ESA4) which may help
               pinpoint the problem time.
               SDWORK+B8 contains 4 words of SDRSN bits
               which may help identify errors.
         RTSD (get address from RTCT+16C) for x'9C4' bytes.
               RTSD+640 contains the dump title which may
               help identify what was happening.
  7. Get the current TCB/RB status for each
     task under DUMPSRV and for the IEAVTSDT task (tcb#2)
     of any address space involved in the dump.
     This is especially important if SDUMP is not processing
     correctly but is not serialized.
  8. Check for any ENQ conflicts involving DUMPSRV.
     Get the ENQ names and status and the asid(s)+tcb@ of who
     owns and who is waiting.
  -
  ================================================================
  C. TO RELIEVE THE HANG
  ================================================================
  - Contact RTM level2 for a recommendation.
  - An IPL will certainly recover dump services.
  - Occasionally cancelling DUMPSRV or ZAP'ng SDUMP serialization
    will work but only if a specific APAR has been verified
    and this has been recommended by level2, caution is advised.
    Be sure that security system definintions permit the
    DUMPSRV started task to restart.
  - For other types of problems, use of the CALLRTM macro
    may provide recovery but again a recommendation by
    level2 and caution are advised.
  ================================================================
  D. RECOMMENDED MAINTENANCE, COMPID 5752SCDMP
     updated - 03/29/2004
  ===============================================================
  * The recommended PTF levels for the main modules:
  *  module    HBB6608 HBB7703 HBB7705 HBB7706 HBB7707 HBB7708
  * IEAVAD00 - UW89731 UW89732 UW89733 UW89734 UW89735
  * IEAVTSDX - UW82077 UW88477 UW88478 UW88479 UW88480
  * IEAVTSDS -
  * IEAVTSDT -
  * IEAVTSDR - UW70235 UA08245 UA08246 UA08247 UA08248 UA08270
  * IEAVTSRR - UW72357 UA09257 UA09258 UA09259 UA09260 UA09261
  * IEAVTDWT - UW80386 UW80387 UW80388
  * IEECB926 - UW80386 UW80387 UW80388
  * IEAVTSSM -         UW74528 UA05785 UA05786 UA05787 UA05788
  * IEAVTSAI -         UW85662 UW85663 UW85664
  * IEAVTSDB -         UA03657 UA03658 UA03659 UA03660
  *
  * See II06226 for SDUMP performance related apars.
  *
  ****************************************************************
  ===============================================================
  

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

• This APAR will assist customers in diagnosing and
  resolving SVCDUMP hang situations.
  JES2INFO VSAMINFO VTAMINFO DB2INFO
  Additional keywords: RC10 RC13 RD10 RD20 R422 R430 R510 R520
                       R522 OS/390 r601 r603
  

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

Applicable component levels