APAR status
Closed as canceled.
Error description
(last update 01/14/2000 jbw) The purpose of this INFOAPAR is to document the known DB2 (5740XYR00 R310 R410 R510 R610) storage problems. Most problems regarding DB2 and Virtual Storage usage will be owned by the ADMF address space and will deal with the use of storage in the High Private Area (LSQA|SWA|229|230|249). What is LSQA|SWA? LSQA = sp203 sp204 sp205 sp213 sp214 sp215 sp223 sp224 sp225 sp233 sp234 sp235 sp253 sp254 sp255 SWA = sp236 sp237 This area is part of the MVS Private Area but NOT part of the Private REGION. Items like IEALIMIT, IEFUSI and REGION=x are things that control the use of 'Private REGION' storage. The DB2 MSTR and DBM1 start PROC should be set for REGION=0K. DB2 control blocks, bufferpools, RDS and EDM pools use SUBPOOL 229 storage. It is NOT uncommon to see a DB2 subsystem with 500m+ of SP229 in use. Be aware that storage for this High Private Area is allocated from the 2gig line DOWN toward the highest 'ALLOCATED' byte of the user Private REGION below it. DB2 uses this particular subpool 229, because SP229 is the only MVS virtual pool that has the attributes of PAGEABLE, PRIVATE HIGH and FETCH PROTECTION. Also note that DBM1 and DDF asids tends to use blocks of Common ECSA storage mainly in SP231 and SP241 (SRBs in sp228). If storage problems are found, first check to make sure bufferpool allocations are reasonable for the workload. Large subsystems are found to have bufferpool allocation from 500 to 800 meg. Tuning issue. Next, check DB2 reason codes associated with storage problems for known fixes within this info apar. The DB2 group also recommends that user move Scheduler Work Area control blocks to the HIGH private area by setting JES2PARM STCCLASS SWA=ABOVE . By doing so, user will avoid LOW private usage errors like ABEND106 rc0C or rc28 when DB2 functions fetch and load IDCAMS modules for various functions. The storage error may be noted by MVS Contents Supervisor messages MSGCSV031I and MSGCSV028I pointing to AMS and DBM1. IDCAMS modules like IDCAL01 and IDCDE01 always reside in LOW user private REGION. Excessive SWA control blocks constructed to process DB2 data sets can use most all of the LOW private area plus eat into the unallocated private REGION storage. If DSMAX (SPRM+76) is set to 10000 max then SWA must be set ABOVE. (See DFSMS infoapar II07347 for related information). Any DB2 user who feels that he is experiencing inordinate DB2 storage usage should check, then apply ALL of the following DB2 APARs applicable to his DB2 release. Most of the problems noted may manifest in various forms of abends. Most common abends would tend to be ABEND878 or ABEND80A with rc10 for Private failures and rc08 for Common storage failures. DB2 errs may tend to be SMC related with a form of rc00E200xx. Some storage errors experienced during Must-Complete windows may cause the DB2 subsystem to ABTERM (DSNV086E). In general, results tend to be unpredictable when a major system component or application like DB2 begins to experience storage related concerns. Excessive CPU cycles, overall slow performance, excessive paging to ESTOR or DASD/AUX (msgIRA200E), agent threads in long waits, timeouts. The error outcome may depend on the type of GETMAIN issued and the subpool requested. The loss of COMMON storage may have an adverse effect on all jobs in the MVS system, and usually requires an IPL to reclaim. See II06335 and pn80921 for related DB2 SVCDUMP information. *************************************************************** APAR RELEASE SUBPOOL/COMMENTS NUMBER R230 R310 --------------------------------------------------------------- PN52302 UN66452 UN66454 SP229 KEY7 Hybrid Join (pn53880) PN55356 UN60475 SP231 CSA EXCESSIVE STORAGE USE PN58827 UN66412 UN66413 SP231 key7 'DPSB BLOCK POOL' PN59301 UN66105 UN66106 SP229 key7 STORAGE GROWTH PN60402 UN66107 UN66108 SP229 STORAGE USE, RDS OP POOL PN64674 UN72121 SP229 KEY7 Move SORTWORK to RDS PN65081 UN71125 SP231 K7 STORAGE FILLS WITH QHD PN67602 UN74118 SP241 ECSA DIST loop IDAHEB *************************************************************** APAR RELEASE NUMBER R230 R310 R410 SUBPOOL/COMMENTS ---------------- (start put9504) ------------------------------ PN64639 UN75945 UN75946 SP229 RC00E20004 PN66441 UN77126 UN77127 UN77128 DB2 DUW CHAIN CORRUPTION PN69247 UN75410 SP229 ADMF STORAGE PN69632 UN75484 SP229 MSTR STORAGE PN70199 UN77845 UN77846 SP230 / SP231 RQA and DEL BLOCK PN70615 UN76793 SP229 AGENT LOCAL PN74387 UN78620 SP229 SQLCODE518 RDS OP POOL PN74773 UN82964 UN82965 SP229 STORAGE OVERLAY PN75052 UN81170 UN81171 SP229 SLOW GROWTH DBM1 PN75361 UN82497 SP241 ECSA DIST asid StoredProc PN77310 UN84707 UN84708 SP231 ECSA ROB storage block PN76737 UN84757 SP229 key7 SRTBLOCK (PN64674) PN77550 UN84833 UN84834 Deferred writes not scheduled -----------------(start put9601)------------------------------- PN77597 UN86067 UN86068 SP229 ADMF RID BLOCKS PN78376 UN86239 UN86240 UN86241 SP229 UPDATE/DELETE RC00E20004 PN78445 UN84756 DSNRUA01 ABEND878 RC10 S0C4 PN79379 UN86236 SP229 DBNM BLOCKS PN75647 UN86746 UN86747 UN86748 SP229 KUC ADMF BCT display cmds PN79151 UN86203 UN86204 SP0 key8 LOGP XLUB or LLUB PN79699 UN86636 SP231 key7 ECSA DIST/DDF asid PN80624 UN88524 SP229 rc00E20004 using UPROCS PN82149 UN88525 SP241 ECSA ADMF GLOBAL CL1 PN85618 UN92466 UN92467 SP229 EDMPOOL overlay PN84316 UN92913 UN92914 ESTOR HIPERPOOL usage PN86383 UN93909 SP241 Lock Mgr Global (TRWA) PN87278 UN94568 SP229 orphan DBET entry (DS) PN86560 UN93062 UN93063 Storage overlay after shortage PN88735 UN95260 UN95261 SP229 fragmented DM/BM POOL PN90919 UN98160 SP241 Utility leaves 'REE' blks PN91609 UN99424 SP229 E20003 DM/BM TRACE POOL -----------------(start put9702)------------------------------- PQ00396 UQ00854 UQ00855 SP229 excessive RID blocks PN92724 UQ02167 UQ02168 SP229 RDS err rc00E20003 PQ04241 UQ05335 SP229-7 R310 for PQ02872 PQ02872 UQ04176 SP229 inactive DDF threads PQ03272 UQ04422 SP229 ADMF AGENT LOCAL POOL PQ04022 UQ05978 UQ05979 sp229 large EDMpool >400meg PQ05990 UQ07107 SP231 ECSA 4k D20x blocks, DDF PQ06377 UQ07393 SP229 same as 2 above (+ZPARM) PQ05118 UQ07234 DB2 SMC Storage Diagnostics PQ07002 UQ08264 SP241 ECSA ADMF Global CL1 PQ00413 UQ08327 SP229 CP Parallelism PQ06147 UQ08886 SP229 SKB blocks from IFI reads PQ08399 UQ09887 sp231 rc00E20022 ECSA DDF agent PQ07877 UQ10316 SP229 CP PARALLELISM DVS PQ08512 UQ11917 sp229 SKB STACK storage (LOOP) PQ08763 UQ10641 sp229 key7 in the DBM1 asid PQ07376 UQ12226 sp241 ECSA Lock Mgr Global TRWA *****************(start put9801)******************************* PQ12552 UQ13956 sp0 MSTR Log Mgr Input buffers PQ13085 UQ14811 sp241 ECSA TRWA PQ11213 UQ15790 UQ15791 sp229 SKB DBM1 loop PQ12983 UQ16347 sp231 IFI monitor rc00E20004 PQ14484 UQ17284 UQ17285 sp229 fragmentation CUBs PQ14498 UQ17947 SP229 PBA BLOCKS PQ06102 UQ21143 sp241 IRLM SKPT Plocks PQ14132 UQ18603 sp229 storage abends, creep PQ10352 UQ19291 sp229 ADMF AGENT LOCAL, TPL blk PQ18595 UQ21631 sp229 SKB storage rc00E20016 PQ18432 UQ22858 sp229 stack storage SKB PQ20334 UQ22924 sp229 DBM1 contraction *****************(start put9901)******************************* PQ24904 UQ28184 UQ28185 sp229 EDMPOOL corruption PQ24852 UQ30325 sp229 bufferpool spills to low PQ29375 UQ33696 sp229 join blocks AGENT Local PQ30396 UQ34844 sp229 SPA blocks overlay PQ30582 UQ36196 sp229 create DB rc00E20004 PQ30620 UQ37251 sp229 CUB AGENT Local PQ33557 uq38986 sp229 3-part name neg SQLCODE --------------------------------------------------------------- (NOTE: See info apar II10817 for DB2 R510 R610 fix list) *********** NON-DB2 ******************************************* 02/06/97 RL SP229 key7 increasing with SKB blocks which are x'A000' bytes in length. There are only a few bytes in use and the blocks appear to be referencing BMC utilities. BMCUTIL IADM BMCAMU BMCARU are the OEM BMC eyecatchers in most allocation. The BMC fixes are all dated May 13, 1996. Load Plus utility releases 2.1.00 & 2.1.01 fix is P285822 Check Plus utility release 1.4.00 fix is P289625 Recover Plus utility releases 2.2.00 and 2.2.01 fix is P289872 Reorg Plus utility release 4.2.01 fix is P289622 --------------------------------------------------------------- 460K ESCA block allocated by PTXMAN (the Detector product) The top of the block contains characters: "CPOOL CELL POOL ITH$ENTY" Contact PLATINUM for fix. --------------------------------------------------------------- Excessive sp229 key7 DBM1 storage fragmentation. Authorization control blocks KPV KCL MSIB ARULA02 left in x'1A000' byte cell. CA/TopSecret users with CAsesm84 module replacing DB2 code by replacing addr. of DSNXA02 in RMFT+2DC with branch address to the OEM logic. Fix is LO17262+(T7M8443 & TE49132) TSS CaseJob x'20000' byte of SHBs. x'1FAE0' byte free. KPV MSIB KSL KCL CA fix L074236. rc00e20004 00e20004 ljy 11/12/2001 added the following on 02/10/04 rjl condition similar to above with control block names MSIB KSL KCL KPV SQLCA SQLDA CLSDBNS2. CA provide a fix from 02/02/02 for CAIENF/DB2 or ENF/DB2. Fix number is QO08502. Contact CA support for details. This was running on os/390 2.10 HBB7703 =============================================================== 11/03/00 (RPS) rc00e20003 showing 600Meg of DB2 MSIB blocks, these blocks were never being released, thus causing excessive storage usage. Found that CA-90 ACF2 was not freeing these blocks. CA fix for this is L074236 ================================================================ OY02574 JES2 support SWA above 16meg II05148 - - SP255 LSQA fills with SSAT blocks. ABEND065 ABEND80A rc10 OY50690 SP252 filled with 8K AMS blocks unfreed - DB2DBM1 terminated End Of Memory ABEND0F9 RC0 after experienced problem described by PN08784. OY57103 DBM1 SRM ESTOR AUX STORAGE OY43909 SP0 key7 DB2DBM1 asid filled with DFP/IDCAMS data. ABEND80A ABEND878 rc10 SSOB SA03 SA02 OY50506 PE FIXED BY OY51287 SP0 key7 DB2MSTR filled with AMS DYNAMIC ALLOC IGDMCSMG SMS PL68064 UL81908 IRLM R502 SP241 RHBS / RLBS CSA CICS OY64236 Excess paging to AUX not ESTOR OW08339 MVS/ESA excessive paging OW15636 MVS/ESA VSMDATA enhancement OW15713 MVS/ESA high paging to ESTOR PN69781 SP228 overlay SRB pool (SDSF) (DB2 symptom rc00E2000E) 285822 289633 SP241 CSA filled with OEM BMC LOAD control blocks. 16k SMP 289622 and 1k CSA control block. see II09269 for OEM errs OW18622 All DFSMS:LSQA overlay various abends in SMC. SKB overlaid unpredictable results. (S04E rc00E20028) OW23320 CSA overlay with eyecatcher of ENF51C in overlaid data OW28845 SYSPLEXDS s40D s878 overlay ow28154 (FIN) rc00F3040A after batch S80A rc00F30905 rc00F304xx DSN3SSI1 OW31042 fix for FIN ow28154 list above II07124 IEFTB728 req00074033 SMFexit sp230 k0 32K blks EXCP type30 DETAIL records E6007FF4 SMFPRMxx set NODETAIL INTERVAL OW40666 Storage overlays Media Manager OW41795 VMM overlays, SHARK or RVA OW40873 ABENDA78 rc18 at DB2 task term IEFDB4F8 req00076376 SMF TCT TCTIOT sp255 LOW LSQA
Local fix
Problem summary
Problem conclusion
Temporary fix
Comments
DB2INFO: This apar summarizes DB2 DSNxxxx asid storage problems. For users of MVS ESA/430 and above, always be sure that you have MVS Common Storage Tracking enabled (DIAG01) thru MVS sysparm. This is a very valuable diagnostic tool, and its activation will cause NO system degradation whatsoever. This diagnostic aide is as valuable as the internal System Trace, and like the Trace, it should be running at all times. In regard to the System Trace, it should be set for no less than the normal ESA/4/5 default, 64k (MVS cmd: TRACE ST,128k). For DB2 storage concerns, the required documentation is rather standard for shortage errors. 2 SVCDUMPs will ususlly be requested by SUPPORT; 1 dump taken abit after DB2 startup, 1 dump taken after the storage concern has been experienced. LVL2 will require that both of these dumps be COMPLETE dumps, and along with MSTR asid, at minimum, these dumps must contain all; RGN, CSA, SQA, LSQA, SWA, TRT and SUM. of the associated DB2 asid in question. Any PARTIAL dump sent in as doc will only slowdown problem diagnosis, cause general confusion, plus waste valuable time. Check for: MSGIEA911E COMPLETE DUMP ON SYS1.DUMPxx or MSG1EA611I COMPLETE DUMP ON SYSx.DUMP.yyyy (from DYNADUMP) -----------********----------------------- Reference SUG apar OY65789: Be aware that at least 1 COMPLETE dump of an abend kind must be captured on a DUMPxx dataset before DAE will even bother to try to suppress subsequent duplicate dumps of said abend kind. Beware of your DUMPSRV MaxSpace setting and small preallocated DUMPxx datasets. (We recommend MAXSPACE=2500m). When using IPCS to view a storage related dump, you may find the following IPCS commands helpful: STATUS RSMDATA RSMDATA VIRTPAGE ASID(x'yy') ASMCHECK VERBX DAEDATA VERBX VSMDATA VERBX VSMDATA 'OWNCOMM' (Check Common Storage Tracking) VERBX VSMDATA 'OWNCOMM DETAIL ALL SORTBY(ASIDADDR)' VERBX VSMDATA 'OWNCOMM DETAIL ASID(ddd) SORTBY(TIME)' VERBX VSMDATA 'NOGLOBAL,JOBNAME(xxxxDBM1)' (See VSM infoapar II05506) VERBX DSNWDMP 'SUBSYS=sssn,SMSA=asid,SM=1' (See DB2 apar PQ05118) WARNING: running IPCS formatting routines on PARTIAL dumps can lead to false results and erroneous conclusions. Note: To access the DB2DATA DSNWDMP thread and storage format routines, the DSNWDMP formatter must be properly linked to your TSO IPCS session. It is in SDSNLOAD. When viewing IPCS formatted DUMP areas like DSNWDMP, it is BEST, to use a 3270 mod5 27/132 emulator. Most formatted areas, ie; VSMDATA, DB2DATA, SYSTRACE are 132 length format. ALSO, the TSO session controlling IPCS should have been started with a REGION size NO smaller than 64000K (64meg). Remember: The DB2 DBM1 address space uses millions of bytes of data for various buffer pools when DB2 is running. This DB2 asid will usually be the largest overall storage user. Do not interpret this NORMAL DBM1 operation as the 'ROOT' of your storage or paging condition. We suggest that users first benchmark their DB2 operation to get an idea of NORMAL DB2 asid (xxxxMSTR xxxxDBM1 xxxxDIST xxxxIRLM) storage usage under the associated MVS/ESA/OS390 in use. Read II06335 for related DB2 SVCDUMP information. ---------------------------------------------------------------- A Storage Management document is currently available as a PDF file at the following internet URL : ftp://ftp.software.ibm.com/software/data/db2zos/DMDB13.pdf ----------------------------------------------------------------
APAR Information
APAR number
II04309
Reported component name
PB LIB INFO ITE
Reported component ID
INFOPBLIB
Reported release
001
Status
CLOSED CAN
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
1990-03-13
Closed date
1990-03-15
Last modified date
2010-02-08
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Applicable component levels
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEPEK","label":"DB2 for z\/OS"},"Platform":[{"code":"PF054","label":"z Systems"}],"Version":"001","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
Document Information
Modified date:
31 July 2023