IV61294: SYSTEM IO HANG IN J2PAGERSERVICE APPLIES TO AIX 7100-02

Obtain the fix for this APAR.

APAR status

• Closed as program error.

Error description

• System may hang with a j2pg thread at:
  pvthread+005700 STACK:
  [000F1658]e_block_thread+000298 ()
  [000F21C8]e_sleep_thread+0000E8 (??, ??, ??)
  [003060F0]j2PagerService+000230 (??)
  [00302194]j2PagerThread+0001F4 (??)
  [003B7774]threadentry+000094 (??, ??, ??, ??)
  [kdb_read_mem] no real storage @ FFFFFFFFFFF91B0
  
  The buffer passed to j2PagerService (in r31 in
  j2PagerService):
  should have the READ flag on:
    0 F1000A01C1CED0A0 00000000DEADBEEF 000080D0 READ
      AGE PFEOF MPSAFE INITIAL BUFX
  The inode associated with the buffer should
  have the ICMLOCK flag bit on:
         DEVICE      I_NUM/FS COUNT TYPE FLAGS
  8000002E00000002 7864324/16 00001 VREG ICHG IUPD ICMLOCK
  Finally, the pdt for that device should show
  that there are no free buffers for this device:
  (0)> pdt 0092
  ...
  free buf_struct list  (bufstr)  : 0000000000000000
  ...
  
  The problem is a deadlock with two threads
  accessing a memory mapped (mmap) file.
  

Local fix

• Make sure that the ioo command tunable
  j2_maxPageReadAhead is less than half of the
  (restricted tunable, use ioo -aF to view)
  j2_nBufferPerPagerDevice tunable.
  

Problem summary

• System may hang with a j2pg thread at:
  pvthread+005700 STACK:
   000F1658 e_block_thread+000298 ()
   000F21C8 e_sleep_thread+0000E8 (??, ??, ??)
   003060F0 j2PagerService+000230 (??)
   00302194 j2PagerThread+0001F4 (??)
   003B7774 threadentry+000094 (??, ??, ??, ??)
   kdb_read_mem  no real storage @ FFFFFFFFFFF91B0
  
  The buffer passed to j2PagerService (in r31 in
  j2PagerService):
  should have the READ flag on:
    0 F1000A01C1CED0A0 00000000DEADBEEF 000080D0 READ
      AGE PFEOF MPSAFE INITIAL BUFX
  The inode associated with the buffer should
  have the ICMLOCK flag bit on:
         DEVICE      I_NUM/FS COUNT TYPE FLAGS
  8000002E00000002 7864324/16 00001 VREG ICHG IUPD ICMLOCK
  Finally, the pdt for that device should show
  that there are no free buffers for this device:
  (0)> pdt 0092
  ...
  free buf_struct list  (bufstr)  : 0000000000000000
  ...
  
  The problem is a deadlock with two threads
  accessing a memory mapped (mmap) file.
  

Problem conclusion

• Modified the code to avoid hang by limiting the readahead to
  half the number of buffers that were allocated when the
  pagerDevice was initialized. We will be dropping code to
  enforce this ratio, but that is NOT the entire solution.  We
  have not decided on the complete solution.
  

Temporary fix

Comments

• 6100-08 - use AIX APAR IV62565
  6100-09 - use AIX APAR IV59740
  6100-09 - use AIX APAR IV59740
  6100-09 - use AIX APAR IV59740
  7100-02 - use AIX APAR IV61294
  7100-03 - use AIX APAR IV61211
  7100-04 - use AIX APAR IV61245
  

APAR Information

• APAR is sysrouted FROM one or more of the following:

  IV61211

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  AIX V7.1

• Fixed component ID

  5765H4000

Applicable component levels

• R710 PSY U867194

     UP15/01/19 I 1000

PTF to Fileset Mapping

• U867194 bos.mp64 7.1.2.19