IBM Support

IV61294: SYSTEM IO HANG IN J2PAGERSERVICE APPLIES TO AIX 7100-02

A fix is available

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • System may hang with a j2pg thread at:
    pvthread+005700 STACK:
    [000F1658]e_block_thread+000298 ()
    [000F21C8]e_sleep_thread+0000E8 (??, ??, ??)
    [003060F0]j2PagerService+000230 (??)
    [00302194]j2PagerThread+0001F4 (??)
    [003B7774]threadentry+000094 (??, ??, ??, ??)
    [kdb_read_mem] no real storage @ FFFFFFFFFFF91B0
    
    The buffer passed to j2PagerService (in r31 in
    j2PagerService):
    should have the READ flag on:
      0 F1000A01C1CED0A0 00000000DEADBEEF 000080D0 READ
        AGE PFEOF MPSAFE INITIAL BUFX
    The inode associated with the buffer should
    have the ICMLOCK flag bit on:
           DEVICE      I_NUM/FS COUNT TYPE FLAGS
    8000002E00000002 7864324/16 00001 VREG ICHG IUPD ICMLOCK
    Finally, the pdt for that device should show
    that there are no free buffers for this device:
    (0)> pdt 0092
    ...
    free buf_struct list  (bufstr)  : 0000000000000000
    ...
    
    The problem is a deadlock with two threads
    accessing a memory mapped (mmap) file.
    

Local fix

  • Make sure that the ioo command tunable
    j2_maxPageReadAhead is less than half of the
    (restricted tunable, use ioo -aF to view)
    j2_nBufferPerPagerDevice tunable.
    

Problem summary

  • System may hang with a j2pg thread at:
    pvthread+005700 STACK:
     000F1658 e_block_thread+000298 ()
     000F21C8 e_sleep_thread+0000E8 (??, ??, ??)
     003060F0 j2PagerService+000230 (??)
     00302194 j2PagerThread+0001F4 (??)
     003B7774 threadentry+000094 (??, ??, ??, ??)
     kdb_read_mem  no real storage @ FFFFFFFFFFF91B0
    
    The buffer passed to j2PagerService (in r31 in
    j2PagerService):
    should have the READ flag on:
      0 F1000A01C1CED0A0 00000000DEADBEEF 000080D0 READ
        AGE PFEOF MPSAFE INITIAL BUFX
    The inode associated with the buffer should
    have the ICMLOCK flag bit on:
           DEVICE      I_NUM/FS COUNT TYPE FLAGS
    8000002E00000002 7864324/16 00001 VREG ICHG IUPD ICMLOCK
    Finally, the pdt for that device should show
    that there are no free buffers for this device:
    (0)> pdt 0092
    ...
    free buf_struct list  (bufstr)  : 0000000000000000
    ...
    
    The problem is a deadlock with two threads
    accessing a memory mapped (mmap) file.
    

Problem conclusion

  • Modified the code to avoid hang by limiting the readahead to
    half the number of buffers that were allocated when the
    pagerDevice was initialized. We will be dropping code to
    enforce this ratio, but that is NOT the entire solution.  We
    have not decided on the complete solution.
    

Temporary fix

Comments

  • 6100-08 - use AIX APAR IV62565
    6100-09 - use AIX APAR IV59740
    6100-09 - use AIX APAR IV59740
    6100-09 - use AIX APAR IV59740
    7100-02 - use AIX APAR IV61294
    7100-03 - use AIX APAR IV61211
    7100-04 - use AIX APAR IV61245
    

APAR Information

  • APAR number

    IV61294

  • Reported component name

    AIX V7.1

  • Reported component ID

    5765H4000

  • Reported release

    710

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Submitted date

    2014-06-06

  • Closed date

    2014-06-06

  • Last modified date

    2016-05-11

  • APAR is sysrouted FROM one or more of the following:

    IV61211

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    AIX V7.1

  • Fixed component ID

    5765H4000

Applicable component levels

  • R710 PSY U867194

       UP15/01/19 I 1000

PTF to Fileset Mapping



Document information

More support for: AIX Enterprise Edition

Software version: 710

Operating system(s): AIX

Reference #: IV61294

Modified date: 11 May 2016