IV34005: JVM CRASH DUE TO STACK OVERFLOW FROM INFINITE RECURSION WITH XTO OLKIT AWT ERROR HANDLERS

APAR status

• Closed as program error.

Error description

• Error Message: JVM terminating with signal 11 (Segmentation
  fault)
  .
  Stack Trace: Native stack trace which shows the recursion:
  #8002 0x936d0ee1 in J2DXErrHandler () from
  java/jre/lib/i386/xawt/libmawt.so
  #8003 0x936d0ee1 in J2DXErrHandler () from
  java/jre/lib/i386/xawt/libmawt.so
  #8004 0x936d0ee1 in J2DXErrHandler () from
  java/jre/lib/i386/xawt/libmawt.so
  #8005 0x949ca149 in _XError () from ./usr/lib/libX11.so.6
  #8006 0x949d2589 in _XReply () from ./usr/lib/libX11.so.6
  #8007 0x9672867d in XShmGetImage () from ./usr/lib/libXext.so.6
  #8008 0x936ceb90 in X11SD_GetImage () from
  java/jre/lib/i386/xawt/libmawt.so
  #8009 0x936ce3ab in X11SD_GetRasInfo () from
  java/jre/lib/i386/xawt/libmawt.so
  #8010 0x9383a83a in Java_sun_java2d_loops_MaskFill_FillAAPgram
  () from
  java/jre/lib/i386/libawt.so
  .
  

Local fix

• Use MToolkit  instead of the default XToolkit .
  export AWT_TOOLKIT=MToolkit
  

Problem summary

• The problem occurred when more than two threads register error
  handler with Xlib at a time. The current implementation allows
  to store only two error handlers (the current error handler and
  the previous error  handler) at a time. If a third thread tries
  to register another error handler, the error handler corresponds
  to the first thread will be lost and if the Xlib system call
  corresponds to the first thread failed, the current and previous
  error handlers cannot handle the error situation and it will
  result in an infinite recursion as it will keep on search for
  the matching error handler which was lost while registering the
  error handler corresponds to the third thread.
  

Problem conclusion

• This defect will be fixed in:
  6.0.0 SR14
  6.0.1 SR6
  7.0.0 SR6
  .
  JVM has been updated to ensure that a maximum of only two error
  handlers will be registered to Xlib at any point of time and
  thereby avoid the recursive condition.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  JAVA CLASS LIBS

• Fixed component ID

  620700130

Applicable component levels

• R600 PSY

     UP

• R700 PSY

     UP