IV20172: CAN'T FEDERATE A NODE ON EZWAS WITH BUILD

Subscribe

You can track all active APARs for this component.

APAR status

  • Closed as program error.

Error description

  • Error Message: Hang was noted in websphere federation module at
    netInterfaces = NetworkInterface.getNetworkInterfaces();
    Failed about 9 out of 10 executions when federating a node in
    WebSphere
    We did check, this has happened on > 10 different systems, and
    it only appears on WebSphere 8.0. The federate did not change,
    however there was a new java build when this started.
    We don't see any sort of error other than a hang on the
    following:
    netInterfaces = NetworkInterface.getNetworkInterfaces();
    .
    Stack Trace: 1B90000000059 t=6BE9C0 c=UNK ./bbgrjtr.cpp+733 tag=
    ... BBOO0221W: WSVR0605W: Thread 'SoapConnectorThreadPool : 0'
    (00000014) has be
     en active for 649908 milliseconds and may be hung.  There
    is/are 1 thread(s) in total in the server that may be hung.
      at java.lang.ClassLoader.loadClass(ClassLoader.java:627)
      at
    org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLo
    ader.java:418)
      at
    org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLo
    ader.java:410)
      at
    org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadCla
    ss(DefaultClassLoader.java:107)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:627)
      at
    com.ibm.websphere.plugincfg.generator.ConfigurationParserHelper.
    getAllLocalAddresses(ConfigurationParserHelper.java:1095)
      .
     .
     .
    Line 1095 in the ConfiguratoinParserHelper.java class is a call
    to the getNetworkInterfaces(), here is a code snipet:
    /*
       * Obtain all IP addresses for this node
       */
      public static Enumeration getAllLocalAddresses()
      {
        Enumeration netInterfaces = null;
        Vector hostAddresses = new Vector(1, 1);
        try
        {
          netInterfaces = NetworkInterface.getNetworkInterfaces();
    //line 1095
        }
        catch (java.net.SocketException x)
        {
          if (tc.isDebugEnabled())
          {
            Tr.debug(tc, 'Exception thrown in
    NetworkInterface.getNetworkInterfaces.  Continue.');
          }
          return null;
        }
    .
    

Local fix

  • Don't pickup fix pack build pmz6460_26sr1fp1-20120109_01 for WAS
    

Problem summary

  • Fix put in for "Deadlock between URL handler/Classloader" to
    synchronize methods of getresource of urlclassloader.java has
    created the problem
    1LKDEADLOCK    Deadlock detected !!!
    NULL           ---------------------
    NULL
    2LKDEADLOCKTHR  Thread 'Non-deferrable Alarm : 0'
    (0x00000000012B8A00)
    3LKDEADLOCKWTR    is waiting for:
    4LKDEADLOCKMON      sys_mon_t:0x0000040014088C00 infl_mon_t:
    0x0000040014088C78:
    4LKDEADLOCKOBJ
    com/ibm/ws/bootstrap/ExtClassLoader@0x0000000010A7F940/0x0000000
    010A7F944:
    3LKDEADLOCKOWN    which is owned by:
    2LKDEADLOCKTHR  Thread 'PluginConfigService : 0'
    (0x0000000002229100)
    3LKDEADLOCKWTR    which is waiting for:
    4LKDEADLOCKMON      sys_mon_t:0x0000040014CD0588 infl_mon_t:
    0x0000040014CD0600:
    4LKDEADLOCKOBJ
    java/lang/Object@0x0000000010121CE0/0x0000000010121CE4:
    3LKDEADLOCKOWN    which is owned by:
    2LKDEADLOCKTHR  Thread 'Non-deferrable Alarm : 0'
    (0x00000000012B8A00)
    3XMTHREADINFO      'Non-deferrable Alarm : 0'
    J9VMThread:0x00000000012B8A00, j9thread_t:0x00000400064427E0,
    java/lang/Thread:0x00000000115A6F90, state:B, prio=5
    3XMTHREADINFO1            (native thread ID:0x612, native
    priority:0x5, native policy:UNKNOWN)
    3XMTHREADINFO2            (native stack address range
    from:0x0000040011EC0000, to:0x0000040011F00000, size:0x40000)
    3XMTHREADINFO3           Java callstack:
    4XESTACKTRACE                at
    java/lang/ClassLoader.loadClass(ClassLoader.java:630(Compiled
    Code))
    4XESTACKTRACE                at
    com/ibm/ws/bootstrap/ExtClassLoader.loadClass(ExtClassLoader.jav
    a:113(Compiled Code))
    4XESTACKTRACE                at
    java/lang/ClassLoader.loadClass(ClassLoader.java:612(Compiled
    Code))
    4XESTACKTRACE                at
    com/ibm/ws/eba/internal/framework/EBAClassLoader.loadClass(EBACl
    assLoader.java:86(Compiled Code))
    4XESTACKTRACE                at
    org/eclipse/osgi/launch/EquinoxFWClassLoader.loadClass(EquinoxFW
    ClassLoader.java:46(Compiled Code))
    4XESTACKTRACE                at
    java/lang/ClassLoader.loadClass(ClassLoader.java:638(Compiled
    Code))
    4XESTACKTRACE                at
    java/lang/ClassLoader.loadClass(ClassLoader.java:612(Compiled
    Code))
    4XESTACKTRACE                at
    sun/reflect/DelegatingClassLoader.loadClass(DelegatingClassLoade
    r.java:51)
    4XESTACKTRACE                at
    sun/misc/Unsafe.defineClass(Native Method)
    4XESTACKTRACE                at
    sun/reflect/ClassDefiner.defineClass(ClassDefiner.java:57)
    4XESTACKTRACE                at
    sun/reflect/MethodAccessorGenerator$1.run(MethodAccessorGenerato
    r.java:437)
    4XESTACKTRACE                at
    java/security/AccessController.doPrivileged(AccessController.jav
    a:203(Compiled Code))
    4XESTACKTRACE                at
    sun/reflect/MethodAccessorGenerator.generate(MethodAccessorGener
    ator.java:433)
    4XESTACKTRACE                at
    sun/reflect/MethodAccessorGenerator.generateMethod(MethodAccesso
    rGenerator.java:85(Compiled Code))
    4XESTACKTRACE                at
    sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAccessor
    Impl.java:54(Compiled Code))
    4XESTACKTRACE                at
    sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethod
    AccessorImpl.java:37(Compiled Code))
    4XESTACKTRACE                at
    java/lang/reflect/Method.invoke(Method.java:611(Compiled Code))
    4XESTACKTRACE                at
    org/eclipse/osgi/framework/internal/protocol/MultiplexingFactory
    .findAuthorizedFactory(MultiplexingFactory.java:128)
    4XESTACKTRACE                at
    org/eclipse/osgi/framework/internal/protocol/StreamHandlerFactor
    y.findAuthorizedURLStreamHandler(StreamHandlerFactory.java:194)
    4XESTACKTRACE                at
    org/eclipse/osgi/framework/internal/protocol/StreamHandlerFactor
    y.createURLStreamHandler(StreamHandlerFactory.java:112)
    4XESTACKTRACE                at
    java/net/URL.getURLStreamHandler(URL.java:1184(Compiled Code))
    4XESTACKTRACE                at
    java/net/URL.<init>(URL.java:405(Compiled Code))
    4XESTACKTRACE                at
    java/net/URL.<init>(URL.java:295(Compiled Code))
    4XESTACKTRACE                at
    java/security/CodeSource.getDecodedURL(CodeSource.java:344(Compi
    led Code))
    4XESTACKTRACE                at
    java/security/CodeSource.implies(CodeSource.java:357(Compiled
    Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/policy/WSDynamicPolicy.getPermissions_s(WSDy
    namicPolicy.java:274(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/policy/WSDynamicPolicy.getPermissions(WSDyna
    micPolicy.java:199(Compiled Code))
    4XESTACKTRACE                at
    java/security/Policy.getPermissions(Policy.java:394(Compiled
    Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/policy/WSDynamicPolicy.getPermissions(WSDyna
    micPolicy.java:1047(Compiled Code))
    4XESTACKTRACE                at
    java/security/Policy.defaultImplies(Policy.java:438(Compiled
    Code))
    4XESTACKTRACE                at
    java/security/Policy.implies(Policy.java:428(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/policy/WSDynamicPolicy.implies(WSDynamicPoli
    cy.java:1079(Compiled Code))
    4XESTACKTRACE                at
    java/security/ProtectionDomain.implies(ProtectionDomain.java:199
    (Compiled Code))
    4XESTACKTRACE                at
    java/security/AccessController.checkPermission(AccessController.
    java:98(Compiled Code))
    4XESTACKTRACE                at
    java/lang/SecurityManager.checkPermission(SecurityManager.java:5
    44(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/core/SecurityManager.checkPermission(Securit
    yManager.java:208(Compiled Code))
    4XESTACKTRACE                at
    org/eclipse/osgi/framework/internal/core/Framework.checkAdminPer
    mission(Framework.java:1177(Compiled Code))
    4XESTACKTRACE                at
    org/eclipse/osgi/framework/internal/core/BundleHost.loadClass(Bu
    ndleHost.java:223(Compiled Code))
    4XESTACKTRACE                at
    org/eclipse/osgi/framework/internal/core/AbstractBundle.loadClas
    s(AbstractBundle.java:1197(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/util/ImplFactory$2.run(ImplFactory.java:623(Compiled
    Code))
    4XESTACKTRACE                at
    java/security/AccessController.doPrivileged(AccessController.jav
    a:254(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/util/AccessController.doPrivileged(AccessCon
    troller.java:125(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/util/ImplFactory.loadClass(ImplFactory.java:621(Compi
    led Code))
    4XESTACKTRACE                at
    com/ibm/ws/util/ImplFactory.loadClassFromKey(ImplFactory.java:37
    3(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/util/ImplFactory.loadImplFromKey(ImplFactory.java:328
    (Compiled Code))
    4XESTACKTRACE                at
    com/ibm/websphere/management/repository/client/ConfigRepositoryC
    lientFactory.getConfigRepositoryClient(ConfigRepositoryClientFac
    tory.java:47)
    4XESTACKTRACE                at
    com/ibm/ws/eba/bla/pluginconfig/EBAPluginConfigUtils.getAssetFro
    mCU(EBAPluginConfigUtils.java:135)
    4XESTACKTRACE                at
    com/ibm/ws/eba/bla/odc/EBAConfigProviderImpl.convertToCUName(EBA
    ConfigProviderImpl.java:217)
    4XESTACKTRACE                at
    com/ibm/ws/eba/odc/EBATreeBuilder.handleDocChange(EBATreeBuilder
    .java:396)
    4XESTACKTRACE                at
    com/ibm/ws/eba/odc/EBATreeBuilder.handleConfigChange(EBATreeBuil
    der.java:360)
    4XESTACKTRACE                at
    com/ibm/ws/eba/odc/EBATreeBuilder.configChanged(EBATreeBuilder.j
    ava:309)
    4XESTACKTRACE                at
    com/ibm/ws/eba/odc/EBATreeBuilder.access$400(EBATreeBuilder.java
    :121)
    4XESTACKTRACE                at
    com/ibm/ws/eba/odc/EBATreeBuilder$LocalEventListener.configChang
    ed(EBATreeBuilder.java:1085)
    4XESTACKTRACE                at
    com/ibm/ws/management/component/ConfigChangeHandler$ConfigChange
    EventDispatcher.run(ConfigChangeHandler.java:365)
    4XESTACKTRACE                at
    com/ibm/ws/management/component/ConfigChangeHandler.alarm(Config
    ChangeHandler.java:167)
    4XESTACKTRACE                at
    com/ibm/ejs/util/am/_Alarm.run(_Alarm.java:133(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1650)
    3XMTHREADINFO      'PluginConfigService : 0'
    J9VMThread:0x0000000002229100, j9thread_t:0x0000040011697280,
    java/lang/Thread:0x000000001514E678, state:B, prio=5
    3XMTHREADINFO1            (native thread ID:0xD88, native
    priority:0x5, native policy:UNKNOWN)
    3XMTHREADINFO2            (native stack address range
    from:0x00000400126E0000, to:0x0000040012720000, size:0x40000)
    3XMTHREADINFO3           Java callstack:
    4XESTACKTRACE                at
    java/net/URL.getURLStreamHandler(URL.java:1169(Compiled Code))
    4XESTACKTRACE                at
    java/net/URL.<init>(URL.java:405(Compiled Code))
    4XESTACKTRACE                at
    java/net/URL.<init>(URL.java:295(Compiled Code))
    4XESTACKTRACE                at
    java/security/CodeSource.getDecodedURL(CodeSource.java:344(Compi
    led Code))
    4XESTACKTRACE                at
    java/security/CodeSource.implies(CodeSource.java:358(Compiled
    Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/policy/WSDynamicPolicy.getPermissions_s(WSDy
    namicPolicy.java:274(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/policy/WSDynamicPolicy.getPermissions(WSDyna
    micPolicy.java:199(Compiled Code))
    4XESTACKTRACE                at
    java/security/Policy.getPermissions(Policy.java:394(Compiled
    Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/policy/WSDynamicPolicy.getPermissions(WSDyna
    micPolicy.java:1047(Compiled Code))
    4XESTACKTRACE                at
    java/security/Policy.defaultImplies(Policy.java:438(Compiled
    Code))
    4XESTACKTRACE                at
    java/security/Policy.implies(Policy.java:428(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/policy/WSDynamicPolicy.implies(WSDynamicPoli
    cy.java:1079(Compiled Code))
    4XESTACKTRACE                at
    java/security/ProtectionDomain.implies(ProtectionDomain.java:199
    (Compiled Code))
    4XESTACKTRACE                at
    java/security/AccessController.checkPermission(AccessController.
    java:98(Compiled Code))
    4XESTACKTRACE                at
    java/lang/SecurityManager.checkPermission(SecurityManager.java:5
    44(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/core/SecurityManager.checkPermission(Securit
    yManager.java:208(Compiled Code))
    4XESTACKTRACE                at
    java/lang/SecurityManager.checkRead(SecurityManager.java:883(Com
    piled Code))
    4XESTACKTRACE                at
    java/io/File.exists(File.java:742(Compiled Code))
    4XESTACKTRACE                at
    sun/misc/URLClassPath$FileLoader.getResource(URLClassPath.java:1
    231(Compiled Code))
    4XESTACKTRACE                at
    sun/misc/URLClassPath.getResource(URLClassPath.java:289(Compiled
    Code))
    4XESTACKTRACE                at
    java/net/URLClassLoader$ClassFinder.run(URLClassLoader.java:1036
    (Compiled Code))
    4XESTACKTRACE                at
    java/security/AccessController.doPrivileged(AccessController.jav
    a:288)
    4XESTACKTRACE                at
    java/net/URLClassLoader.findClass(URLClassLoader.java:429(Compil
    ed Code))
    4XESTACKTRACE                at
    com/ibm/ws/bootstrap/ExtClassLoader.findClass(ExtClassLoader.jav
    a:198(Compiled Code))
    4XESTACKTRACE                at
    java/lang/ClassLoader.loadClass(ClassLoader.java:646(Compiled
    Code))
    4XESTACKTRACE                at
    com/ibm/ws/bootstrap/ExtClassLoader.loadClass(ExtClassLoader.jav
    a:113(Compiled Code))
    4XESTACKTRACE                at
    java/lang/ClassLoader.loadClass(ClassLoader.java:612(Compiled
    Code))
    4XESTACKTRACE                at
    com/ibm/xylem/types/ObjectFactory.findProviderClass(Bytecode
    PC:63(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/xylem/types/JavaObjectType.getJavaType(Bytecode
    PC:25(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/xylem/types/JavaObjectType.getImplementationType(Bytecod
    e PC:5(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/xylem/types/StreamType.getImplementationType(Bytecode
    PC:9(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/xylem/codegen/StreamOptimizedFunctionGenerationStyle.gen
    erateFunction(Bytecode PC:272)
    4XESTACKTRACE                at
    com/ibm/xylem/Program.generateCode(Bytecode PC:276)
    4XESTACKTRACE                at
    com/ibm/xtq/xslt/drivers/XSLTLinker.compileProgram2(Bytecode
    PC:776)
    4XESTACKTRACE                at
    com/ibm/xtq/xslt/drivers/XSLTLinker.compileProgram(Bytecode
    PC:79)
    4XESTACKTRACE                at
    com/ibm/xtq/xslt/drivers/XSLTLinker.compileProgram(Bytecode
    PC:13)
    4XESTACKTRACE                at
    com/ibm/xtq/xslt/drivers/XSLTCompiler.linkAndCodeGen(Bytecode
    PC:24)
    4XESTACKTRACE                at
    com/ibm/xtq/xslt/drivers/XSLTCompiler.compile(Bytecode PC:45)
    ......
    4XESTACKTRACE                at
    java/security/AccessController.doPrivileged(AccessController.jav
    a:254(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/security/util/AccessController.doPrivileged(AccessCon
    troller.java:125(Compiled Code))
    4XESTACKTRACE                at
    com/ibm/ws/management/AdminServiceImpl.invoke(AdminServiceImpl.j
    ava:1228)
    4XESTACKTRACE                at
    com/ibm/websphere/plugincfg/initializers/PluginTask.generatePlug
    inConfig(PluginTask.java:686)
    

Problem conclusion

  • This defect will be fixed in:
    6.0.1 SR1 FP1
    6.0.0 SR11
    .
    Fix to revert the fix which created the regression. Fix for
    "Deadlock between URL handler/Classloader". The original
    deadlock issue is considered as a limitation with Java 5.0, 60
    and 6.0.1. The deadlock issue fixed in Java 7 through parallel
    class loading feature which cannot be back ported.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IV20172

  • Reported component name

    JAVA CLASS LIBS

  • Reported component ID

    620700130

  • Reported release

    600

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt

  • Submitted date

    2012-04-24

  • Closed date

    2012-04-25

  • Last modified date

    2013-09-17

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    JAVA CLASS LIBS

  • Fixed component ID

    620700130

Applicable component levels

  • R600 PSY

       UP



Rate this page:

(0 users)Average rating

Add comments

Document information


More support for:

Runtimes for Java Technology
Java Class Libraries

Software version:

6.0

Reference #:

IV20172

Modified date:

2013-09-17

Translate my page

Machine Translation

Content navigation