IBM Support

MustGather: Core Security problems for WebSphere traditional and Liberty

Troubleshooting


Problem

This document describes the process for collecting data for problems with the IBM WebSphere® Application Server traditional and WebSphere Liberty core security components. The core security component includes authentication, authorization, user and group search, membership and members, user and group role assignment, SSO (except web SSO), and so on. Gathering this MustGather information before calling IBM support helps you understand the problem and save time analyzing the data.

Avoid delay:
For Web Single Sign-on (SSO) component issues, such as SAML, OpenID, OpenID Connect or OAuth, see: MustGather: Web Single Sign-on problems with WebSphere Application Server.   
For SPNEGO issues, see: MustGather: Problems with SPNEGO.  

Resolving The Problem


  • Read first and MustGathers

    MustGather: Read first for WebSphere Application Server and Liberty

    Java 2 security problem SSL/JSSE problem SPNEGO problem Web Single Sign-on TAI problem


    For a listing of all technotes, downloads, and educational materials specific to the Security component, search the WebSphere Application Server support site.
  • Exchange data with IBM Support

    To diagnose or identify a problem, it is sometimes necessary to provide Technical Support with data and information from your system. In addition, Technical Support might also need to provide you with tools or utilities to be used in problem determination. You can submit files by using one of following methods to help speed problem diagnosis:

    • Service Request (SR)
    • Email
    • FTP to the Enhanced Customer Data Repository (ECuRep)

    Instructions for exchanging information with IBM Support

  • Core security trace specifications
    • WebSphere traditional
      Avoid trouble: The trace strings for WebSphere traditional must be entered as one line with no breaks or spaces.

      Core security trace string:

      *=info:com.ibm.ws.security.*=all:com.ibm.websphere.security.*=all:com.ibm.websphere.wim.*=all:com.ibm.wsspi.wim.*=all:com.ibm.ws.wim.*=all

       
      • If the problem is related to authentication to an Enterprise JavaBean™, append the following to the trace specification:

        :SASRas=all:ORBRas=all

      • If the problem is related to security domains, append the following to the trace specification:

        :SecurityDomain=all

      • Only if requested by IBM Support, collect communication (COMM) traces by setting the following in the Generic JVM arguments for the JVM being traced:

        -Dcom.ibm.CORBA.Debug=true
        -Dcom.ibm.CORBA.CommTrace=true

    • Liberty
      Core security trace string:

      com.ibm.ws.security.*=all:com.ibm.ws.webcontainer.security.*=all

  • Collect data for WebSphere traditional
    • This section is for collecting data for WEBSPHERE TRADITIONAL. If you want to collect data for Liberty, see the Collect data for Liberty section.
    • Diagnostic questions

      Answer the following questions about the problem

      1. What activities were taking place when the problem was noticed? (Starting a server, enabling security, a user logging on, changing security configuration, managing users or groups, and so on.)
      2. Are you able to consistently make the problem happen or is there no apparent trigger or timing?
      3. If you can consistently make the problem happen, what steps do you take?
      4. On which JVMs are you seeing the problem?
        • For WebSphere traditional: deployment manager, node agent, server1, cluster member, and so on.
      5. Are there messages you can point us to in the SystemOut.log, or something else that has a timestamp of the problem?
        • If not, provide your best estimate of the date and time and where (what JVM) the problem occurred.
      6. Have you used the WebSphere Application Server support portal to research this problem?
      7. Have you taken any steps to try to resolve this issue?
        • If so what steps did you take?
    • Step-by-step
      • Before you collect data, be sure to answer the questions in the Diagnostic questions section.
      • You can choose to follow this step-by-step document or you can watch the video in the Video section.
      • Security issues on WebSphere might be difficult to troubleshoot. Make sure to collect all the information described. When all the information for your issue is ready, follow the instructions on Exchanging information with IBM Technical Support for problem determination to send the information and files that you collected.

      SET UP WEBSPHERE TRADITIONAL FOR CORE SECURITY TRACING

      1. Enable trace core security trace
        • If the administrative console is not functioning:
        • If you can use your administrative console:
          1. In the administrative console, expand Troubleshooting > Logs and Trace > server_name > Diagnostic trace service
          2. Configure the trace file specifications
            1. Under Trace Output, select File
            2. Set Maximum File Size and Maximum Number of Historical Files
              • Security traces are large. You need to change the Maximum Number of Historical Files and Maximum File Size to a sufficient number to capture the problem. Set the Maximum Number of Historical Files to at least 20 and the Maximum File Size to at least 20MB.
            3. Click Apply
          3. Set the trace specification
            1. Click Change log level details
            2. If there is a trace specification in the box, delete it
            3. Enter your trace string:
              • Start with this core security trace string:

                *=info:com.ibm.ws.security.*=all:com.ibm.websphere.security.*=all:com.ibm.websphere.wim.*=all:com.ibm.wsspi.wim.*=all:com.ibm.ws.wim.*=all

              • If the problem is related to authentication to an Enterprise JavaBean™, append the following to the trace string:

                :SASRas=all:ORBRas=all

              • If the problem is related to security domains, append the following to the trace string:

                :SecurityDomain=all

            4. Click Apply
          4. Click Save
      2. Enable COMM trace (only if requested)


       

      COLLECT WEBSPHERE TRADITIONAL CORE SECURITY TRACES

      Avoid delay: Unless otherwise instructed, WebSphere core security traces must be gathered from server startup.

      For each WebSphere server that you are tracing:
      1. Stop the server.
      2. Backup and clear the logs and FFDC directories.
      3. Start the server
      4. Reproduce the problem, making note of time when the problem occurs

      Remember to 'undo' the tracing after the issue is resolved.

       

      GATHER WEBSPHERE TRADITIONAL CORE SECURITY DATA TO SEND TO IBM

      Avoid delay: All of the following data is required for proper problem determination of most issues. Do not send a subset of this data unless you were instructed to do so by IBM support.
       
      File to send
      Instructions
      Diagnostic questions Answer the questions in the  Diagnostic questions section.
      A collector jar

      Note: You need to run the collector tool on each <PROFILE_ROOT> in which you enabled trace.
             		 From a temporary directory, run the Collector Tool, collector.sh or collector.bat, which is located in the <PROFILE_ROOT>/bin directory.

      If there is a message about the collector tool being deprecated, ignore it. IBM support needs you to run this tool.
      waspolicies If you are using multiple security domains, send us all the files and directories under:

      <PROFILE_ROOT>/config/waspolicies

      See the information in Exchanging information with IBM Technical Support for problem determination to send this diagnostic information to IBM support.
       
    • Video

      This section is for collecting data for WEBSPHERE TRADITIONAL. If you want to collect data for Liberty, see the Collect data for Liberty section.

      • Before you collect data, be sure to answer the questions in the  Diagnostic questions section.
      • You can choose to watch this video or follow the step-by-step instructions in the Step-by-step section.
      • Security issues on WebSphere might be difficult to troubleshoot. Make sure to collect all the information described in the video. When all the information for your issue is ready, follow the instructions on Exchanging information with IBM Technical Support for problem determination to send the information and files that you collected.
      • The following video goes over the necessary steps to collect data for a core security problem on WebSphere traditional:
  • Collect data for Liberty
    This section is for collecting data for LIBERTY. If you want to collect data for WebSphere traditional, see the Collect data for WebSphere traditional section.
     

    SET UP LIBERTY FOR CORE SECURITY TRACING

    1. Set up the Liberty server for core security tracing
    2. Verify that your tracing is working as intended
      1. Stop the Liberty Server
      2. Delete any existing logs files found under the logs directory:
        <LIBERTY_HOME>/usr/servers/<serverName>/logs
      3. Restart the Liberty Server and review the logs to confirm that they are recent.
      4. Verify that the new Liberty trace setting is picked up by reviewing the top of the trace.log file.
     

    COLLECT LIBERTY CORE SECURITY TRACES

    Avoid trouble: Core security traces must be gathered from Liberty server startup.
    1. Stop the Liberty server
    2. Restart the Liberty server
    3. Reproduce the problem, making note any information that might be useful when the trace file is processed, such as:
      • Relevant user and group names used
      • Exact URL strings accessed
      • General time stamps
     

    GATHER LIBERTY DATA TO SEND TO IBM SUPPORT

    • Use the "dump" command to generate a .zip file containing the logs and config files that can be sent to support.
      • For Windows platforms, run:
        <LIBERTY_HOME>\bin\server.bat dump <serverName>

        For UNIX platforms, run:
        <LIBERTY_HOME>/bin/server dump <serverName>
      • Collect the resulting dump .zip file with date & time. These files can be found under the following path:
        <LIBERTY_HOME>/usr/servers/<serverName>

        File name example:
        (myserver.dump-17.03.20_22.20.57.zip)
    • Collect the java.security file from your JDK. This file can be found under the following path:
      JAVA_HOME\lib\security\java.security

    Avoid delay: Make sure that traces that you send to IBM support covers a timeframe where your issue occurs.

    See the information in Exchanging information with IBM Technical Support for problem determination to send the trace files and supplemental information to IBM support.

Note:

This document uses the term WebSphere traditional to refer to WebSphere Application Server v9.0 traditional, WebSphere Application Server v8.5 full profile, WebSphere Application Server v8.0 and earlier, WebSphere classic, traditional WebSphere, traditional WAS, and tWAS.  

[{"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Security","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"}],"Version":"9.0;8.5;8.0;7.0","Edition":"Base;Express;Network Deployment","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
07 September 2023

UID

swg21470063

Page Feedback