IBM Support

II12026: MustGather: Read first/Fix list for OMPROUTE for z/OS Communications Server TCPIPINFO

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as canceled.

Error description

  • This informational APAR will describe documentation needed to
    diagnose most OMPROUTE problems and list some of the common
    problems.
    
    Before reporting a problem to support, please examine SYSLOGD,
    JES MSG output and the console log for errors.
    
    DOCUMENTATION:
      Most OMPROUTE problems are best documented by obtaining an
    OMPROUTE -t2 -d1 trace and by providing a dump of the TCPIP and
    OMPROUTE address spaces. For IPv6, add -6t2 and -6d1 to the
    trace options. A trace from startup is ideal because some
    information is only shown in the trace at startup and
    the time for problem determination and resolution is quicker
    when the trace captures the entire flow of events rather than
    just a small subset of events.
      An OMPROUTE trace from startup can be enabled by
    coding the trace options after the forward slash in the PARM
    field of the OMPROUTE catalogued procedure.
    An example of the recommended OMPROUTE catalogued procedure
    can be found at this URL:
    http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS
    /f1a1b470/2.5.1?SHELF=EZ2ZO10J.bks&DT=20070604132827
    
      If a trace cannot be enabled from startup, then the following
    commands can dynamically enable and disable tracing.
     To enable:   /F omproute,TRACE=2 (TRACE6=2 for IPv6)
                  /F omproute,DEBUG=1 (DEBUG6=1 for IPv6)
     To disable:  /F omproute,TRACE=0 (TRACE6=0 for IPv6)
                  /F omproute,DEBUG=0 (DEBUG6=0 for IPv6)
    Trace output is sent to one of the following locations:
      1)  destination referenced by OMPROUTE_DEBUG_FILE environment
          variable which is coded in the STDENV DD dataset
      2)  STDOUT DD - but trace output is only output to this
          location if OMPROUTE_DEBUG_FILE is NOT defined, and
          the trace is started at initialization.
      3)  /{TMPDIR}/omproute_debug  - (TMPDIR is usually /tmp)
    
    By default OMPROUTE will create 5 debug files, each 200K in
    size, for a total of 1 Meg worth of trace.  The size and number
    of trace files can be controlled with the OMPROUTE_DEBUG_CONTROL
    environment variable.  A 1 Meg trace will only cover a few
    minutes, so it is highly recommended that the size and number of
    trace files be increased.
    
    APAR PQ76172 adds new functionality to the OMPROUTE CTRACE
    and changes the output destination of the OMPROUTE trace to
    the CTRACE facility.  Use of DEBUGTRC option introduced by
    APAR PQ76172 is highly recommended.
    
    A dump should be taken with the following console command:
    /DUMP COMM=('title')
    /R xx,JOBNAME=(tcpip,omproute),DSPNAME=('tcpip'.*),CONT
    /R xx,SDATA=(ALLNUC,CSA,LPA,LSQA,PSA,RGN,SQA,SWA,TRT),END
    
    *** NOTE:  the strings "tcpip" and "omproute" in the above
               dump command should be replaced with the jobnames
               for TCPIP and OMPROUTE
    
    LE User abends (U40xx abends):
      LE will often try to clean up the cause of the abend BEFORE
    reporting the U40xx abend.  It is recommended you code the
    following LE runtime options in the OMPROUTE proc to provide
    abend dumps taken at the actual time of error, rather than
    after condition handling has occurred.
             TRAP(ON,NOSPIE) TERMTHDACT(UAIMM)
    A SYSMDUMP DD card must be coded in the OMPROUTE procedure
    in order to capture a usable dump.  Coding a CEEDUMP DD
    card in addition to the SYSMDUMP DD will provide a formatted
    LE dump as well as an abend dump.
    
    
    COMMON PROBLEMS:
      1) OSPF Adjacency failures, performance tuning
          A.  Verify OMPROUTE is being dispatched and has not
              been swapped out.
              Also verify that OMPROUTE is set at a dispatching
              priority one less than the TCPIP address space,
              and one higher than any other normal application.
              Note that applications started via JCL with
              PGM=BPXBATCH may not be treated as started tasks
              and may not receive the proper dispatching priority.
              It is highly recommended that OMPROUTE be started
              via JCL with PGM=OMPROUTE
          B.  Code the OMPROUTE_OPTIONS=hello_hi environment
              variable to treat Hello packets at a higher priority
    
      2) OMPROUTE is DR or BDR
          A.  This problem may occur if OMPROUTE is elected the
              OSPF Designated Router (DR) or Backup (BDR).  OMPROUTE
              should never become a DR - this task is better left
              to a hardware router.  In order to ensure that
              OMPROUTE never becomes a DR, it is highly recommended
              that ROUTER_PRIORITY=0 be coded on EVERY
              OSPF_INTERFACE statement.  A Router Priority of zero
              informs routers that OMPROUTE will not become a DR.
          B.  LE Runtime options indicate storage should be
              kept rather than freed.
              The LE default setting for HEAP management is:
                  HEAP((32K,32K,ANYWHERE,KEEP,8K,4K),OVR)
              This statement allocates 32K heaps and does not
              free the heap (HANC control block) after it is
              no longer needed.
              To prevent storage growths, the following setting
              is recommended for OMPROUTE:
                  HEAP(,,,FREE)
              This parameter can be specified on the JCL PARMS
    
      3)  Various routing problems can be caused because
          one or more interfaces are not defined in the
          OMPROUTE configuration file.  All interfaces
          defined to TCPIP MUST be defined to OMPROUTE
          as OSPF_INTERFACE, RIP_INTERFACE, or INTERFACE.
          Failure to correctly define ALL interfaces
          will lead to a variety of routing problems.
          Please note that the minimal definition
          for any type of interface includes IP address,
          subnet mask, and MTU size unless the defaults
          are acceptable.  The default subnet mask is the
          class mask of the IP address, and the default
          MTU is 576.  In most cases, these values are not
          acceptable and will result in expected/undesired
          network and subnet routes, and an MTU of 576
          on dynamic routes
    
      4)  Various problems may occur if the routing table is
          over populated.  OMPROUTE can handle approximately
          1000-2000 routes with tracing active.  After 2000
          routes, OMPROUTE performance will suffer.  Factors
          including number of routes, system load, HFS file I/O
          rate (if tracing is active), and amount of traffic
          generated to create the 2000+ routes will all influence
          OMPROUTEs performance.
          It is highly recommended systems running OMPROUTE be
          placed in a OSPF stub area.
    
      5)  If using OSPF on ANY interface, then VIPA should
          be defined as an OSPF_INTERFACE attached to the
          same OSPF area as a physical interface.  If not
          using OSPF, then VIPA interfaces should be defined
          as INTERFACE (even for RIP).  Not following this
          recommendation may result in various routing problems.
          Also note that in OSPF environments, VIPA can be defined
          as an INTERFACE and will be included in OSPF if
          AS_BOUNDRY_ROUTING import_direct_route=yes is coded.
          In this case, the VIPA will appear as an AS External
          route which may or may not be desirable.
    
      6)  OMPROUTE display commands are not working,
          and OMPROUTE appears to be looping using high
          CPU.  This may lead to adjacency loss and loss of
          connectivity.  Loss of connectivity to the VIPA
          is reported often when this problem is encountered.
          Verify that LE apar PQ61992 is applied.
    
      7)  Stack or Network Hardware Problems
          A problem in OMPROUTE can be the first sign of a stack or
          network hardware problem.  If such problems are suspected,
          the diagnostic approach used by L2 support will be to
          isolate the layer where the problem is occurring.  The
          layers that may need to be examined include:
    
          LAYER                  BEST DOCUMENTATION
          -----------------------------------------------
          APPLICATION            OMPROUTE trace
          PRESENTATION (USS/OE)  CTRACE COMP=SYSOMVS,OPTIONS=(ALL)
          SESSION     (sockets)  CTRACE COMP=SYSTCPIP,OPTIONS=(PFS,
                                 IOCTL,SESSION)
          TRANSPORT   \          CTRACE COMP=SYSTCPIP,OPTIONS=(UDP,
                        (stack)         RAW)
          NETWORK     /          CTRACE COMP=SYSTCPIP,OPTIONS=(
                                        INTERNET)
          DATA LINK (VTAM/IOS)   CTRACE COMP=SYSTCPIP,OPTIONS=(
                                 LCS,VTAM,VTAMDATA)   and/or
                                 a VIT trace (for VTAM) and/or
                                 a CCW trace (for channel devices)
          PHYSICAL (hardware)    sniffer trace and/or external
                                 hardware displays/traces.
          NOTE:
          1)  Contact L2 before obtaining any of the above
              documentation.  L2 will provide a more tailored
              documentation request based on the symptoms of the
              problem.
          2)  ALL of the above documentation would rarely be
              required.
          3)  All CTRACEs should be filtered on the TCPIP and
              OMPROUTE jobnames.
          4)  For suspected hardware problems, the time to problem
              resolution can expedited by engaging both software
              and hardware support. Note that the diagnostic
              approach used by Software support will be to first
              eliminate any suspected software problems.
      8) OMPROUTE address space is being cancelled every 5 minutes
         by TCPIP.  For problem determination, refer to technote
         1192776 at this URL:
         http://www-1.ibm.com/support/docview.wss?&uid=swg21192776
    
      9) The STDENV dataset must be VB.  If it is not,
              a) ABEND0C4 or U4083  occurs when issuing
                 F OMPROUTE,GENERIC commands
              b) when running OMPROUTE traces, no data is captured
                 in the trace files.
    
    GENERAL INFORMATION:
      1)  Meaning of interface capability flags on EZZ7883I message.
          The flags are a bitmask representation of the network
          capabilities of the interface:
          0x01     Interface is up (required)
          0x02     Interface is broadcast capable
          0x04     Interface driver is in debug mode (rarely seen)
          0x08     interface is in loopback only mode
          0x10     Interface is a Point-to-Point interface
          0x20     Interface does not support Trailer encapsulation
          0x40     Interface is running (required)
          0x80     Interface is ARP incapable (cannot ARP)
          0x100    Interface in promiscuous mode (sees ALL frames)
          0x200    Interface is receiving all multicast packets
          0x400    Interface is multicast capable
          0x800    Interface is point-to-multipoint (P2MP)
          0x1000   Interface supports Token Ring bridging
          0x2000   Interface supports extended SAP
          0x4000   Interface is a VIPA
          Common capability flags:
          C51 = 0x800 + 0x400 + 0x40 + 0x10 + 0x01 =
                P2MP  + multicast + running + P2P + up
                C51 is often seen on CIP/CLAW devices
          463 = 0x400 + 0x40 + 0x20 + 0x03 =
                Multicast + running + notrailers + broadcast + up
                463 is often seen in multi-access broadcast capable
                interfaces (10/100 Mb Ethernet, T/R, etc)
          4041= 0x4000 + 0x40 + 0x01 = VIPA + running + up
                4041 are VIPA interfaces
    
      2)  OROUTED to OMPROUTE conversation tool information
          In z/OS 1.2, OROUTED has a new command line option (-c)
          that creates a sample OMPROUTE configuration file from
          the information found in: OROUTED profile, OROUTED
          gateway dataset, and PROFILE.TCPIP.  It is important to
          note that the resulting OMPROUTE configuration file
          generated by the OROUTED -c tool is only a sample.  As
          documented, the conversion tool will generate an OMPROUTE
          configuration file that must be reviewed by the user.
          Further user action may be required, and is usually noted
          in comments in the sample OMPROUTE configuration file.
    .
    NOTE: For z/OS Commserver hints and tips go to:
    http://www.ibm.com/software/network/commserver/support/
    

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

  • cancel
    

APAR Information

  • APAR number

    II12026

  • Reported component name

    PA LIB INFO ITE

  • Reported component ID

    INFOPALIB

  • Reported release

    001

  • Status

    CLOSED CAN

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    1999-08-22

  • Closed date

    1999-12-09

  • Last modified date

    2010-09-14

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

Applicable component levels

[{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SG19N","label":"APARs - OS\/390 environment"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"001","Edition":"","Line of Business":{"code":"","label":""}},{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SG32M","label":"APARs - VSE\/ESA environment"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"001","Edition":"","Line of Business":{"code":"","label":""}},{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SG19M","label":"APARs - z\/OS environment"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"001","Edition":"","Line of Business":{"code":"","label":""}},{"Business Unit":{"code":null,"label":null},"Product":{"code":"SG19O","label":"APARs - MVS environment"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"001","Edition":"","Line of Business":{"code":"","label":""}},{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SSSN3L","label":"z\/OS Communications Server"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"001","Edition":"","Line of Business":{"code":"LOB35","label":"Mainframe SW"}},{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SG27M","label":"APARs - z\/VM environment"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"001","Edition":"","Line of Business":{"code":"LOB16","label":"Mainframe HW"}}]

Document Information

Modified date:
14 September 2010