IBM Support

MustGather: Collect Troubleshooting data, Read First/Fix List for TCP/IP SOCKET API on z/OS Communications Server

Troubleshooting


Problem

This document describes problem diagnostics, recommended maintenance, and miscellaneous information for the SOCKET API on z/OS IP Communications Server. This information was previously maintained in information APARs II11996, II12136 and II12449.

Resolving The Problem

Table of Contents

Common Problems

Gathering Diagnostic Output

Recommended Maintenance

Socket Connection State Descriptions

Common Problems:

Problem: RC=-1 , ERRNO=positive number (other than 0)
  • RC=-1 ERRNO=48 (EADDRINUSE)
    This error occurs on a BIND socket call when the port that
    the application is being bound to is already in use by
    another application.

    Analysis:
    Verify that the port is reserved in the TCPIP profile with the correct job name.
    Use NETSTAT, and verify that the port is not used by another application.

  • RC=-1 ERRNO=54 (ECONNRESET)

    This error occurs on a receive type socket call when the
    connection with the remote host has been reset by the peer
    host.

    Analysis:
    Obtain an IP Packet Trace, as per technote #1292013


  • RC=-1 ERRNO=60 (ETIMEDOUT)

    This error occurs on a CONNECT socket call when the connection
    to another host fails because TCPIP on the other host fails to
    acknowledge the connect request before the 3-minute connection
    establishment timer expires.
    Analysis:
    Obtain a sniffer or packet trace of the failing situation to determine whether TCPIP acknowledged the connection request. See technote #1292013 for instructions on gathering a packet trace.
  • RC=-1 ERRNO=61 (ECONNREFUSED)
    This error occurs on a CONNECT socket call when the connection
    to another host fails because no application is listening on
    the port that your application is trying to connect to.

    Analysis:

    - Determine whether the application you are connecting to
    has issued a LISTEN for the port to which the client
    application is attempting to connect

    - NETSTAT CONN can help determine whether the port that
    you are trying to connect is in the listen state.

  • RC=-1 ERRNO=9 OR ERRNO = 113 (EBADF)

    This error occurs on a GIVESOCKET, SELECT, or TAKESOCKET call
    when the descriptor that the application is managing is
    invalid.
    Analysis:

    - Determine why the descriptor is invalid.

    - In the case of the GIVESOCKET, ensure that the socket
    being given is a descriptor obtained either with the
    SOCKET or TAKESOCKET call.

    - In the case of the TAKESOCKET, ensure that the socket being
    taken has been given away previously with the GIVESOCKET call

    - When using the CICS Listener provided with the CICS Sockets
    Interface:
      • A new connection request causes the listener to issue EXEC CICS START to start a child task using the received transaction ID. It passes data that includes an 8-byte listener task name.
      • The listener then issues a GIVESOCKET to transfer the socket to the child task.
      • The child task retrieves the data passed by the listener with an EXEC CICS RETRIEVE and then issues a TAKESOCKET that includes the exact task name passed from the listener.
      • If the listener task name does not match, then the TAKESOCKET call will fail with EBADF.
      [The above problem could also be caused by a C program using (Copy String) STRCPY () or STRNCPY() to copy the task name. Using MEMCPY with length of 8 should be used instead.]

    - In the case of SELECT, ensure that any socket descriptor
    specified in the descriptor set is a socket obtained either
    with the SOCKET ,TAKESOCKET or ACCEPT call.

    - Obtain a TCPIP CTRACE, filtering on the application job name, to determine which descriptor is invalid
    TRACE CT,ON, COMP=SYSTCPIP,SUB=(tttt)
    r xx,WTR=xxxx,JOBNAME=(application jobname),OPTIONS=(SOCKAPI, PFS, TCP,SOCKET), END
    where xxxx is your writer proc name and tttt is the TCPIP job name

  • RC=-1 ERRNO=22 or ERRNO = 121 (EINVAL)
    This error can occur on any socket call when an incorrect
    argument for the socket call is specified.
    Analysis:

    - Determine which parameter is invalid.

    - In the case of the ACCEPT call, ensure that the SOCKET,
    BIND and LISTEN calls have been issued correctly and
    that the ACCEPT call uses the file descriptor used
    in the BIND and LISTEN call.

    - In the case of the BIND call, ensure that the socket
    used in the BIND call is not already bound to another
    port.

    - In the case of the CONNECT call, ensure that the length
    of the NAME parameter is correct and that all of the
    NAME parameters are specified correctly.

    - In the case of the GIVESOCKET and TAKESOCKET calls,
    ensure that the CLIENTID parameter is specified correctly

Gathering Diagnostic Output

Symptom: RC= -1 ERRNO = POSITIVE NON-ZERO VALUE
    • TCPIP Component Trace, filtering on the application job name
      TRACE CT,ON, COMP=SYSTCPIP,SUB=(tttt)
      r xx,WTR=xxxx,JOBNAME=(application jobname),OPTIONS=(SOCKAPI, PFS, ENGINE,TCP,SOCKET), END
      where xxxx is your writer proc name and tttt is the TCPIP job name
    • If using the CICS sockets interface, include a dump of the CICS REGION

Symptom: TCPIP application not working

    • TCPIP Component Trace, filtering on the application job name
      TRACE CT,ON, COMP=SYSTCPIP,SUB=(tttt)
      r xx,WTR=xxxx,JOBNAME=(application jobname),OPTIONS=(SOCKAPI, PFS, ENGINE,TCP,SOCKET), END
      where xxxx is your writer proc name and tttt is the TCPIP job name

    • IP Packet trace, documented in technote #1292013
    • OMVS Component Trace with OPTION=ALL
    • If using the CICS sockets interface, include a dump of the CICS REGION



Symptom: Application does not receive data

    • TCPIP Component Trace, filtering on the application job name

      TRACE CT,ON, COMP=SYSTCPIP,SUB=(tttt)
      r xx,WTR=xxxx,JOBNAME=(application jobname),OPTIONS=(SOCKAPI, PFS, ENGINE,TCP,SOCKET), END
      where xxxx is your writer proc name and tttt is the TCPIP job name
    • IP Packet trace, documented in technote #1292013

Symptom: Application failed to connect to another client or server

    • TCPIP Component Trace
      TRACE CT,ON, COMP=SYSTCPIP,SUB=(tttt)
      r xx,WTR=xxxx,OPTIONS=(SOCKAPI, PFS, ENGINE,TCP,SOCKET), END
      where xxxx is your writer proc name and tttt is the TCPIP job name

    • IP Packet Trace filtered on the remote IP Address, documented in technote #1292013

Back to top


Recommended Maintenance

High Impact (Hiper) TCP/IP Socket Interface APARs
PI57645
R220 UI36082INCORRECT SO_RCVTIMEO AND SO_SNDTIMEO PROCESSING IN V2R2
PI52347R220 UI34846 EZASMI TYPE=CANCEL CALL NOT TAKING EFFECT
PI46678
R220 UI33614 INCORRECT PROCESSING OF IOVS FOR READV/WRITEV/
SENDMSG/RECVMSG
PM18321R1B0 UK59547
R1C0 UPK59548
Abend S0C4 in module EZBSOUTL or EZBITKSS during GIVESOCKET/TAKESOCKET
processing. Can also manifest as hung CICS sockets applications.
CICS Socket Interface Hiper APARs
PI57216
R220 UI35917Improve CICS performance
PM10451R1B0 UK56202CICS ends with ABEND878 when it has many CICS Sockets subtasks (NTASKS).
PM09427R1B0 UK55530CICS Listener unexpectedly ends with message EZY1308E.
PM23649R1B0 UK61653
R1C0 UK61654
Abend702 AKEB during CICS region shutdown

Back to top

Socket Connection State Descriptions

  • ACTIVE OPEN (local application initates connection establishment):


  •       SYN  OUT   SEQ 1480267197 ACK          0  STATE    SYN_SENT   
       ACKSYN   IN   SEQ 2414265383 ACK 1480267198  STATE    SYN_SENT   
          ACK  OUT   SEQ 1480267198 ACK 2414265384  STATE ESTABLISHED   
    SYN_SENTThe local application has issued a socket open call. This results in a SYN packet flowing to the remote stack. The local stack then waits for an ACKSYN packet. The connection remains in SYN_SENT state when this packet is received.
    ESTABLISHEDFollows from SYN_SENT state when the local stack sends an acknowledgement for the received ACKSYN packet, completing the three-way handshake. Both sides can now send and receive data.

  • PASSIVE OPEN (remote application initiates connection establishment):


  •                                                 STATE      LISTEN  
           SYN  IN   SEQ    2817827 ACK          0  STATE    SYN_RCVD  
        ACKSYN OUT   SEQ 2529238365 ACK    2817828  STATE    SYN_RCVD  
           ACK  IN   SEQ    2817828 ACK 2529238366  STATE ESTABLISHED   
    LISTENWaiting for a connection request from the address and port listed in the Foreign Socket column of NETSTAT.
    • "HOSTA..*" means waiting for a connection request from any port on host HOSTA.
    • "*..100" means waiting for a connection request from port 100 on any host.
    • "*..*" means waiting for a connection request from any port on any host.
    SYN_RCVDThe remote application has issued a socket open call. This results in a SYN packet flowing to the local stack. When the SYN packet is received by the local stack, an ACKSYN is sent back to the remote application. The connection remains in SYN_RCVD state when this packet is sent.
    ESTABLISHEDFollows from SYN_RCVD state when the local stack receives an acknowledgement for the sent ACKSYN packet, completing the three-way handshake. sides can now send and receive data.

  • ACTIVE CLOSE (local application initiates connection termination):

  •  ACKPSHFIN OUT   SEQ 1187839881 ACK  956151930  STATE  FIN_WAIT_1   
           ACK  IN   SEQ  956151930 ACK 1187839882  STATE  FIN_WAIT_2   
        ACKFIN  IN   SEQ  956151930 ACK 1187839882  STATE  FIN_WAIT_2   
        ACKPSH OUT   SEQ 1187839882 ACK  956151931  STATE   TIME_WAIT   
    FIN_WAIT_1Follows from ESTABLISHED state when the local application issues a socket close or shutdown. This results in a FIN packet flowing to the remote stack. The local stack is now waiting for an acknowledgment of the FIN.
    FIN_WAIT_2Follows from FIN_WAIT_1 state when the local stack receives an acknowledgment for the FIN that was sent. The acknowledgment number must be 1 greater than the sequence number of the sent FIN to cause the state transition.
    CLOSINGFollows from FIN_WAIT_1 state when the local stack receives an acknowledgement for the FIN that was sent and the FIN flag is also set (simultaneous close). The acknowledgment number must be 1 greater than the sequence number of the sent FIN to cause the state transition.
    TIME_WAITFollows from FIN_WAIT_2 state or CLOSING state when the local stack receives a FIN and sends an acknowledgement. The acknowledgement number must be 1 greater than the sequence number of the received FIN to cause the state transition.
    CLOSEDFollows from the TIME_WAIT state when 2MSL (twice the maximum segment lifetime) has passed (MSL = 60 seconds).



  • PASSIVE CLOSE (remote application initiates connection termination):


  •    ACKPSHFIN  IN   SEQ 3661816120 ACK 3656855726  STATE   ESTABLISHED   
             ACK  OUT  SEQ 3656855726 ACK 3661816121  STATE   CLOSE_WAIT   
       ACKPSHFIN  OUT  SEQ 3656855726 ACK 3661816121  STATE   LAST_ACK   
          ACKPSH  IN   SEQ 3661816121 ACK 3656855727  STATE   CLOSED   
    CLOSE_WAITFollows from ESTABLISHED state when the remote application issues a socket close or shutdown. This results in a FIN packet flowing from the remote application. The local stack must receive the FIN and acknowledge it. The acknowledgement number must be 1 greater than the sequence number of the received FIN to cause the state transition.
    LAST_ACKFollows from CLOSE_WAIT state when the local application issues a socket close or shutdown. This results in a FIN packet flowing to the remote stack.
    CLOSEDFollows from the LAST_ACK state when the local stack receives an acknowledgement for the sent FIN. The acknowledgment number must be 1 greater than the sequence number of the sent FIN to cause the state transition.


    Back to top

[{"Product":{"code":"SSSN3L","label":"z\/OS Communications Server"},"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Component":"--","Platform":[{"code":"PF035","label":"z\/OS"}],"Version":"1.11;1.12;1.13;2.1;2.2","Edition":"","Line of Business":{"code":"LOB35","label":"Mainframe SW"}}]

Document Information

Modified date:
15 June 2018

UID

swg21385164