Analyze the output for other waiting threads

The threads that are waiting but not involved in any mount latch or file latch contention are listed last in the message BPXO063I under the OTHER WAITING THREADS heading. In case of resolving latch contention, it might be helpful to know why these threads are waiting and how to release them from the wait status. For example, refer to Figure 1.
Figure 1. Example: OTHER WAITING THREADS section
OTHER WAITING THREADS: 
  USER     ASID   TCB         PID           AGE
--------------------------------------------------
  USER01   0021 00908070     1234         00:12:41    
    IS DOING: NFS Readdir / Running             
    FILE: nfsdirname                    (33,5432)
    FILE SYSTEM: HOST12.AJAX.DIRECTORY                          
    HOLDING: File System Latch #123 SHR
  USER03   0041 00908070   786534         00:12:41    
    IS DOING: BRLM Wait 
    FILE: FileNameIsHere                (22,845)    
    FILE SYSTEM: AJAX.DS88.ZFS                           
  USER04   0051 00908070       15         00:00:49    
    IS DOING: File Latch Wait-Latch 1379              
    FILE: somefilename                  (88,1234)
    FILE SYSTEM: HOST12.AJAX.DIRECTORY                          
    HOLDING: File System Latch #123 SHR
  USER05   0071 00908070   378992         00:08:51    
    IS DOING: ZFS Write / OSI_WAIT 
    FILE: zfsfilename                       
    FILE SYSTEM: AJAX.DS23.ZFS
The IS DOING field as highlighted in Figure 1 tells what the waiting task (USER) is doing at the time of display. In addition to the waiting type discussed in previous sections, other possible types include:
File Share Wait
The thread is suspended during the open() of a file because the NFS server has placed a share reservation on the file. Issue the MODIFY mvsnfs,listlock= command to show the share reservation that NFS server has on the file, and the MODIFY mvsnfs,release= to break the reservations if necessary. For more information on the commands, see z/OS Network File System Guide and Reference.
BRLM Wait
The thread is waiting to obtain a byte range lock on a file. Take the following steps for diagnosis:
  1. Issue the DISPLAY OMVS,A=ALL command to get the PID or PIDs corresponding to the address space ID of the thread.
  2. Issue the DISPLAY OMVS,PID=www,BRL command to show the threads in the byte-range lock wait. (In this case, the PID obtained in step 1 for the waiting process is www.) The BRLWAIT line in the message shows the PID of the process that is blocking this waiting process from obtaining the lock.
  3. If the blocking process also appears in other part of the waiters display, continue the diagnosis procedure there.
  4. If the blocking process resides on another system within the sysplex, use the system name in the BRLWAIT line and continue the diagnosis procedure there.
  5. On the system where the blocking process resides, issue the DISPLAY OMVS,PID=bbb,BRL command to show the name of the program or command that is running. (In this case, the PID obtained in step 2 for the blocking process is bbb.)

    Note that the blocking process might not be hung, or at least not hung for z/OS® UNIX Services reasons. Programs can obtain byte range locks and hold them indefinitely.

  6. If the blocking process is the NFS server, the command shown is GFSAMAIN. In this case the lock was obtained by some NFS client. You can use MODIFY mvsnfs,listlock= and MODIFY mvsnfs,release= commands to solve the problem.
Quiesce Wait
A file system has been quiesced by the BPX1QSE callable service because some backup products are reading the underlying data sets. You can do the following steps to find more information about the file system in problem:
  • Issue the DISPLAY OMVS,FILE command to show the job name and PID of the process that has used BPX1QSE. And if a file system is quiesced for too long, the message BPXF034I THE FOLLOWING FILE SYSTEM HAS BEEN QUIESCED FOR MORE THAN 10 MINUTES: filesystem is issued.
  • A file system can be quiesced while it is being moved or recovered.
    • In a shared file system configuration, issue the MODIFY BPXOINIT,FILESYS=d,exception and MODIFY BPXOINIT,FILESYS=d,FILESYSTEM=filesystemname to show more information of the file systems in moved or recovery state.
    • In either a single system or a shared file system configuration, issue the DISPLAY OMVS,f,exception command.
  • You can forcibly unquiesce a quiesced file system through the File_system menu in the Ishell dialog under ISPF.
zFS xxxxx/ OSI Wait
The thread waiting within in the zFS file system has been quiesced by the zfsadm quiesce shell command, or by a backup product that is reading underlying data sets. Use the zfsadm aggrinfo shell command to show the state of zFS aggregate. Use the DISPLAY OMVS,FILE command to find the aggregate name if it is not the same with the file system name.
File Latch Wait
The thread is waiting for a file latch. The latch number is shown and you can also use the DISPLAY GRS,LATCH,CONTENTION command for the holders. File latches are usually obtained just before it goes into the PFS that supports the file, so the holders information might also be shown in other parts of the waiters display if they are hung up in the PFS.

See Problem diagnosis for shared file system for more information on analysis in a shared file system configuration. Also see the MODIFY BPXOININT,FILESYS=DISPLAY command in z/OS MVS System Commands for information related to discussions here.

Parent topic: Procedure: Diagnosing and resolving latch contention