MustGather: Remote Journal (RJ) Issues

Resolving The Problem

For issues concerning remote journal issues, this document includes the recommended data collection for debug. This is by no means all inclusive; however, it is meant as a starting point for document collection.

Remote Journal issues specific MustGather information

If you encounter a remote journal issue, collect the following information using either method A or method B as it is related to the issue:
 

MustGather: How to obtain and Install QMGTOOLS and keep it current

Data collection:

1. ADDLIBLE QMGTOOLS

2. GO MG

3. Option 13 - Check IBM for updated QMGTOOLS

4. Option 6 - Database menu

5. Option 4 - Remote journal

6. Set the following parameters:

Journal/Library Name to the source (local) journal and source (local) library name.

Note: If you are using the tool on the target (not recommended - you'd only collect data from that target side and still need to collect data on source side), you should enter the name of the journal and library on the target system.

RDB Associated with Remote System - set this to the name of the Relational Database associated with the remote journal.  You must enter in a value that allows you to TELNET to that IBM i.
*RMTSYS is the default and, if specified, will pass the same name for the RDB as the remote system.
If you do not know the RDB name, it can be determined by the doing the following:

• From the Source system:
  • - WRKJRNA JRN(LIB/JRN)
    - Press F16 and note the relational database name.
    • It's the first column of the remote journal entry we are working on.

User Profile on Remote System - You must enter a valid/enabled user profile on the remote system because data needs to be moved from system to system to complete the collection.

Password of User on Remote Sys - Matching password for the above user profile.

7. Press Enter to set Case information.

8. Put in full Case number and specify FTP option.

    Anonymous is no longer allowed (GDPR) - Supported FTP options (Search for FTP Type)

9. Press Enter to start data collection.

If you find that no remote data is sent in (all files are zero length), you should ensure that the user profile/password can successfully sign on to the remote system.
The FTP_Output.txt file can be reviewed to see what remote commands failed.
You should try to run any failing commands on the remote system using the same user profile.
Still having issues?  Print out the joblog using DSPJOBLOG OUTPUT(*PRINT) and send it in.

NOTE: You can start with Method B but be aware you still may be required to use Method A if requested by IBM Support. 

PTF information:

1. Version, release, and modification level of system (for example, V5R4M5, V6R1M0, and so on).

2. Issue WRKPTFGRP and press Enter; then press F6 to print a spooled file.

3. Issue DSPPTF OUTPUT(*PRINT)

4. Ensure PTFs listed in document N1010584, are applied and ensure that the problem still persists. 

5. PTF levels of the From and To program if an error message is involved:

o Look at the job log to locate the From and To program (example below).
o Issue DSPOBJD OBJ(QSYS/FromPGM) OBJTYPE(*PGM or *SRVPGM) DETAIL(*SERVICE) OUTPUT(*PRINT)
o Issue DSPOBJD OBJ(QSYS/ToPGM) OBJTYPE(*PGM or *SRVPGM) DETAIL(*SERVICE) OUTPUT(*PRINT)

Note: Change the FromPGM and ToPGM to the programs listed in the job log. Use WRKOBJ on the program to determine if OBJTYPE is *PGM or *SRVPGM.
If there is no library listed or program starts with #, skip that one.

Example:
CPD4342 - 40 02/25/10 18:30:53,112435 QQQVALID QSYS *STMT QSQRUN1 QSYS *STMT

The program QQQVALID is the FromPGM.
The program QSQRUN1 is the ToPGM.

DSPOBJD OBJ(QSYS/QQQVALID) OBJTYPE(*PGM) DETAIL(*SERVICE) OUTPUT(*PRINT)
DSPOBJD OBJ(QSYS/QSQRUN1) OBJTYPE(*SRVPGM) DETAIL(*SERVICE) OUTPUT(*PRINT)


Send in Spool files generated by step 2 and 3 (this is not sent by QMGTOOLS).

Data collection:

1. To be done on the source system:

Issue WRKJRNA JRN(LIBNAME/JRNNAME) OUTPUT(*PRINT)
Issue WRKJRNA JRN(LIBNAME/JRNNAME) OUTPUT(*PRINT) DETAIL(*RMTJRN)

NOTE: Replace JRNNAME with Journal name and LIBNAME with Library.

To be done on the target system:

WRKJRNA JRN(LIBNAME/JRNNAME) OUTPUT(*PRINT)

NOTE: Replace JRNNAME with Journal name and LIBNAME with Library.

2. For R540 and R610 send in Total Retransmissions count for 5 Remote Journal Links:

o On the source system's command line, type NETSTAT and select Option 3.
o Press F15 to work with subsets. Type 3777 for the Remote port range Lower value. This gives a list of all Remote Journal connections.
o Press F11 to display byte counts, and press F5 to refresh to see which connections are the most active.
o Select Option 5 by the port to display the details. Scroll down two times; retransmission information will be at the top.

If the value for Total retransmissions is climbing, there is a network issue that can affect Remote Journal. Even if the value for Current Retransmissions is non-zero (yet not climbing), it can still reflect a network problem that is affecting Remote Journal.

3. Date and time of when the problem occurred.

4. Collect on both source and target:

History log:
DSPLOG MSGID(CPC6983 CPC6984 CPF70D3 CPF70D4 CPF70D5 CPF70D7 CPF70DB CPF70DC CPF70C4

CPF70C5 CPI7012 CPI7016 CPE3426 CPE3447) OUTPUT(*PRTSECLVL)

5. Journal flight recorder on both source and target.

From Source: CALL QJODSPAL PARM('RJ' 'JRNNAME' 'LIBNAME')
From Target: CALL QJODSPAL PARM('RJ' 'JRNNAME' 'LIBNAME')

Replace JRNNAME with Journal name and LIBNAME with Library.
This will create a spooled file in the job that runs the command.

Note: Make sure the command used on the target system is for the Remote Journal Type. If you are not sure what the library or journal name is, issue the following command on the source:

WRKJRNA JRN(LIBNAME/JRNNAME)

Replace the LIBNAME and JRNNAME with the source journal. When this is displayed, press F16 to see the remote journal information. The library and journal name listed are the names that should be used on the target system on the CALL QJODSPAL call.

Note: If Journal is in an IASP, use the following command:

CALL QJODSPAL PARM('RJ' 'JRNNAME' 'LIBNAME' 500 33)

Replace JRNNAME with Journal name and LIBNAME with Library.
Replace 33 with the IASP number.
If the IASP number is not known, a DSPLIB on a library in the iASP will provide that information.  

6. Job logs of QDBSRV01, QDBSRV02, and QDBSRV03.

If an IPL has not been performed since the problem occurred, do the following:

o Issue the WRKACTJOB JOB(*SYS) command.
o Press F11 two times.
o Write down the job numbers for each job (QDBSRV01 / QDBSRV02 / QDBSRV03)
o DSPJOBLOG JOB(Job Number/QSYS/QDBSRV01) OUTPUT(*PRINT)
o DSPJOBLOG JOB(Job Number/QSYS/QDBSRV02) OUTPUT(*PRINT)
o DSPJOBLOG JOB(Job Number/QSYS/QDBSRV03) OUTPUT(*PRINT)

If an IPL has been performed, do the following:

o Issue the WRKJOB JOB(QDBSRV01) command and select the job in OUTQ.
o Select Option 4 and review the QPJOBLOG spooled file to determine if the problem happened during the timeframe of the job.
o Repeat for QDBSRV02 and QDBSRV03.

7. Vlogs on both source and target:

A. Issue the STRSST command.
B. Select Option 1. Start a service tool.
C. Select Option 5. Licensed Internal Code log.
D. Select Option 2. Dump entries to printer from the Licensed Internal Code log.
E. Select Dump Option 1 1=Header
F. Press Enter.
G. Select Dump option 3 3=Header and entire entry
H. Specify the date and time of 10 minutes before and 10 minutes after the problem occurred.
I. Press Enter.
J. Press F12 to exit.
K. Select Option 7. Display the status of the Licensed Internal Code log.
L. Refresh until Dump requests not complete is zero.
M. Press F3 three times,and exit out of service tool.Check QSYSOPR for messages.

** All spool files and save file generated from above steps needs to be sent to IBM. **

MustGather: Instructions for Sending Data to IBM i Support