MustGather: Integration issues when using Engineering Workflow Management

Note: Further information to help troubleshoot issues with Rational Team Concert integrations is also available in the Integration troubleshooting section of the Deployment wiki on jazz.net.

AppScan

Rational Build Forge

Rational Team Concert Connector for ClearCase

Rational Team Concert Connector for ClearQuest

ClearQuest Bridge

Eclipse Client

Microsoft Visual Studio Client

Subversion

Git integration

Hudson/Jenkins plug-in

• Verify that the AppScan version used is supported.
• Does the issue occur during the IBM Rational Jazz Team Server (JTS) setup?
• Does the issue occur when a user logs in?
• What error messages are seen (errors from web UI, log files or others - include screenshots)?
• What version of Jazz Team Server is running?
• Is Jazz Team Server running on Apache Tomcat or IBM WebSphere Application Server (provide version)?

Log files to include

• Apache Tomcat log files located in <server install>/server/tomcat/logs (log files include, but are not limited to: catalina, stdout, stderr)
• Apache Tomcat Configuration Files
  server.xml located in <server install>/tomcat/conf
  web.xml located in <server install>/tomcat/webapps/<jts and acs>/WEB-INF
• Jazz Team Server log files located in <server install>/server/logs
• Jazz Team Server teamserver.properties file, located in <server install>/server/conf

Enabling LDAP Trace

Before enabling LDAP Trace consider searching CRJAZ2797E messages in jts.log. The error description next to that message can help you in determining the root cause of the issue.

Note: LDAP Trace is already included out of box, but needs to be turned on. Turning on LDAP Trace can be costly to the performance of the server and should be used in troubleshooting, but turned off during normal server use.

1. Open the JTS log4j.properties file, located in <server install>/server/conf/jts
    
2. Remove the # sign in front of the following log4j lines to enable the trace:
   
   log4j.logger.com.ibm.team.repository.service.jts.internal.userregistry.ldap.LDAPUserRegistry=DEBUG
   log4j.logger.com.ibm.team.repository.service.internal.scheduler=DEBUG
    
3. Save the file
    
4. The trace will come into effect during the next restart of the server. To avoid restarting the server, navigate to the following URL:
   
   https://<server>:9443/jts/admin?internal=true#action=com.ibm.team.repository.admin.reloadLoggingSettings
    
5. Run the LDAP tests. The LDAP trace will be found in the jts.log file.
   
   Note: If the issue occurs during setup, the only errors that appear will be in the setup wizard. Nothing will be logged in the jts.log file.

• Verify migration mode is off (Scheduler Migration Mode Enabled)
   
• Verify the Rational Build Forge settings
  
  
   
• Verify that the counter menu item shows that the async tasks for Rational Build Forge are incrementing upward. The Counters page is only available from the Internal Admin page.
  
  Example: https://<servername>:9443/ccm/admin?internal=true#action=com.ibm.team.repository.admin.viewCounters

• This will need to be configured under the Rational Build Forge services layer running on either Tomcat or WebSphere Application Server (WAS)
  
  com.ibm.rational.connector.buildforge.level=ALL
  
  Tomcat: <TOMCAT_HOME>/conf/logging.properties

  The debug setting must be added to the Tomcat logging.properties file.

  Tomcat (and Rational Build Forge) will need to be restarted to take affect.

  
  
  WAS: WAS console UI > Troubleshooting > Logging and tracing > <serverName > Change log detail levels

















 

• Add the following lines to the log4j.properties file (../server/ccm/conf)
  
  log4j.logger.com.ibm.rational.buildforge=DEBUG

• Reload the log4j.properties file by accessing the following link:
  
  https://<server>:9443/ccm/admin?internal=true#action=com.ibm.team.repository.admin.reloadLoggingSettings
  
  (This will avoid having to restart the server)
   
• Reproduce the issue and provide the ccm.log file as well as the Rational Build Forge Log

• Verify that the ClearCase version used is supported.
   
• Verify synchronization arguments
   
• Verify ClearCase Connector version: refer to the \client\eclipse\.TeamConcertproduct file
   
• Verify that the ClearCase resource (VOB) exists and can be accessed from the Synchronization Host by running the command:
  
  cleartool describe <my_branch>@\<my_vob>
• Collect needed data:
   
  • Full build log (always useful)
     
  • The stack trace where the failure occurred (there can be more than one of these stack trace files - they should all be collected as part of the problem report)
     
  • The list of properties
     
  • The list of synchronized files and folders (such as "sync roots") from before the synchronization attempt. These are only interesting if the synchronization attempt occurred immediately after adding or removing a synchronized file or folder from the synchronization stream.

• Trace all WCM and CC calls. Modify sync stream properties:
   
  • Jazz and ClearCase: LOG_MINIMUM_DURATION=0 - The minimum number of milliseconds that a WVCM call needs to take for it to be logged. If this is set to 0, then all ClearCase calls are traced.
     
  • ClearCase: REPORT_ACTIVITY_MORE_THAN=0 - The minimum number of milliseconds that a ClearCase command needs to take for it to be logged. If this is set to 0, then all ClearCase calls are traced.
     
  • Turn on ClearCase Connector Logging: follow instructions detailed on the CCC Team Wiki.

• Verify that the ClearQuest version used is supported.
   
• Collect and verify the cqconnector.properties located in /gateway
   
• From the machine where the ClearQuest Gateway is running, verify that the ClearQuest DB can be accessed from a browser using credentials as stored in the cqconnector.properties
   
• From the machine where the ClearQuest Gateway is running, verify that the IBM Jazz Team Server can be accessed from a browser using credentials as stored in the cqconnector.properties
   
• Verify the ClearQuest Gateway was started and check the gateway status:
  
  -http://localhost:8081/cqconnector/gateway
  
  Note: The gateway status should display “Accepting ALL Requests”. If the status is accepting only setup or reset requests, the setup was not completed correctly and should be re-configured.

• Verify the URL defined in the external connection can be reached using a Web Browser
  
  Note: According to the documentation, the http connection should point to port 8081, whereas the SSL connection should use https syntax along with port 8444
   
• Verify that Project Area(s) below the External Connection have sync rules attached.
   
• Verify that each sync rule can be opened and edited without warning or errors (it fetches information from ClearQuest)
   
• Verify that the cqconnector user has both the ClearQuest connector CAL and synchronization permissions within the project area
   
• To trace the Synchronization, add the following line in the log4j.properties and reload the
  log4j.properties in the server log4j.logger.com.ibm.team.interop=DEBUG
  
   

• Stop the CQ Gateway - rotate the gateway log - copy cqgateway.log to cqgateway.log.old
   
• Edit the CQ Gateway log4j.properties file (located in cq_gateway_installdir/gateway)
  
  Ensure the lines below are uncommented:
  
  log4j.rootLogger=DEBUG, stdout
  log4j.logger.com.ibm.rational.interop.level1=DEBUG, cqgateway
  
  Then save your changes
   
• Start the CQGateway and reproduce the issue.
   
• Collect the cqgateway.log file located in the \gateway\tomcat\logs directory

• Screenshot of advanced properties Bridge Properties section

The Eclipse log is stored in the workspace directory under ../metatdata and is named .log

• When a problem arises in the Eclipse client, there are additional traces and must gathers depending on the component where the problem occurred (such as SCM, process customization, and others).

• Verify .NET http://msdn.microsoft.com/en-us/kb/kb00318785.aspx
   
• Run the following tool to determine if there are multiple versions installed:
  
  http://www.asoft.be/prod_netver.html

• EWM system requirements

• Get the trace file located in IBM\Rational\Team Concert Visual Studio Client.
  
  For example: C:\Documents and Settings\ApplicationData\IBM\Rational\Team Concert Visual Studio Client
  
  Note: Trace files are time stamped
   
• Get the RTC log data: To launch it, navigate to Team Concert->Windows->RTC Log.

• Refer to this MSDN article.
• More information in the Escalation Request.

• Screenshot of the encoding type selected on the SVN Import wizard page 2
   
• From client, request Dump File, or a copy of some SVN revisions (as described here) to try to reproduce the issue.

1. Stop the EWM Eclipse Client
    
2. Copy the sample ".options" file attached below to (for instance, the same directory as the "TeamConcert.exe" or "eclipse.exe")
   
    
3. Modify the shortcut that launches the EWM Client (or create a new one by copying and pasting the existing one) and modify the target line:
   
   eclipse.exe -debug \.options –consolelog
    
4. Start EWM; you should now see a console window. When you do an import, information will be written to that window.
    
5. You should increase the screen buffer height of the window (such as: the number of lines), by clicking on the upper left icon and choosing Properties from the drop down window.
    
6. Click on the Layout tab and increase the screen buffer height to a larger number (such as: 9999).
    
7. Perform the import and send the copy of the console output.