IBM Support

Rational Build Forge "Jobs Died Unexpectedly" or "JVM out of Memory Error"

Technote (troubleshooting)


Problem(Abstract)

One of two things happens.
1. Jobs are dying without any sufficient explanation. Nothing in the step log or Catalina/db.log. Just an indication that the job has died unexpectedly.

2. Build Forge JVM out of memory error causes Build Forge to crash. It must be restarted.

The solution for both of these issues is the same fix (typically)

Cause

Jobs "Died Unexpectedly" This error can occur because the database is not able to receive enough connection. For Example: MySQL default database connections is 100. Build Forge should have a minimum of 200 connections coming from Build Forge | Entering the Database.

JVM out of Memory Error" This can be caused because Build Forge is not making enough connections to the database and or the database does not allow enough connections. The perl processes that are scheduled to be committed into the database, begin to build up. Since there are not enough connections the processes build up and the JVM grows and eventually run out of memory. Increasing the JVM memory is only a temporary fix.


Environment

This has been seen in MySQL and Oracle.

Diagnosing the problem

Navigate to the Build Forge Home directory and find the db.log. Inside this file, look for the two following errors.

Tue Mar 26 00:51:52 2013: CRRBF1253I: 5609: Build Forge: Process for id [9DE87694-95D6-11E2-93DC-94E617AF7435] [B] died unexpectedly.

OR

Java.Lang Exception Java Virtual Machine OOM.

Resolving the problem

MYSQL

Check the number of available connections. If it is not set to 200. Set it to 200

Check Connection:
show variables like "max_connections";

Set Connections
set global max_connections = 200;

This is setting the global setting. This means you do not have to restart the database before this takes effect. Makes it good for testing purposes. Remember that if the DB is restarted, this value is lost and it is set back to the originally set value.

Build Forge:
Find the Build Forge home directory and navigate to the "buildforge.conf: file. In this file there is a parameter called "db_pool". Set this parameter to =200. This will ensure that Build Forge is making 200 connections.

ORACLE
Set Connections:
ALter system set sessions=500 scope=spfile;

Set the connections to 200.

Build Forge:
Find the Build Forge homedirectory and navigate to the "buildforge.conf: file. In this file there is a parameter called "db_pool". Set this parameter to =200. This will ensure that Build Forge is making 200 connections.


NOTE: These values can be set higher. If you have them set at 200 and have periods of very high volume, these values can be set to 500. If the issue still occurs after setting both values to 500. You will need to open up a PMR with Rational Build Forge Support.

Document information

More support for: Rational Build Forge Enterprise Edition
Database: All Databases

Software version: 7.1

Operating system(s): AIX, HP-UX, Linux, Solaris, Windows

Software edition: All Editions

Reference #: 1634022

Modified date: 11 April 2013


Translate this page: