IBM Support

Out of threads error when running a performance test

Technote (troubleshooting)


Problem(Abstract)

You are unable to start the required number of transactions when running a performance test on the IBM Rational Integration Tester Agent (RIT-Agent) of IBM Rational Performance Test Server (RPTS).

Symptom

You see the following symptoms.

  • The test console in IBM Rational Integration Tester (RIT) does not show the correct number of started iterations.

  • The RIT-Agent console displays the following error message:

    INFO: TestEngine (0): Already started 300 threads, but none are available.


Cause

The Java engine used to run the tests has 300 internal processing threads available to run parallel test iterations.

If you start tests at a higher rate than the tool completes, the pool of threads is gradually used up and no new tests can start.


A typical performance test uses a request-reply message pattern to test a remote application server.
The cause of this issue is that the system under test cannot process inbound messages quickly enough.


Diagnosing the problem

Consider the following:

  • Check the "started" value displayed in the RIT test console.
    Status messages are displayed every 5 seconds so you should see 5 x iterations per second for each entry.

    For example you run with a setting of 100 iterations, you should see started (500) on each line.

  • Run the RIT-Agent on the command line in order to see its console output.

  • Display Test engine > Threads in use on your results chart.

Resolving the problem

While it is possible to increase the number of threads in the Agent.config file this is not recommended. This action does not solve the underlying problem, when you start tests at a faster rate than they are completing. An increased number of threads also requires an increased management overhead in the Java environment and may actually decrease throughput.

Consider the following guidelines:

  1. Ensure that the test sequence is as simple as possible.
    Remove unnecessary test actions such as debug Log steps.
    Simplify the test actions, you may be able to turn off message validation, for example.

  2. Ensure that the test has a clear and rapid exit path.
    Set timeout values on message Subscribe or Receive Reply actions, and make them as short as possible. It is better to be alerted by tests failing due to timeouts than to run out of threads.
    For example, you are testing at 100 messages per second which is a message every 10 milliseconds, set the timeout value close to this.

  3. Remove iterators from the test as they will result in slow execution and invalid timing information.

  4. Use the Input Mappings tab of the Load Generating Test configuration to map test data into your test.
    Do not use the Test Data Iterator test action.


If the above steps do not resolve the problem, it might be that the target system simply cannot process the number of messages that it receives.
  • Speak to the administrators of the system under test as they may have monitoring tools to diagnose its performance under load.

  • Add probes to the performance test to record system load and message queue depths.

Document information

More support for: Rational Performance Test Server
RIT Load Agent

Software version: 8.0, 8.5

Operating system(s): AIX, Linux, Solaris, Windows

Reference #: 1645303

Modified date: 22 August 2013