IBM Support

Intermittent "Error in AFC:!java.lang.OutOfMemoryError!" or "Error in AFC: !Java heap space!" when running consolidation jobs that include AFCs

Troubleshooting


Problem

User runs a consolidation (e.g. Consolidate with Status) which includes AFC calculations. After this has finished running, user opens the 'batch queue'.
Inside the 'batch queue' there is an error message (in the job details).
  • Depending on the circumstances, the error may be intermittent.

Symptom

Example #1
<time> Advanced Formula Calculation is started.
Error in AFC:!java.lang.OutOfMemoryError!
Execution of AFC not OK for: xxxxx
<time + x> Advanced Formula Calculation is finished

NOTE: The error does appear all the time (intermittent issue).
 

Example #2

image-20190715155316-1

Error in AFC: !Java heap space!

Cause

There are several different possible causes for the generic "Error in AFC:!java.lang.OutOfMemoryError!" message.
  • For more examples, see separate IBM Technote #1636718.

This IBM Technote shall specifically *only* relate to the scenario where the JAVA JRE environment (which hosts the AFC calculation engine) has run out of memory.

More Information:
Modern versions of Controller (from 10.2.0 onwards) all use a 64-bit JRE.

The default (out of the box) maximum RAM value = 512 Mb
  • However, this memory limit can be manually increased to match the needs of the customer (see later).

Environment

Problem could potentially affect any environment, but it is much more likely for customers who are:

(1) Using versions of Controller *before* 10.1.1 FP1 (10.1.1671)

(2) Running multiple batch process servers.

  • For example, in the following real-life example the customer is running 3:
  • The intermittent nature of the problem is explained by the fact that (randomly) there may have multiple concurrent AFC jobs running that swamp the available memory for the JAVA JRE environment.

Diagnosing The Problem

Launch Task Manager on the controller application server, and search for the process 'CCRProxy.exe'.
  • You should notice that its memory used increases (over time)

By default (for most Controller versions) CCRProxy.exe will reach a limit of approximately 650Mb before the problem occurs. This limit is because:
  • By default, most Controller version's JRE has a maximum limit of 512Mb
  • However, there will also be a small amount of extra RAM utilised by 'CCRProxy.exe' and 'jvm.dll', which will then take the total up to around 650Mb or so.

Resolving The Problem

Fix:
Increase the JAVA maximum memory limit (by creating/modifying the file 'CCRProxy.options').
 

Steps:

The following assume that the customer has already upgraded to Controller 10.2 (or later), and still gets the problem:

1. If necessary, increase the amount of RAM on the Controller application server.

  • In other words, please ensure that your Controller application server has enough free RAM (physically installed in the server) to cope with reserving this amount for your JAVA service.
  • As a general guide, make sure that the Controller application server has at least twice the RAM that you specify inside "CCRProxy.options" (see later).
 

2. Logon to the Controller application server as a Windows administrator


3. Browse to the folder ...\server
  • TIP: By default, this will be in a location similar to: C:\Program Files\IBM\cognos\ccr_64\server

4. Check to see if the file CCRProxy.options exists.
  • By default this file will not exist. If it does exist then (as a precaution) create a backup of the file (for example, copy it to CCRProxy.options.backup)

5. Save the relevant file CCRProxy.options (attached to this Technote) inside that folder
  • TIP: Several example files are attached to this Technote. Download the appropriate ZIP file (for your needs) and extract the file:
    • 1Gb_maximum.zip = suitable for basic needs only (typically for low-powered server with 32-bit JAVA)
    • 2Gb_maximum.zip = suitable for standard 32-bit JAVA needs
    • 4Gb_maximum.zip = suitable for standard 64-bit JAVA needs
    • 6Gb_maximum.zip = suitable for advanced 64-bit JAVA needs
  • More Information: The file 'CCRProxy.options' should contain two parameters, which contains the memory sizes for initial and maximum heap size, for example:
    • -Xms512M
      -Xmx1024M

6. Right-click on "My Computer" and choose "Manage"
7. Open the section "Services"
8. Restart the Windows service "IBM Cognos Controller Java Proxy"

Workarounds:

Either:

(1) Reduce the number of batch services running to 1

or (2) Use less-complicated JAVA-related functions (for example, less complicated AFCs).

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS9S6B","label":"IBM Cognos Controller"},"Component":"Controller","Platform":[{"code":"PF033","label":"Windows"}],"Version":"All Versions","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
15 July 2019

UID

swg21616770