Intermittent "Error in AFC:!java.lang.OutOfMemoryError!" when running consolidation jobs that include AFCs

Technote (troubleshooting)


Problem(Abstract)

User runs a consolidation (e.g. Consolidate with Status) which includes AFC calculations. After this has finished running, user opens the 'batch queue'. Intermittently, inside the 'batch queue' there is an error message (in the job details).

Symptom

<time> Advanced Formula Calculation is started.

Error in AFC:!java.lang.OutOfMemoryError!
Execution of AFC not OK for: xxxxx
<time + x> Advanced Formula Calculation is finished

NOTE: The error does appear all the time (intermittent issue).


Cause

There are several different possible causes for the generic "Error in AFC:!java.lang.OutOfMemoryError!" message.

  • For more examples, see separate IBM Technote #1636718.

This IBM Technote shall specifically *only* relate to the scenario where the JAVA JRE environment (which hosts the AFC calculation engine) has run out of memory.

More Information:
The default memory limitations of the JRE depends on which version of Controller you are using:
    10.1.1308 (and earlier) :
    This has the following memory limitations:
    • Initial RAM = 256 Mb
    • Maximum RAM = 512 Mb

    10.1.1691 and 10.1.2681 (inclusive) 32-bit version (which is default for these versions)
    By default, the 32-bit version of these version's JRE has its RAM settings 'dynamically' configured.
    • In other words, the JRE will attempt to allocate as much RAM as it needs (using its own algorithm) up to the 32-bit maximum limit (2Gb).

    10.2.0 onwards (plus also 10.1.1691 and 10.1.2681 inclusive 64-bit version, which is optional)
    • Maximum RAM = 512 Mb
    However, this memory limit can be manually increased to match the needs of the customer (see later).

Environment

Problem could potentially affect any environment, but it is much more likely for customers who are:

(1) Using versions of Controller *before* 10.1.1 FP1 (10.1.1671)

(2) Running multiple batch process servers.

  • For example, in the following real-life example the customer is running 3:

  • The intermittent nature of the problem is explained by the fact that (randomly) there may have multiple concurrent AFC jobs running that swamp the available memory for the JAVA JRE environment.

Diagnosing the problem

Launch Task Manager on the controller application server, and search for the process 'CCRProxy.exe'.

  • You should notice that its memory used increases (over time)

By default (for most Controller versions) CCRProxy.exe will reach a limit of approximately 650Mb before the problem occurs. This limit is because:
  • By default, most Controller version's JRE has a maximum limit of 512Mb
  • However, there will also be a small amount of extra RAM utilised by 'CCRProxy.exe' and 'jvm.dll', which will then take the total up to around 650Mb or so.

Resolving the problem

Fix:

If necessary, upgrade to the latest version of Controller (ideally 10.2 or later).

  • Afterwards, increase the JAVA maximum memory limit (by creating/modifying the file 'CCRProxy.options').

Steps:

1. If necessary, upgrade to the latest version of Controller (ideally 10.2 or later).

  • TIP: Controller 10.2 automatically comes with a 64-bit JAVA proxy

2. If upgrading to a version between 10.1.1691 and 10.1.2681 (inclusive) then modify the JAVA proxy from 32-bit to 64-bit
  • For instructions, see separate IBM Technote #1618609.

3. If necessary, increase the amount of RAM on the Controller application server. In other words, please ensure that your Controller application server has enough free RAM (physically installed in the server) to cope with reserving this amount for your JAVA service.
  • As a general guide, make sure that the Controller application server has at least twice the RAM that you specify inside "CCRProxy.options" (see later).

4. Logon to the Controller application server as a Windows administrator
5. Browse to the folder ...\server

  • TIP: By default, this will be in a location similar to: C:\Program Files\IBM\cognos\ccr_64\server

6. Check to see if the file CCRProxy.options exists.
  • By default this file will not exist. If it does exist then (as a precaution) create a backup of the file (for example, copy it to CCRProxy.options.backup)

7. Save the relevant file CCRProxy.options (attached to this Technote) inside that folder

TIP: Several example files are attached to this Technote. Download the appropriate ZIP file (for your needs) and extract the file:
  • 1Gb_maximum.zip = suitable for basic needs only (typically for low-powered server with 32-bit JAVA)
  • 2Gb_maximum.zip = suitable for standard 32-bit JAVA needs
  • 4Gb_maximum.zip = suitable for standard 64-bit JAVA needs
  • 6Gb_maximum.zip = suitable for advanced 64-bit JAVA needs
    More Information: The file ' CCRProxy.options' should contain two parameters, which contains the memory sizes for initial and maximum heap size, for example:
      -Xms512M
      -Xmx1024M

8. Right-click on "My Computer" and choose "Manage"
9. Open the section "Services"
10. Restart the Windows service " IBM Cognos Controller Java Proxy"

Workarounds:

Either:


    (1) Reduce the number of batch services running to 1

    or (2) Use less-complicated JAVA-related functions (e.g. less complicated AFCs).

Related information

1573307 - Slow performance and intermittent "!java.lang
1618609 - How to configure Controller Java subsystem to
1636718 - ** Troubleshooting ** "Error in AFC:!java.lan
JVM 7 - default memory settings

Document information


More support for:

Cognos Controller
Controller

Software version:

10.1.1, 10.2.0, 10.2.1

Operating system(s):

Windows

Reference #:

1616770

Modified date:

2015-07-31

Translate my page

Content navigation