IBM Support

WebSphere Application Server Lightweight Third Party Authentication (LTPA) token life time expiration can cause disk full condition if not set long enough to transfer large data files

Troubleshooting


Problem

When archiving via IBM Content Collector (ICC), the upload fails with the SECURITY_ANONYMOUS_DISALLOWED error on the largest files, causing left over files to fill up the inbound directory space preventing automatic cleanup.

Symptom

Values for Content Retrieval and FileNet.Content.GetBlockSize KB are already set to optimize upload speed as described in the P8 Performance Tuning Guide.

The disk full condition can occur when ICC archiving is running and one or more of the following conditions occurs:
1) The files being stored are large enough to take longer than the LTPA token life time value to copy to the Inbound directory
2) The network is running slowly
3) There is not enough disk space in the Inbound Directory to hold the largest files simultaneously

Cause

The LTPA token life time expiration value in WAS defaults to 2 hours, however it can be set to any desired value up to 10 hours. In the event any of the files take longer than this configured time frame to transfer, the token can become invalid and WAS reports it as an anonymous (unauthenticated) call. This will cause the copy process to be aborted, leaving behind partially copied files for the Content Engine automatic cleanup process to handle.

If enough large files abort, and there is not enough space to hold all of them, the files can build up causing out of disk space which leads to the inability to create the lock file required to remove the left over files when the cleanup is attempted.

Diagnosing The Problem

Errors similar to the two below would be seen in Content Engine logs:

2014-07-05T02:34:17.142Z 53320797 ENG  FNRCS0001E - ERROR method name: <init> principal name: myUserId/myPassword Global Transaction: null User Transaction: null Exception Info: Access to Content Engine was not allowed because the request was made anonymously instead of by an authenticated user. The application server reported that the user was anonymous.

2014-07-05T02:26:32Z    Error    An error occurred while evaluating the task route 'MyTaskRouteName':
Task Method 'ibm.ctms.p8connector.p84x.ArchiveTask' failed for entity with id
'\\fileshare01\legal_hold$\cd04\backup\cd04$pd_uvrump01\loadactive\full\cd04$pd_ump01.bak':
Status=error; Message='SECURITY_ANONYMOUS_DISALLOWED : Access to Content Engine was not allowed because the request was made anonymously instead of by an authenticated user.
The application server reported that the user was anonymous.


To determine if there is not enough disk space to acquire a lock preventing the cleanup check Content Engine error logs for the following errors:

2014-07-10T14:48:00.988Z D7902F9C CSTG FNRCE0000W - WARN ContentQueueDirectoryCleanUp.acquireLock: received unexected exception trying to acquire lock to cleanup inbound directory.  Skipping cleanup for this interval.
java.io.IOException: There is not enough space on the disk.
    at java.io.File.createNewFile(File.java:894)

2014-07-05T13:05:32.703Z CC90DE17 ENG  FNRCC0132E - ERROR method name: write principal name: m37382 Global Transaction: false User Transaction: false Exception Info: A document could not be created, checked into a library, or moved from one storage area to another. State: [stateId={C956AB27-ECF4-46EA-A3E8-A96D3BA5232D}; actualObjectId={5465F178-ED05-4614-BDAA-56DB29105E6E}; tempId={197E04FD-A7E2-4A44-849E-BFD75CDA0B0A}; initialOffset=0; lastOffset=0]
com.filenet.api.exception.EngineRuntimeException: FNRCC0132E: CONTENT_PC_WRITE_FAILED: A document could not be created, checked into a library, or moved from one storage area to another. State: [stateId={C956AB27-ECF4-46EA-A3E8-A96D3BA5232D}; actualObjectId={5465F178-ED05-4614-BDAA-56DB29105E6E}; tempId={197E04FD-A7E2-4A44-849E-BFD75CDA0B0A}; initialOffset=0; lastOffset=0]
...
Caused by: java.io.IOException: There is not enough space on the disk.
    at java.io.FileOutputStream.writeBytes(Native Method)

Resolving The Problem

Ensure you have checked the values for Content Retrieval and FileNet.Content.GetBlockSize KB are already set to achieve optimal performance for your environment as described in the P8 Performance Tuning Guide.

Calculate and set the LTPA timeout value that best covers the needs of your business.

To determine the ideal setting for the LTPA timeout value in your environment you need to obtain the average network speed between the data files and the Inbound directory and calculate the LTPA based on the expected time to transfer the size of the largest files you expect to be handling. Consider room for file growth as you calculate this value.

Having a reasonable amount of disk space available on the Inbound directory to account for the cases where slow network speeds cause authentication failures is also recommended.

Follow instructions in the P8 Performance tuning guide for setting the LTPA value.

[{"Product":{"code":"SSNVNV","label":"FileNet Content Manager"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"Content Engine","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"}],"Version":"5.0;5.1.0;5.2.0","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
17 June 2018

UID

swg21682646