In cases where the running orchestrations on a WebSphere Cast Iron V220.127.116.11 appliance have variables greater than 20KB and persistence is on, the WIP disk usage grows to 100% and does not get cleaned up on a reboot.
The WIP disk usage on WebSphere Cast Iron V18.104.22.168 appliance rises to 100% when the running orchestrations have variables greater than 20KB and persistence is on. This issue does not affect earlier versions of WebSphere Cast Iron.
When using persistence in orchestrations, WebSphere Cast Iron stores the contents of large variables temporarily on disk. However, because of a code defect in WebSphere Cast Iron V22.214.171.124, the files on disk do not get properly removed after a job is complete. This can cause the disk work-in-progress (WIP) partition to fill up. Customers will notice a gradual increase in disk usage over time and it can result in an appliance outage. File system usage can be observed through the Resource Utilization graph as "% Disk Used" in the WebSphere Cast Iron Web Management Console.
Do not turn persistence off. Persistence should remain turned on to avoid out of memory and other issues. See "Best Practices for Memory Management" for more details:
APAR LI77193 has been used to address this issue and includes further details and a local workaround.
A permanent fix for this issue has been delivered through WebSphere Cast Iron fix pack 126.96.36.199. The announcement and description of this fix pack is available here.
Announcements of new fix packs are made on the WebSphere Cast Iron Support Portal.
For planned fixpack maintenance release dates, see this support announcement.
Fix installation instructions:
1) Before installing the fix, run the following command from the Command Line Interface (CLI) on the appliance to clear out any remaining internal files and associated records that may not have been cleaned up earlier:
'system clean running orchmon'
Please note that this command will also remove any currently running project data and orchestration monitoring data. For more details on this command, see the related documentation.
2) Stop all of the orchestrations on the appliance.
3) Install the fix for both the appliance and the Studio.
4) Run the 'system clean running orchmon' command from the CLI again to clear out any remaining data that may have been generated between the time you ran the command the first time and the time you stopped the orchestrations.