Temporary file cleanup and database optimization

If you are concerned about disk space, you can occasionally clear some files and directories to lower your disk usage. You can also run database tools to ensure good repository database performance.

Many of these files contain diagnostic information useful in troubleshooting a problem. Before deleting any files, consider whether there is currently a problem that needs to be resolved. If these files are taking up too much space, the file size and count of certain sets of log files can be configured, instead of manually deleting these files yourself.

Running IBM Support Assistant Lite for IBM InfoSphere Information Server

You can run the general diagnostic health checker with this tool to identify old dump files and improperly managed logs. The tool is available at http://www-01.ibm.com/support/docview.wss?uid=swg24022700

After running IBM® Support Assistant Lite, when you no longer need the reports and collections for problem analysis, you can remove them. These files are in the path specified when you generate the report, or they are in the default installation folder.

  • Linux cue graphicUNIX cue graphicis_install_dir/ISALite
  • Windows cue graphicis_install_dir\ISALite

The services tier

Logs generated by IBM WebSphere® Application Server can grow quite large. You can occasionally check the size of the logs and clear the information you no longer need. However, these logs are of great value when diagnosing problems; therefore, delete them only if you are concerned with limited disk space. Before you can clear the logs, you must shut down the application server. The logs are in the WebSphere Application Server profile directory.

IBM WebSphere Application Server Liberty Profile
  • Linux cue graphicUNIX cue graphicIS_install_path/wlp/usr/servers/iis/logs
  • Windows cue graphicIS_install_path\wlp\usr\servers\iis\logs

Instead of deleting the messages.*.log and console.log files, their size and count can be configured.

IBM WebSphere Application Server Network Deployment
  • Linux cue graphicUNIX cue graphicAppserver_profile_dir/logs/Server_name
  • Windows cue graphicAppserver_profile_dir\logs\Server_name

Appserver_profile_dir is the location of the InfoSphere® Information Server profile for a stand-alone WebSphere Application Server configuration. For a clustered configuration, there are multiple profile locations, including the deployment manager profile on the services tier and the profile for each managed node in the cluster. The cluster might be local to the services tier system or on a remote computer.

Server_name is the name of the application server, node agent, or deployment manager server.

The following path names are typical default locations (shown with Linux path names):

opt/IBM/WebSphere/AppServer/profiles/InfoSphere/logs/server1
opt/IBM/WebSphere/AppServer/profiles/Dmgr01/logs/dmgr
opt/IBM/WebSphere/AppServer/profiles/Custom01/logs/nodeagent
opt/IBM/WebSphere/AppServer/profiles/Custom01/logs/server1
opt/IBM/WebSphere/AppServer/profiles/Custom01/logs/server2
opt/IBM/WebSphere/AppServer/profiles/Custom02/logs/nodeagent
opt/IBM/WebSphere/AppServer/profiles/Custom02/logs/server3

Instead of deleting the SystemOut and SystemErr files, their size and count can be configured.

Some of the following files and directories might be on your computer, depending on the product components that are installed. If they exist and the product is not currently running, these files can be deleted. The product components recreates them, if needed, when they are restarted.

In Appserver_profile_dir:

ascl-customattributes-Server_name.log
bg.log
iasHandler-Server_name.log
iasServer-Server_name.log
matching-Server_name.log
mmac-web-Server_name.log
workbench-admin-services-Server_name.log
xmeta-server.log

In Appserver_profile_dir/logs/:

glossary-Server_name.log
glossary-api-Server_name.log
glossary-web-Server_name.log
ISauditLog*.log.*
workbench.log
workbench-aspect.log
workbench-web-activity.log

If your services tier experienced outages because of environmental or program errors that have then been fixed, you can remove any heapdump.* or javacore.* files in the Appserver_profile_dir.

You can occasionally clear the contents of the reporting workspace. To clear the reporting workspace, you must first stop the application server. The following directory is the default location of the workspace:

  • Linux cue graphicUNIX cue graphictmp_dir/informationServer/Reporting[MachineName][AppServerNodeName][AppServerName]
  • Windows cue graphictmp_dir\informationServer\Reporting[MachineName][AppServerNodeName][AppServerName]

If you changed the reporting workspace per the instructions at https://www.ibm.com/support/docview.wss?uid=swg21317914, remove the files from there instead.

The InfoSphere Information Governance Catalog workspace can also be cleared. The following directories indicate the default locations.

  • Linux cue graphicUNIX cue graphic

    tmp_dir/BG_vrdata
    tmp_dir/WB_VRDATA

  • Windows cue graphic

    tmp_dir\BG_vrdata
    tmp_dir\WB_VRDATA

If you changed the InfoSphere Information Governance Catalog workspace per the instructions at http://www.ibm.com/support/docview.wss?uid=swg21413637, remove the files from there instead.

The metadata repository tier

Use database vendor tools to ensure good performance of the database. It is important that the optimization statistics are updated after any significant data updates in your repositories. Configuring your database system to automatically update statistics may improve system performance. Also consult your vendor documentation for the location of logs that you can clear.

If your repository is a DB2® database, you can run reorgchk, reorg, and runstats on a regular basis. Find more information about these tools in the DB2 documentation or at developerWorks®.

The RUNSTATS command

Periodically run RUNSTATS when significant change to data volumes or data content occurs. Even if the number of rows in a table remains consistent, changes in data contents can result in different and usually better access paths using RUNSTATS statistics. Periodically, after RUNSTATS is run, you can rebind DB2 packages that contain static SQL by using the db2rbind command. Doing so ensures that the static SQL uses optimal access paths.

The REORGCHK command

Periodically run REORGCHK, and perform its recommended REORGs. Similar to RUNSTATS, run REORGCHK when significant changes to data occur.

Log files

Occasionally check the size of the logs in the following path and clear the information you no longer need. Before you can clear the logs, you must shut down the database.

  • Linux cue graphicUNIX cue graphicdb2_user_dir/sqllib/db2dump
  • Windows cue graphicdb2_user_dir\SQLLIB\db2dump

The engine tier

Some files can be deleted from the following directories:

is_install_dir/ASBNode
is_install_dir/ASBNode/bin
is_install_dir/Server/DSEngine
is_install_dir/Server/DSEngine/bin

The following files can be deleted from these directories:

orbtrc.*.txt
orbmsg.*
Snap*.trc
asbagent_startup.err
asbagent_startup.out
heapdump.*.phd
javacore.*.txt
Note: To delete the asbagent_startup.* file, the agent must first be stopped.

From the is_install_dir/ASBNode/logs/logging_component_name directories, you can delete the *log.err files. Do not delete the *.log or *.pos files from these directories.

Monitor the size of the files in &PH& and &COMO& directories in runtime projects. These directories are in is_install_dir/Server/Projects/project_name

  • &PH& contains output files that are normally cleared when a job completes or the next time that the job starts. Running jobs with unique instance IDs, such as WISD, can result in an accumulation of files. Purge files that have not been updated in the last 24 hours.
  • &COMO& is used if tracing is enabled on the server. Files and subdirectories in this location can be deleted when no longer required. Tracing can be enabled or disabled in the Administrator client.

Monitor the contents of the scratch space directories that are defined by one or more Parallel Engine configuration file scratchdisk directives. These scratchdisk directives specify the names of one or more directories where intermediate or temporary data is stored for sort operations and virtual data sets, as a couple of examples. These directories are normally empty when no InfoSphere DataStage® jobs are active. However, files can accumulate if jobs that are running are forcibly stopped by a kill -9 command or the jobs crash with an unhandled exception. You can safely clear the contents of the directories that are defined by the scratchdisk directives, preferably when the InfoSphere DataStage Engine is stopped or when no InfoSphere DataStage jobs are running.

Temporary files can be monitored and cleaned up when they are not in use by the engine and when they are no longer needed. One such set of temporary files are the IPC files, which have the naming convention itag.project_name.job_name, where the default itag name is ade.

Temporary files are in the temporary directory (the directory specified by the TEMP environment variable). Temporary files are also in the location specified by the UVTEMP tunable.

Linux cue graphicUNIX cue graphic
Run the following command sequence to determine the location of UVTEMP tunable:
cd `cat /.dshome`
. ./dsenv
./bin/smat -t | grep UVTEMP
Windows cue graphic
To determine the location of the UVTEMP tunable, run the smat tool and page through until you find the UVTEMP tunable.
cd is_install_dir\Server\DSEngine
bin\smat -t

When operational metadata is collected, files are generated in the following directory:

  • Linux cue graphicUNIX cue graphicis_install_dir/Server/DSOMD/xml
  • Windows cue graphicis_install_dir\Server\DSOMD\xml

Files in this location are typically removed when the operational metadata has been imported successfully. If they still exist after the import is complete, you can remove them.

The dstage_wrapper_trace_n.log and dstage_wrapper_spy_n.log files can be monitored and cleared. A maximum of 20 logs of each type are created, with older logs being overwritten. But, you can delete the older logs manually, if desired. The logs are in the following directory:

  • Linux cue graphicUNIX cue graphicEngine_user_home/ds_logs
  • Windows cue graphicEngine_user_home\ds_logs

Engine_user_home is the home directory for the credential-mapped operating system user that is running InfoSphere DataStage jobs.

The client tier

Some files can be deleted from the following directories:

is_install_dir\ASBNode
is_install_dir\ASBNode\bin
is_install_dir\Clients\Classic
is_install_dir\Clients\ISC
is_install_dir\Clients\istools\cli
is_install_dir\Clients\istools\manager

The following files can be deleted from these directories:

orbtrc.*.txt
orbmsg.*
Snap*.trc
heapdump.*.phd
javacore.*.txt
*config.bak*

You can delete dstage_wrapper trace and spy files, in the user_home_directory\ds_logs directory.

You can delete the contents of the following directories in the user_home_directory.

.unicorn\
fasttrack-workspace\
ismanager_workspace\
istool_workspace\
Application Data\IBM\InformationServer\DataStage Client\temp_directories
Parent topic: Administering