The TSM backup/archive client uses lots of memory during an incremental backup in order to determine which files are new or changed and need to be backed up, and which files are deleted and need to be expired on the TSM server. In some situations, this memory demand may cause the TSM backup/archive client to issue the message "ANS1030E The operating system refused a TSM request for memory allocation." and fail the backup. How much memory is required and are there any ways to reduce this memory requirement?
For incremental backup, by default, the TSM client will use on average about 300 bytes of memory per file system object. An object is a file or directory. This 300 bytes includes the path and name of the object and attributes, so this average will be increased if longer file or directory names or more deeply nested directories are used. For a file system with 1 million files and directories, the TSM client will require about 300 MB of virtual memory.
This virtual memory requirement can be reduced by any of the following methods:
- Using the TSM client option memoryefficientbackup diskcachemethod. The maximum virtual memory used by the client process in this case will usually be less than 20 MB. A significant amount of client disk space may be required (see the TSM client User's Guide for details). Refer to document 1254744 for optimizing performance when using the disk cache method.
- Using the TSM client option memoryefficientbackup yes. The maximum virtual memory used by the client process in this case becomes 300 bytes times the maximum number of files in any one directory.
- Using the TSM client option virtualmountpoint (UNIX only) to define multiple virtual mount points within the single file system, each of which can be backed up independently by TSM.
- If the TSM client option resourceutilization is set to a value greater than 3 and there are multiple file systems being backed up (or could be backed up such as in the case of failover in an active-active cluster), then reducing the value of resourceutilization to 3 or below, or using testflag maxproducers:1 will limit the process to incremental backup of a single file system at a time, resulting in a reduction in virtual memory requirements. If backup of multiple file systems in parallel is required for performance reasons, and the combined virtual memory requirements exceed the process limits, then multiple backup processes should be used in parallel.
- Using the TSM client option incrbydate, which executes an incremental-by-date backup.
- Using the TSM client include/exclude options to back up only what is necessary.
- Using the TSM journal-based incremental backup function (Windows client, AIX client, and Linux with TSM 6.3 client, or later). Note that a full incremental backup must be completed before a journal-based backup will be possible. (see document 1155524 for details)
- Using the TSM image backup function to backup the entire volume. This may actually require less time and resources than using incremental backup on some file systems with a very large number of very small files.
- Spreading the data across multiple file systems.
Virtual memory refers to the memory that is addressable by an application process. The virtual memory size can be greater or smaller than the computer's real (physical) memory size. For example, a 32-bit application can address 4GB of virtual memory. The application may be running on a workstation that only has 1GB of real memory, or it may be running on a server that has 16GB of real memory.
Only part of an application's virtual memory may be used for allocating data structures. The maximum virtual memory available for allocation by a 32-bit process depends on the operating system. For Windows, this is 2GB for most processes. Note that the TSM Windows client can use up to 3GB for data structures if the client system has been booted using the /3gb boot.ini flag - see Microsoft Knowledge Base article 291988 (http://support.microsoft.com/kb/291988). For AIX, this is 9 memory segments, or 2.25GB. Other operating systems may have different limits. Certain operating system quotas or limits may need to be set to allow a process to use the maximum virtual memory (refer to the ulimit command on UNIX systems).
The maximum number of files/directories that TSM can back up on a 32-bit system (2GB of process memory) using a single incremental backup process and the default option memoryefficientbackup no is about (2 * 1024 * 1024 * 1024) / (300 * 1000 * 1000) = 7 million. If the TSM client process tries to exceed the operating system process virtual memory limit, the backup will fail. If the TSM client process memory requirements exceed the amount of real memory on the system but do not exceed the virtual memory limit, then the backup may be successful however it will likely exhibit poor performance due to paging.
Using a 64-bit client, if available for the platform in question, will essentially eliminate any concern about the virtual memory requirements, but may still require significant real memory for optimal performance.