SETOBJACC/CLRPOOL Commands

Resolving The Problem

1.0 SETOBJACC/CLRPOOL

The IBM® OS/400® V2R2 Set Object Access (SETOBJACC) command can load a database file, a database index, or a program into a storage pool. This allows the user to exercise some control over the contents of a storage pool. The storage pool can be a private pool, the pool the job is running in, or one of the system's shared pools.

The primary use of the command is to provide improved application performance for jobs that access a database file in a random order or a series of jobs that access the same file. If the pool is large enough to contain all the data, once the object is loaded, all read accesses to the file are done without a disk read, thereby eliminating much of the disk time of input processing for that file. As a result, dramatic improvements in job run time may be realized.

If a private pool contains only the preloaded data, the data will stay in memory until the object is explicitly purged (using SETOBJACC), overlaid with another file as a result of another SETOBJACC (not recommended), or the pool is cleared (using another new command, Clear Pool (CLRPOOL).

A database file, a database index, or a program can be explicitly removed from all of main storage (not just an individual pool) using the SETOBJACC *PURGE option. This is the recommended procedure to use to manage the contents of a pool that has been set aside for explicit object content control.

This command allows changes in application design methodology, primarily because of the major improvements during random processing of data base files. If the application currently does Get by Key or Get by Key followed by Get Next, the data is read synchronously one record at a time. With SETOBJACC, if the file fits, the system will block load it into the pool and not purge the data thereafter. This provides very efficient application processing regardless of how the file is accessed by the application. Since much of the I/O for random processing is eliminated, it can eliminate the need to sort or reorganize the data for files that fit in a pool.

This article discusses SETOBJACC usage considerations and procedures, how the command works, and results from a test run using it.

1.1 Considerations for Use

1.1.1 Prior to Using SETOBJACC

1.	Select the jobs that have large amounts of database disk I/O and determine the files that are contributing the most to job run time. Often this would be files that are accessed randomly (Get by Key).
2.	Determine if these physical and logical files, or a subset of them, will fit into main memory. If not, determine if they all need to be in memory at once or if they can be loaded separately. This is dependent on the particular job stream run environment.
3.	Decide the storage pool you want to use: you can use the pool the job runs in (not recommended), one of the system's shared pools, or a separate private pool. We recommend using a separate private pool. If you're going to use the current job's pool, there is no guarantee that the data will stay in memory. The data is not pinned in memory and can be stolen due to page fault or other processing that require storage pool memory. If you're using a shared pool, make sure that you do not intend to use IBM's QPFRADJ or one of the OEM performance tuners which change the pool size due to a lack of paging activity.
4.	Decide the size the pool should be to contain the data you want pre-loaded. This requires an understanding of the file size and the amount of activity. To determine if a data base file is a good candidate, it should have high synchronous read activity (for example, lots of random Read by Key and very little disk I/O overlapped with processing) and fit into the pool. Sequential input and output files probably aren't very good candidates.
5.	Consider the effect of Workload Compression. A job's run time is made up, for the most part, of CPU time and disk I/O time. If, through the use of SETOBJACC, the disk time is reduced, the job will run in a shorter elapsed time. While the amount of CPU time used for the job is about the same as before, it takes less time to use it because the disk I/O time has been reduced. This causes a corresponding increase in overall CPU utilization. Basically, the system is moving from one bottleneck (disk I/O) toward another (CPU capacity) as the CPU part of the elapsed time is compressed into less clock time.

1.1.2 Other Considerations

1.	For batch work where the same file is used in multiple steps, SETOBJACC can be used to preserve the file in main storage across job steps.
2.	Consider experimenting with SETOBJACC *PURGE. If a separate pool is not used and if a job step uses several files and one or more are going to be used in the subsequent job step, try SETOBJACC *PURGE for the files that are not used on the next step. This may help keep around some of the files that are used.
3.	If you use SETOBJACC for batch processing, you should occasionally review the job's job logs to assure that preloaded objects continue to fit in the pool.

1.2 Procedures

To create a private pool in a separate subsystem, do the following:

1.	CRTSBSD sbsd name ((1 size 1))
2.	STRSBS sbsd name

To add a private pool to a current subsystem (assuming sbs pool ID 3 is available), do the following:

1. CHGSBSD sbsd name ((3 size 1))


To determine file activity and file sizes on a system, do the following:

1.

File Activity Use WRKJOB, Option 14, Display Open Files. This will show the currently open files by name and number of I/O operations.

2.

File Size Use DSPFD over the files identified in the previous step to determine their sizes. The SETOBJACC loaded file size will be about 20 to 30 kb more than the product of the number of records (including number of deleted records) times the record size. If a file has many deleted records and you're not using the Reuse Deleted Records option, you should reorganize it beforehand to reduce its memory requirement.

The amount of memory needed to load a program is less than the size shown on DSPOBJD; however, the exact requirement can be determined only using SETOBJACC on the program and using the information supplied with the completion message CPC1141 issued by the command. This message tells you the amount of memory used to load the executable instructions and other parts of the program needed for execution. It also tells you the unused space in the pool (K available in the pool) before loading the object (not after). You should look at the amount of space used to put the object in the pool and the amount of pool space available at the start of the command to assure that there is adequate space.

1.3 How Things Work with SETOBJACC and CLRPOOL

1.	There will be few, if any, database reads once the file has been preloaded using SETOBJACC. Database writes will still be handled the same way. Database writes are usually done asynchronously to the user's job by system output tasks. Since database reads are processed much faster once a file is preloaded, database writes will be done more frequently by a program that is also doing updates, adds, or deletes. This may cause occasional overrun of the system output tasks, resulting in more database writes being done synchronously rather than asynchronously to the user's job. Therefore, the job may not perform as well as expected. However, the job will still be faster when using SETOBJACC.
2.	If an object doesn't fit into the pool, part of it may be overlaid during SETOBJACC.
3.	Each SETOBJACC assumes that the entire pool is available. Therefore, objects in the pool will be overlaid (if necessary) to process the current request. Using SETOBJACC *PURGE option for objects no longer needed will reduce the possibility of overlaid pages.
4.	Object loading uses asynchronous, block transferred disk operations. SETOBJACC issues simultaneous asynchronous reads for the object's data. As a result, data sequence is not preserved by the loading process. Thus, if the object is larger than the pool, there's no guarantee that the first part of the object is overlaid with the latter part of the object.
5.	If an object has been partially loaded into a pool other than the job's pool, faults that occur on the object will cause a fault in the job's storage pool. This could also happen when processing added records.
6.	Database write (purge) still makes the page eligible to steal. Pages that have been purged with SETOBJACC *PURGE or with CLRPOOL are the highest priority steal candidates. If the pages are in a pool reserved for the SETOBJACC objects, they will be replaced only when another SETOBJACC command is issued.
7.	SETOBJACC will attempt to put all pages of an object in the specified storage pool by first purging the object from memory. If other jobs are using the object, they may be reloaded into other pools again before the SETOBJACC is able to load all the pages. Pages added to a file after the object has been loaded to memory are not kept in memory. They would reside in the job's pool and would be eligible for purge after they are written to disk.

1.4 CLRPOOL

The CLRPOOL command removes everything from a storage pool. The command writes all changed pages to disk and indicates that all pages in the pool are empty.

Storage management uses some pool pages as work areas; clearing a pool may not always make 100 percent of the pool immediately available.

1.	The first time you use CLRPOOL on a storage pool, it may run for a very long time depending upon the number of changed pages that are written. The long run time is most likely after starting the subsystem to define the pool.
2.	CLRPOOL should not be used frequently or indiscriminately. It can run for a very long time depending on both the size of the pool and the number of changed pages in the pool.
3.	CLRPOOL is not interruptible. Once it starts, it does not check to see if you want to cancel it. The command will run until the pool has been cleared.
4.	Running CLRPOOL on a D70 with unmirrored 9336's and nothing else on the system did about 50 synchronous disk operations per second and used about 2 to 3 percent of the CPU. For a 1-meg pool fully populated with changed pages, that would be 2,000 synchronous disk operations requiring about 40 seconds. You could extrapolate that to say that clearing a 100-meg pool containing only changed pages would take 4000 seconds. This is a worse-case situation. Most pools will have a small percentage of changed pages.
5.	If you have set up a separate pool for data and that pool exists at IPL time, the best time to clear it is immediately after IPL. At this time, the pool has the least number of changed pages in it, and the command will run much faster.
6.	It's better and faster to explicitly purge an object using SETOBJACC rather than CLRPOOL. SETOBJACC passes a list of object addresses to storage management. Storage management can efficiently locate pages of the objects and asynchronously block writes to disk. CLRPOOL causes storage management to go through the pool one page at a time and synchronously write each changed page.
7.	SETOBJACC *PURGE option allows you to manage the pool's contents. If you keep track of the objects loaded and purged from a private pool, CLRPOOL should only be used right after the pool is defined.
8.	If you define a pool and then use SETOBJACC on it immediately without a CLRPOOL, it can take some amount of time initially since it may have to force a lot of pages out of the pool before completing the object load. It could potentially cover up some of its own pages in an uncleared pool if there are none available, as all pages would be eligible for purge. Priority for stealing pages is empty pages, changed pages, unreferenced pages.
9.	You can use the CLRPOOL command to measure a job's paging characteristics; for example:
	a. Clear the pool b. Set the pool size c. Run the test and record the paging rates Do these three steps again in the same order. Do not change the size and then clear the pool. Changing a pool's size does not remove any pages from main storage. Some of the valid data pages may still be in memory, and your results will not be valid.

1.5 Examples

1.

100 mb of data uses 200,000 pages. If the application reads half of the pages synchronously with .020 seconds for each read, that's 2,000 seconds. The time to load that much data with the SETOBJACC command is very short because of the overlapped, asynchronous blocked reads being done to load the file.

2.

A series of queries on a dedicated IBM® AS/400® D70. There were 58,529 records, each 456 bytes long in a database file queried three different ways. The tests were first run without preload. The job, programs and data were all in the same 32,000kb pool. Then the tests were run with the logical and physical files preloaded in a 32,000kb pool. The pools containing the data were cleared before each test was run.

No Object Preload

Type of Test	ELP Time	CPU Time	Sync DIO	Async DIO
Test 1A, Arrival Sequence	30.4	9.545	383	1026
Test 1B, Arrival Sequence	26.4	9.780	138	1026
Test 3, Seq with Existing Index	153.1	22.753	3004	15113
Test 4, Seq with Query Built Index	154.1	37.606	2536	15606

Figure 1: Test results without preloading data base file

With Object Preload

Type of Test	ELP Time	CPU Time	Sync DIO	Async DIO
Test 0, SETOBJACC Object Preload	18.7	.157	32	979
Test 1A, Arrival Sequence	15.4	9.365	346	2
Test 1B, Arrival Sequence	12.0	9.674	98	2
Test 3, Seq with Existing Index	29.8	19.620	326	28
Test 4, Seq with Query Built Index	45.9	35.230	330	70

Figure 2: Test results with data base file preloaded


1.6 Job Stream example

PGM    NITEJOB
             STRSBS SBSD(NITE)
             DLYJOB   DLY(10)           /* allow time for allocation of pool memory  */
             CLRPOOL   POOL(NITE 1)
             DLYJOB               DLY(60)     /*  allow time for clearing pool  */
             SETOBJACC OBJ(CUSTMAST) OBJTYPE(*FILE) POOL(*PURGE)
             SETOBJACC OBJ(CUSTMAST) OBJTYPE(*FILE) POOL(NITE 1)
             SBMJOB CMD(CALL PGM(EODBATCH)) JOB(EODBATCH) JOBQ(QBATCH)
ENDPGM

This program will allocate memory to the pool defined to subsystem NITE (see procedures in 1.2 above) when the subsystem is started. The pool will then be cleared. The delay times may need to be adjusted based on activity on the system. CUSTMAST will then be purged from all active pools and "Set" into the NITE subsystem pool. The end of day batch job is then submitted to the batch job que for processing. Requests for customer master data will be available without going to disk.