Assessing disk performance with the iostat command

Begin the assessment by running the iostat command with an interval parameter during your system's peak workload period or while running a critical application for which you need to minimize I/O delays.

The following shell script runs the iostat command in the background while a copy of a large file runs in the foreground so that there is some I/O to measure:
# iostat 5 3 >io.out &
# cp big1 /dev/null
The AIX® operating system maintains a history of disk activity. If the disk I/O history is disabled, the following message is displayed when you run the iostat command:
Disk history since boot not available.

The interval disk I/O statistics are unaffected by this.
To enable disk I/O history, from the command line enter smit chgsys and then select true from the Continuously maintain DISK I/O history field.
The following examples show that you must consider all available data on the system to troubleshoot a problem.
  1. The following command stores three data samples on a system, which has three disks in the io.out file. These samples are collected at an interval of 5 seconds between samples.
    # iostat -T 5 3 > /tmp/io.out &
    
    # cp bigfile /dev/null
    
    System configuration: lcpu=4 drives=4 ent=1.00 paths=3 vdisks=2
    
    tty:      tin         tout    avg-cpu: % user % sys % idle % iowait physc % entc  time
              1.2          9.6                0.6   1.4   98.0      0.0   0.0    2.7  13:26:46
    
    Disks:         % tm_act     Kbps      tps    Kb_read   Kb_wrtn  time
    cd0               0.0       0.0       0.0          0         0  13:26:46
    hdisk1            0.0       0.0       0.0          0         0  13:26:46
    hdisk0            0.0       0.0       0.0          0         0  13:26:46
    hdisk2            0.0       0.0       0.0          0         0  13:26:46
    
    tty:      tin         tout    avg-cpu: % user % sys % idle % iowait physc % entc  time
              0.2          3.6                0.3  13.8   75.1     10.8   0.2   16.8  13:26:51
    
    Disks:         % tm_act     Kbps      tps    Kb_read   Kb_wrtn  time
    cd0               0.0       0.0       0.0          0         0  13:26:51
    hdisk1            0.0       0.0       0.0          0         0  13:26:51
    hdisk0           62.8   52428.8     205.4     262144         0  13:26:51
    hdisk2            0.0       0.0       0.0          0         0  13:26:51
    
    tty:      tin         tout    avg-cpu: % user % sys % idle % iowait physc % entc  time
              0.0          0.0                0.5   1.5   97.9      0.1   0.0    2.8  13:26:56
    
    Disks:         % tm_act     Kbps      tps    Kb_read   Kb_wrtn  time
    cd0               0.0       0.0       0.0          0         0  13:26:56
    hdisk1            0.0       0.0       0.0          0         0  13:26:56
    hdisk0            0.4       0.8       0.2          0         4  13:26:56
    hdisk2            2.6      33.6       7.8          0       168  13:26:56
    Note: If the iostat command is run without specifying a time interval, the output indicates a summary of the system data since the last system reboot, and not the current values.
    • The first and third intervals show that the three disks were mostly idle, along with the CPU utilization, which is also shown as idle in the tty report.
    • The second interval shows the activity that is generated by using the cp command, which was started for this test. This activity can be viewed on both the CPU activity (tty report) which shows 13.9% sys CPU and also on the disk report. The cp command took 3.14 seconds to run during this interval. In the report, the second interval shows 62.8 % for the hdisk0 disk under the tm_act metric. This example means that the hdisk0 disk was busy for 62.8 % of the time interval (5 seconds). If the cp command is the only process generating disk I/O to hdisk0, then the cp command took 62.8% of the 5 second interval, or 3.14 seconds, which is the total time the cp command took to run.
  2. The following command stores five data samples with a 2 seconds interval between samples on a system, which has three disks in the io.out2 file:
    # iostat -T 2 5 > /tmp/io2.out &
    
    # cp bigfile /dev/null
    System configuration: lcpu=4 drives=4 ent=1.00 paths=3 vdisks=2
    
    tty:      tin         tout    avg-cpu: % user % sys % idle % iowait physc % entc  time
              3.0         24.0                0.4   0.8   98.8      0.0   0.0    1.8  13:29:51
    
    Disks:         % tm_act     Kbps      tps    Kb_read   Kb_wrtn  time
    cd0               0.0       0.0       0.0          0         0  13:29:51
    hdisk1            0.0       0.0       0.0          0         0  13:29:51
    hdisk0            0.0       0.0       0.0          0         0  13:29:51
    hdisk2            0.0       0.0       0.0          0         0  13:29:51
    
    tty:      tin         tout    avg-cpu: % user % sys % idle % iowait physc % entc  time
              0.5          1.0                0.2   8.2   85.5      6.1   0.1   10.1  13:29:53
    
    Disks:         % tm_act     Kbps      tps    Kb_read   Kb_wrtn  time
    cd0               0.0       0.0       0.0          0         0  13:29:53
    hdisk1            0.0       0.0       0.0          0         0  13:29:53
    hdisk0           39.5   30464.0     120.5      60928         0  13:29:53
    hdisk2            0.0       0.0       0.0          0         0  13:29:53
    
    tty:      tin         tout    avg-cpu: % user % sys % idle % iowait physc % entc  time
              0.0          0.0                0.2  21.5   62.9     15.4   0.3   25.7  13:29:55
    
    Disks:         % tm_act     Kbps      tps    Kb_read   Kb_wrtn  time
    cd0               0.0       0.0       0.0          0         0  13:29:55
    hdisk1            0.0       0.0       0.0          0         0  13:29:55
    hdisk0          100.0   83712.0     327.0     167424         0  13:29:55
    hdisk2            0.0       0.0       0.0          0         0  13:29:55
    
    tty:      tin         tout    avg-cpu: % user % sys % idle % iowait physc % entc  time
              0.0          8.0                1.3   7.2   87.5      4.0   0.1   10.4  13:29:57
    
    Disks:         % tm_act     Kbps      tps    Kb_read   Kb_wrtn  time
    cd0               0.0       0.0       0.0          0         0  13:29:57
    hdisk1            0.0       0.0       0.0          0         0  13:29:57
    hdisk0           20.9   16898.0      66.5      33792         4  13:29:57
    hdisk2           10.4      82.0      19.0          0       164  13:29:57
    
    tty:      tin         tout    avg-cpu: % user % sys % idle % iowait physc % entc  time
              0.0          0.0                0.2   0.6   99.2      0.0   0.0    1.3  13:29:59
    
    Disks:         % tm_act     Kbps      tps    Kb_read   Kb_wrtn  time
    cd0               0.0       0.0       0.0          0         0  13:29:59
    hdisk1            0.0       0.0       0.0          0         0  13:29:59
    hdisk0            0.0       0.0       0.0          0         0  13:29:59
    hdisk2            0.0       0.0       0.0          0         0  13:29:59
  • The first and fifth interval show that the three disks were mostly idle, along with the CPU utilization, which is also shown as idle in the tty report.
  • The second interval shows the activity that is generated by using the cp command, which was started for this test. The cp command took 3.14 seconds to run during this interval. In the report, the second interval shows 39.5 % for the hdisk0 disk under the tm_act metric. In third and fourth interval shows 100 % and 20.9 % respectively for the hdisk0 disk under the tm_act metric. This means that the hdisk0 disk was busy for 100 % of the time interval (2 seconds) during the third interval and the hdisk0 disk was busy for 20.9 % of the time interval (2 seconds) during the fourth interval.
Both examples shows that the %tm_act variable only indicates that the disk was busy. You cannot use the %tm_act variable to evaluate a performance problem. To troubleshoot an issue, you might need to consider other options such as running the iostat -D flag, which provides real service times (both read and write ) and queuing information for the disks on the system.