The concurrency level attribute

The concurrency level attribute was introduced with Virtual I/O Server (VIOS) version 2.2.2.0 and is used to control the amount, and configuration, of resources that are allocated to a partition mobility operation by the mover service partition (MSP). The actual resources that are associated with a specific concurrency level value might change when new VIOS versions are released, but lower concurrency level values always equate to more resources allocated and in general, lower migration times.

From the VIOS versions 2.2.2.0 to 2.2.3.x, the concurrency level attributes controlled the amount of memory that is allocated for partition mobility operations. Starting with the version 2.2.4.0, the concurrency level also controls the number of threads that are used to send and receive the memory pages of the mobile partition. More threads require more processor and network bandwidth to be fully utilized, a strict limit on the number of partition mobility threads that are running is imposed to prevent the VIOS partition from being over loaded. This limit results in a lower number of allowed concurrent operations when concurrency level values less than 4 are used. It is recommended that the default value be used in most cases. The table provides use cases and recommendations for changing the concurrency level either for all migrations or for a specific partition mobility operation.

Table 1. Setting the concurrency level
VIOS Version	Recommended usage
VIOS Version	Concurrency level	Usage
2.2.2.0 - 2.2.3.x	5	Recommended concurrency level if a previous partition mobility operation failed because of insufficient memory.
	4	Not a recommended concurrency level.
	3	The default value, and is the recommended concurrency level for most situations, including but not limited to the following scenarios: Running concurrent LPM operations. System evacuations. Note: The default concurrency_lvl value that is changed to 4 from a value of 3 in VIOS version 2.2.4.0.
	2	Not a recommended concurrency level.
	1	Not a recommended concurrency level.
2.2.4.0, or later	5	Recommended concurrency level when any of the following scenarios are true: If a previous partition mobility operation fails because of insufficient memory. If the partition mobility operation runs on a lower speed network (less than 10 GB) and the migrating partition had failed previously, or rebooted because an application that is running on the partition had a heartbeat timer or Dead Man Switch (DMS) trigger. When you are migrating from a MSP with a high-speed network to a MSP with a low speed network. Note: Migrating a partition from a high-speed network to a low speed network is not recommended. However, if this situation cannot be avoided, using a concurrency level of 5 provides a higher probability of success.
	4	The default value, and is the recommended concurrency level for most situations, including but not limited to the following scenarios: Running concurrent LPM operations. System evacuations. Note: The default concurrency_lvl value changed to 4 from a value of 3 in VIOS version 2.2.4.0.
	3	The recommended concurrency level only when all the following scenarios are true: At least 20 Gb (gigabits) of network bandwidth is available for the MSP for each concurrent operation that is planned. Both source and destination MSPs are assigned with at least two processors assigned. Client logical partitions are configured with at least 50 GB of memory. Both the source and destination hypervisors are at version 8.4.0, or later. Both source and target MSPs are at VIOS version 2.2.4.0, or later. Note: A maximum of four concurrent partition mobility operations can be run for each MSP pair at this concurrency level.
2.2.4.0, or later (continued)	2	The recommended concurrency level only when all the following scenarios are true: At least 28 Gb (gigabits) of network bandwidth is available for the MSP for each concurrent operation that is planned. Both source and destination MSPs are assigned with at least 2.5 processors. Client logical partitions are configured with at least 50 GB of memory. Both the source and destination hypervisors are at version 8.4.0, or later. Both the source and target MSPs are at VIOS version 2.2.4.0, or later. Note: A maximum of three concurrent partition mobility operations can be run at this concurrency level. The limit is two if the operations are run with the strict flag.
2.2.4.0, or later (continued)	1	The recommended concurrency level only when all the following are true: Greater than 30 Gb (gigabits) of network bandwidth is available to the MSP for each concurrent operation that is planned. Both the source and destination MSPs are assigned with at least three processors. Client logical partitions are configured with at least 100 GB of memory. Both the source and destination system hypervisors are at version 840, or later. Both the source and destination MSPs are at version 2.2.4.0, or later. Note: A maximum of two concurrent partition mobility operations can be run for each MSP pair at this concurrency level.

If the concurrency level value on the source and destination MSP differ or if the MSPs are at different VIOS versions, the source and destination MSPs negotiate to a common set of resources. This generally results in either the source or destination MSPs negotiating to match the resources of the other. For migrations where you do not want the resources to be negotiated, the Hardware Management Console (HMC) version 8.4.0 and the VIOS 2.2.4.0 introduced the option of strict requirements. By specifying that the concurrency level value of strict requirements, the partition mobility validation fails if the requested resources cannot be satisfied by both the source and destination MSPs.

If you determine that the default concurrency level is not ideal for a specific partition mobility operation or for all partition mobility operations that use a specific VIOS as a MSP, you can perform one of the following actions:

Change the concurrency level value for all of the partition mobility operations that use a specific VIOS. The value can be set by using either the VIOS chdev command, or the HMC migrlpar command. For more information about changing the value of the concurrency level, see Live Partition Mobility pseudodevice.
To change the concurrency level value for a single partition mobility operation, the VIOS must be at version 2.2.4.0, or later, and the HMC must be at version 8.4.0, or later. The HMC command line provides a concurrency level override option. For a single migration operation, run the following command:
```
migrlpar -o v -m <srcCecName> -t <srcCecName> -p <lparName> -i 
"concurr_migration_perf_level=<overrideValue>" 
```
where the valid override values are 1, 2, 3, 4, 5, 1r, 2r, 3r, 4r, and 5r.
For multiple migration operations, run the following command:
```
migrlpar -o v -m <srcCecName> -t <srcCecName> -p <lparName> -i  
multiple_concurr_migration_perf_levels="<lparName_1>/<lparID_1>/<perfLvl_1>,  
<lparName_2>/<lparID_2>/<perfLvl_2>,...<lparName_n>/<lparID_n>/<perfLvl_n>" 
```
Where the values 1 - 5 indicate the concurrency level and the values 1r - 5r indicate that the concurrency level must be strictly enforced and the migration validation fails if the resources requested by the concurrency level value cannot be fully satisfied.
If either the source or destination MSP is at VIOS version 2.2.2.0, or earlier, the concurrence level value is ignored and the migration runs with a predefined buffer configuration and uses a single thread for sending data. This is applicable only if you select the concurrency level values in the range 1-5. If the you select the concurrency level values in the range 1r-5r, validation fails because the MSPs does not support multi-threading.

Last updated: Tue, March 12, 2019