Setting up the package size for BW Load and Extract data transfers

Answer

BW Load stage sets Packet Size dynamically.

The Packet Size can be set in SAP BW by using the transaction code RSCUSTV6

(BW: Threshold Value for Data Loading: Details). The packet size specifies the number of data records within a package that are delivered to SAP when you load data from the DataStage job. In this transaction the default value is 1,000 records.
Note: This setting applies to data transfer from non-SAP source systems. DataStage application registers as the non-SAP, External (3rd Party) source system.

SAP recommends: Do not spread the data set across too many packages because this has a negative impact on performance. The number of data packages per load process should not be higher that 100. The basic setting should be between 5,000 and 20,000 records. To load a large volume of transaction data, the packet size should be changed from the default value of 1,000 to a value between 10,000 and 50,000 data records per package.

The BW Load stage determines a value of the packet size dynamically in the following places in this order (first value found is used):

1. The BW system packet size parameter (tcode RSCUSTV6 )
2. The environment variable ASC_BWLOAD_PACKETSIZE
3. The BW Load stage default value (currently 100,000 records)

You can tune the packet size on each InfoPackage also. When you open the InfoPackage in BI, on the right hand side, do a right click on the InfoPackage name and select "DataS. Default Data Transfer". In the table that displays you can change the Maximum size of a data packet in rows. You should experiment with these values by setting them to a small value (something around 10,000) or medium ones (80,000-90,000 rows).

To manage the packet size value from within DataStage job with the BW Load stage these two environment variables can be used:
ASC_BWLOAD_PACKETSIZE = Sets the packet size, if not set on the SAP side in RSCUSTV6; maximum size is 100,000 records
ASC_BWLOAD_PACKETSIZE_OVERRIDE=Overrides the packet size in RSCUSTV6; maximum size is 100,000 records

BW Extract stage uses the static Packet Size defined on the BI side.

• InfoSpoke: On the InfoSpoke, by default, Full is the Extraction Mode with 10,000 lines/records per Data Package. You can change the value and Save /Activate InfoSpoke.
• Open Hub: On the Data Transfer Process (DTP) of the Open Hub Destination, by default, Full is the Extraction Mode with 50,000records per Data Package. You can change the value and Save /Activate DTP.

The packet size describes the number of data records that are contained in one individual data package that is read by the extractor. For data with more than one segment, like hierarchies, the packet size indicates the number of data records for all segments contained in one package. For this reason, a package could contain more data records than specified by the packet size.

It is recommended you use the default packet size. Using an InfoSpoke/DTP packet size greater than the default would impact SAP BW server's performance and exhaust its resources.
To manage the packet size value from within the DataStage job with BW Extract stage use BW Packet Size property in the BW Extract Stage in Grid style properties.

To setup the BW packet size open the BW stage in Grid style and provide a value for the BW Packet Size field.

Note: BW stages write the packet size/count entry to the job log.