Specifying partitioning style

Based on partitioning style, InfoSphere DataStage can group partitioners into categories based on key locality. This makes it possible to determine if the inputs and outputs of an operator are appropriately partitioned, even if the operator is not associated with a particular partitioning method.

Unlike using APT_Operator::setPartitionMethod to stipulate a specific partitioning method for an operator, specifying partitioning style does not prevent you from inserting your own custom partitioner in a data flow, and does not prevent an operator user from inserting a partitioner in a data flow. A warning is issued when an operator user inserts an unnecessary or inappropriate partitioner, but the partitioner is not removed.

The partitioning styles are:

eLocalKeys. All keys are placed in the same partition.
eDistributedKeys. There is no guarantee that keys are in the same partition.

Partitioning style is defined by the partitioningStyle enumeration of the APT_PartioningStyle class defined in $APT_ORCHHOME/incude/apt_framework/paritioningStyle.h.

Set and get the partitioning style of an operator input data set

To set and get the partitioning style of an operator input data set, use the APT_Operator functions below:

void setInputPartitioningStyle
	(APT_PartitioningStyle::partitioningStyle, int inputDS)
APT_PartitioningStyle::partitioningStyle
	getPartitioningStyle(int inputDS)
void setOutputPartitioningStyle
	(APT_PartitioningStyle::partitioningStyle, int outputDS)

These functions allow you to:

Direct InfoSphere DataStage to insert appropriate partitioners in a data flow.
Require that two inputs be identically partitioned.
Determine whether two inputs are identically partitioned

Set and get the partitioning keys of an operator input data set

To set and get the partitioning keys of an operator input data set, use the APT_Operator functions below:

void setInputPartitioningKeys(APT_PropertyList keys, int inputDS);
APT_PropertyList APT_Operator::getPartitioningKeys(int input)

The syntax for APT_PropertyList is:

{ key = {value=fieldname}, key = {value=fieldname}, ...}

For example:

{ key={value=lastname}, key={value=balance} }

Obtain the partitioning style of a custom partitioner

To obtain the partitioning style of a custom partitioner, use this APT_Partitioner function:

virtual APT_PartitioningStyle::partitioningStyle getPartitioningStyle() const=0;

This function must be overridden by partitioners to define the partitioning style which they fit into.

Get the name of the partitioner last used to partition an operator input

To get the name of the partitioner last used to partition an operator input, use this APT_Operator function:

APT_String APT_Operator::getPartitioningName(int input)

By default, InfoSphere DataStage performs partitioner insertion.

Disable partitioner insertion

To disable partitioner insertion, add the global option -nopartinsertion to your osh command line, or specify the environment variable APT_NO_PART_INSERTION.