IBM Support

InfoSphere Information Server, Version 11.3 job compatibility

Troubleshooting


Problem

IBM InfoSphere DataStage and QualityStage, Version 11.3 introduce new features and functions that affect existing Version 9.1 jobs. When you import jobs from InfoSphere DataStage or InfoSphere QualityStage, Version 9.1 to Version 11.3, you need to recompile all jobs. In addition, you might need to make changes to your parallel jobs to accommodate some changes in Version 11.3.

Resolving The Problem

This topic has the following sections:



Changes to Parallel Jobs
Changes to Parallel Jobs

Below are the environment variables that you might need to set to revert back to the behavior prior to version 11.3.

DescriptionEnvironment Variable
In InfoSphere Information Server Version 9.1 and earlier, the sequential file stage used the value of the job NLS map as the character set for the field names when writing the file. This could result in the column names being in a different character set than the data. In Version 11.3, if the output NLS Map setting is defined on the sequential file stage, it will use that value as the character set for the field names.Set the environment variable APT_EXPORT_FIELD_NAMES_USING_IMPEXP_ CHARSET to use the job NLS map to define the character set for the field names in the output file instead of the NLS Map setting on the sequential file stage. This variable reverts to the behavior from 9.1 and earlier.
Prior to InfoSphere DataStage Version 11.3, aggregation reduce output fields were set as nullable fields. In InfoSphere DataStage Version 11.3 aggregation reduce output fields are defined such that their nullability matches the nullability of the input field that is being reduced. Consequently, if in any of the upstream stages in the job the aggregation reduced output field has been defined as nullable but the input field nullablility of that column is not nullable, then that upstream stage does not produce null output for that reduced column. In Version 11.3, set the APT_AGGREGATOR_NULLABLE_OUTPUT variable to cause the group operator to define reduce fields in its output schema as nullable, regardless of the nullable setting of the corresponding field. By default, the operator matches the null setting of the output schema to the null setting of the input schema that is being reduced.
When APT_NO_JOBMON is defined, job monitoring is turned off. However, starting with release 11.3, this variable will not disable the polling thread since doing so can cause job hangs for operators like the surrogate key stage that needs this thread to run. To revert to the same behavior from before 11.3, set both environment variables APT_NO_JOBMON and APT_NO_POLLING_THREAD.

Setting both will turn off job monitoring and all threads it creates including the polling thread. This setting can cause job hangs if they have stages that are dependent on the polling thread to run.
In InfoSphere DataStage Version 11.3 the behavior of the Sequential File stage was changed so that a file pattern is converted into a fileset before the files are imported. This change allows the files to be processed in parallel rather than sequentially and also support better data lineage.Set the APT_IMPORT_PATTERN_USES_CAT environment variable to force the behavior of prior versions of InfoSphere DataStage. This prior behavior was to concatenate all the files that match a file pattern before importing the files sequentially.

Changes to Server Jobs
Changes to Server Jobs

In Version 11.3, the UniData 6 stage is no longer supported. You can use the Connector Migration Tool to upgrade to the current UniData stage.

Updates to Data Sets
Updates to Data Sets

There are no updates to data sets. Data sets are compatible between InfoSphere Information Server Versions 9.1 and 11.3.

InfoSphere QualityStage Updates, Enhancements, and Fixes
InfoSphere QualityStage Updates, Enhancements, and Fixes

Updates:
There is no duplicate period in the separation list of email addresses.
Rule set affected: VEMAIL

The rule set has been corrected to remove warnings in the log file.
Rule set affected: IEADDR

Enhancements:
There is removed conversion from MEDIA to MEDICAL.
There is improved processing of individual last names that are missing an apostrophe.
Rule set affected: USNAME

There is added processing to handle P.O. Box references that are preceding street addresses.
There is added processing to handle the standard highway abbreviation in addresses (HWY).
Rule set affected: USADDR

There is added processing for new diplomatic post office (DPO) military addresses.
Rule set affected: USAREA

There is removed conversion from MEDIA to MEDICAL.
There is corrected handling of embedded D', L', and O'.
There is improved processing of individual last names that are missing an apostrophe.
Rule set affected: CANAME

There is improved processing of French street addresses.
Rule set affected: CAADDR

The inconsistency in street suffix processing has been fixed.
Rule set affected: DEADDR

There is improved processing of individual last names that are missing an apostrophe.
Rule set affected: IENAME

Updates that Require Job Recompilation
Updates that Require Job Recompilation
All jobs require recompilation or the jobs will fail to run.

[{"Product":{"code":"SSVSEF","label":"IBM InfoSphere DataStage"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Not Applicable","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"},{"code":"PF033","label":"Windows"}],"Version":"11.3","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}},{"Product":{"code":"SSZJPZ","label":"IBM InfoSphere Information Server"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":" ","Platform":[{"code":"","label":""}],"Version":"","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
16 June 2018

UID

swg21677107

Page Feedback