New features and changes for IBM InfoSphere Information Server, Version 11.3

New features and changes are introduced for IBM® InfoSphere® Information Server, Version 11.3, along with documentation updates. The new and changed features and documentation updates are described in the following sections.

Contents

Suite-wide features
Information Governance
Data Integration
Data Quality
Deprecated features
Documentation introduced or enhanced with Version 11.3

Suite-wide features

Secure integration platform
InfoSphere Information Server Version 11.3 provides a more secure integration platform with these security features:
  • Single sign-on: All the InfoSphere Information Server browser-based clients now support single sign-on so that when you authenticate with one of the clients you can interact with the other clients.
  • SSL communication: InfoSphere Information Server now uses SSL to provide communication security for all client interfaces. Self-signed certificates are created during installation for you to use, but you can choose to install your own CA-verified certificate afterward. For more information, see Managing certificates.
  • Stronger encryption: The default encryption mechanisms for InfoSphere Information Server are now RSA 2048 and SHA-512.
  • Cell sharing: InfoSphere Information Server now uses the WebSphere® Application Server standard security domain. So InfoSphere Information Server can be deployed into a cell that is managed by a secured deployment manager without disrupting the profiles and applications that are already deployed in the cell.
WebSphere Application Server Liberty Profile
InfoSphere Information Server provides the option to install WebSphere Application Server Liberty Profile, a fast-starting, dynamic application server runtime environment for the services tier. It is quicker to install and easier to maintain, and it uses fewer system resources. For more information about using this application server or WebSphere Application Server Network Deployment, see Options for installing the application server software.
Application server cluster installation
Installation with an application server cluster is simplified. You can select information about the clusters that you have available, and you can install against a cluster member rather than a front-end web server. The configuration of the front-end web server can be done after installation when the necessary components are available and known. For more information about setting up a clustered WebSphere Application Server configuration, see Options for installing the application server software.

InfoSphere Information Governance Catalog

New to this release, InfoSphere Business Glossary, InfoSphere Business Glossary Anywhere, and InfoSphere Metadata Workbench are consolidated into the new product InfoSphere Information Governance Catalog. InfoSphere Information Governance Catalog delivers business-driven information to the enterprise faster for more confident business decisions. It builds and deploys trusted information by connecting business requirements to IT assets and by supporting the entire information delivery lifecycle.

InfoSphere Information Governance Catalog includes these key features:

Streamlined user interface
The overall design of InfoSphere Information Governance Catalog enhances your ability to easily manage and govern your assets. InfoSphere Information Governance Catalog has a streamlined user interface with enhanced embedded assistance. You can browse, search, and query all assets in the catalog, and then run lineage reports from within the catalog, without the need to open a different application.
Lineage features
  • Lineage report results are faster and more comprehensive due to a changed architecture and a new Lineage Viewer. The architecture stores the links between assets so that these links do not need to be recalculated for each lineage report. The Lineage Viewer no longer requires Adobe Flash Player. As a result, the Lineage Viewer can display many hundreds of assets and relationships, and quickly pan and zoom the report.
  • Lineage filters can focus attention on data assets that your organization finds most useful by hiding other assets, or other assets and their data flows, from the lineage graph. For more information, see Managing lineage filters.
  • Graphic rendering and filtering are done at much faster speeds.
  • Lineage reports can be displayed on iPhones, iPads, and other mobile devices.
  • Lineage reports now include references to assets from jobs or mapping documents even when the assets are not specifically imported into the metadata repository. These asset references can help to improve data governance.
Glossary features
  • Business-level vocabulary and information governance policies that govern data sources to specify data quality, data movement, lifecycle, data privacy, and other processes.
  • Comprehensive workflow for business glossary authoring, approvals, and publishing.
  • Business-friendly interfaces for accessing business and IT information from anywhere.
Integration with other applications
InfoSphere Information Governance Catalog is integrated with these applications:
  • InfoSphere Data Click to copy selected database tables, data files, data file folders, and Amazon S3 buckets from the catalog to a target distributed file system. For more information, see Moving data by using InfoSphere Data Click.
  • InfoSphere Information Analyzer to display its data rule definitions, data rule set definitions, data rules, and data rule sets.
  • The operational metadata that is generated when InfoSphere DataStage® and InfoSphere QualityStage® jobs are run is stored in a new location and automatically copied to the metadata repository. For more information, see Operational metadata.
Other features
  • User-managed metadata collections support collaborative development of analytics and other information delivery projects. You can group assets that you want to focus on, or to share with other users, into a collection.
  • You can display the relationships of a term in a hierarchical tree format. This format helps you to better understand the meaning of the term and its relationships with other terms. For more information, see Browsing terms in the Term Type Hierarchy view.
  • New roles are defined. For more information, see Security roles.
  • You can display, edit, and browse Amazon S3 assets.
  • Operational metadata that is generated when a job is run can now be purged from the metadata repository by using a new istool command. For more information, see workbench purgeOMD command.

InfoSphere Metadata Asset Manager

Import support was added for the following tools and types of metadata:

InfoSphere Data Click

The self-service data integration feature of the InfoSphere Information Server suite is greatly expanded in this release with the following enhancements:

Home page function
InfoSphere Data Click has an updated home page where you can create and monitor activities.
Authoring
InfoSphere Data Click has a new web-based and streamlined way to create and run activities from within InfoSphere Data Click. For more information, see Creating activities.
Monitoring
The monitoring in InfoSphere Data Click is enhanced and now has links to the InfoSphere DataStage and QualityStage Operations Console for more advanced monitoring. For more information, see Monitoring InfoSphere Data Click activities.
InfoSphere Information Governance Catalog integration
You can now use InfoSphere Data Click to move data that you select in InfoSphere Information Governance Catalog. For more information, see Moving data by using InfoSphere Data Click.
Cloud data store support
You can move data to and from Amazon S3 by using InfoSphere Data Click.
Big data support
You can move data to InfoSphere BigInsights®, where you can perform advanced analytics on your data. You can access and read data out of a Hadoop by using BigSQL and other JDBC methods. For more information, see Scenarios for integrating InfoSphere Data Click and InfoSphere BigInsights.
Relational database connector expansion
In addition to previously supported connectors, you can now also use the JDBC, Oracle, and DB2® connectors to move data between a much broader set of data sources. For more information, see Making assets available for use in activities.
Workload management
Jobs that are generated by InfoSphere Data Click activities are now managed by the workload management features of InfoSphere Information Server, which are available through the InfoSphere DataStage and QualityStage Operations Console. For more information, see Configuring the workload management queue.
Asset interchange
You can use asset interchange to import and export InfoSphere Data Click activities between InfoSphere Information Server instances. For more information, see InfoSphere Data Click assets.
Access control
With InfoSphere Data Click, you can directly associate activities with project areas and their user groups for more granular security and control. For more information, see Creating InfoSphere Data Click authors and users.

InfoSphere DataStage and InfoSphere QualityStage

REST web services
You can use the Hierarchical Data stage (previously called XML Stage) to design jobs that interact with REST (Representational State Transfer) web services by using HTTP methods. For example, you can design jobs that perform tasks such as posting message to social networking sites, interacting with systems such as Microsoft Sharepoint, or using maps and directions. For more information, see REST web services.
-url option for commands
The InfoSphere DataStage CLI now includes a new option, -url, for the logon clause of the dsjob and dsadmin commands. The option specifies a full format URL for the domain to log on to. For more information, see The logon clause.
Operations Console
If the capturing of monitoring data is enabled, the AppWatcher process is automatically started when the engine tier computer is started. For more information, see Managing the data collection processes.
Workload management
The workload management system is now enabled by default. For more information, see Administering workload management.
Big Data File stage compatibility
The Big Data File stage is now compatible with Hortonworks 2.1, Cloudera 4.5, and InfoSphere BigInsights 3.0.
Sort stage optimization
To reduce I/O operations for large sorts, the Sort stage converts bounded length fields to variable length before the sort, and converts them back to bounded length after the sort. Sort optimization also improves run time during which implicit sorts are sometimes required, for example, in InfoSphere QualityStage matching and join steps. For more information, see APT_TSORT_NO_OPTIMIZE_BOUNDED.
Improved flexibility in record delimiting
The Sequential File stage and associated import operator can now import records that do not contain all the fields in the import schema. For more information, see APT_IMPORT_HANDLE_SHORT.

Connections to data sources

Greenplum connector
You can use the new Greenplum connector stages in your jobs to read data from Greenplum databases, write data to Greenplum databases, or look up data in the contexts of those jobs. You can also import metadata from Greenplum databases by using InfoSphere Metadata Asset Manager (IMAM).
InfoSphere Master Data Management connector
You can use the new InfoSphere Master Data Management connector stage in your InfoSphere DataStage jobs to perform read and write operations. The MDM Connector stage can be configured to read data from or write data to InfoSphere MDM. For more information, see IBM InfoSphere Master Data Management.
Amazon S3 connector
You can use the new Amazon S3 connector to connect to Amazon Simple Storage Service (S3) and perform various read and write functions. For more information, see Amazon S3 connector.
Write to Microsoft Excel files
You can now design jobs that write data to Microsoft Excel files by using the Unstructured Data stage.

InfoSphere Information Analyzer

Performance
InfoSphere Information Analyzer has performance improvements in several areas:
  • Multi-column primary key analysis: A new, more scalable algorithm is used for multi-column primary key detection.
  • Data rule execution: Data rules now use the native DB2 connector for writing to the analysis database. Using the DB2 connector improves performance because the DB2 connector can use the bulk load connector method.
  • Column analysis: Job execution is faster because InfoSphere Information Server engine resources use a higher degree of parallelism and as a result run more efficiently.
  • User interface: Performance is improved because of optimizations in communication between the InfoSphere Information Analyzer client and the server.
More database support
InfoSphere Information Analyzer now supports Netezza® and Hive databases as data sources.
Ease-of-use
Configuration of the Data Rules stage in InfoSphere DataStage is easier because of the addition of drag support.
Integration with other applications
  • Access to InfoSphere Metadata Asset Manager to import metadata is now provided from within the InfoSphere Information Analyzer client.
  • Unpublished data rule definitions and data rule set definitions are displayed, and all InfoSphere Information Analyzer data quality assets are easier to access in InfoSphere Information Governance Catalog.

InfoSphere Information Governance Dashboard

SQL views and reports
You can use the fully documented SQL views plus the included Cognos® reports and dashboards to begin assessing your governance progress and customize them to suit your needs. For more information, see Working with the Cognos reports. If you want to create your own dashboard or reports, you can use the SQL views to do so. For more information, see Implementing your own information governance reports with the SQL views of InfoSphere Information Governance Dashboard.
Objects that you can query
Full documentation of the objects that you can query in the metadata repository is provided, with diagrams and definitions. The objects include the following categories:
  • InfoSphere Information Governance Catalog policies, information governance rules, categories, terms, stewards, custom attributes, and labels
  • InfoSphere Information Analyzer projects, rules, rule definitions, rule sets, and metrics
  • Implemented data resources that are imported into the metadata repository, including host computers, databases, schemas, database tables, and database columns
Automatic integration of InfoSphere Information Governance Dashboard and companion SQL Views
InfoSphere Information Governance Dashboard and the companion SQL Views are now integrated into the installation of InfoSphere Information Server. You do not need to deploy the SQL views manually. The instructions and necessary packages for installing the IBM Cognos reports and dashboards are installed with InfoSphere Information Server. The reports, dashboards, and views are automatically refreshed to ensure that you are viewing the most up-to-date information.
Access to IBM Cognos components
InfoSphere Information Governance Dashboard also includes entitlement to a select number of components of IBM Cognos. The limited entitlement to the following components is included for displaying the dashboards and reports that are included with InfoSphere Information Governance Dashboard.
  • Cognos Administration
  • Cognos Connection
  • Cognos Report Studio
  • Cognos Viewer
For more information, see Configuring components.
Launchpad link to workspace
You can now set up a link on the InfoSphere Information Server Launchpad to open the InfoSphere Information Governance Dashboard workspace. For more information, see Setting up a link to InfoSphere Information Governance Dashboard from the InfoSphere Information Server Launchpad.

Deprecated features

UniData stage
The UniData 6 stage is no longer supported. Use the UniData stage to connect to UniData version 7.0 or earlier. For more information, see IBM UniVerse and UniData.
IBM Cognos GO and OSLC link types
IBM Cognos GO and OSLC link types are no longer supported.
InfoSphere Metadata Workbench Model Viewer
The Model Viewer from InfoSphere Metadata Workbench is not included in InfoSphere Information Governance Catalog.
InfoSphere Information Services Director bindings
InfoSphere Information Services Director no longer supports the following bindings:
  • REST. The REST2 binding continues to be available.
  • SOAP over JMS.
InfoSphere Information Services Director web catalog
The InfoSphere Information Services Director web catalog is discontinued.
InfoSphere DataStage mainframe jobs
Mainframe jobs are not supported in this version of InfoSphere Information Server.

Documentation introduced or enhanced with Version 11.3

InfoSphere Information Governance Dashboard
HTML documentation for the SQL Views
You can access the HTML documentation for the SQL Views from the InfoSphere Information Governance Dashboard documentation. For more information, see SQL views for InfoSphere Information Governance Dashboard.
Examples of custom reports from SQL Views
Several examples of how to implement your own reports that are based on the SQL Views are provided. For more information, see Implementing your own information governance reports with the SQL views of InfoSphere Information Governance Dashboard.
InfoSphere DataStage
To help troubleshoot problems, more InfoSphere DataStage error messages are documented.
Connectivity
The connectivity section of the documentation is newly organized to provide easier access to the content. For more information, see Connecting to data sources.
Asset interchange
The material on asset interchange is updated.