What's new in IBM Integration Bus for WebSphere Enterprise Service Bus users?

Message modeling and message processing

Message modeling and message processing overview

Business data flows between service providers and service consumers over various protocols (HTTP, JMS, WebSphere MQ, EIS, and so on) in various data formats such as comma-separated value, delimited, fixed width, and COBOL. Different protocols have different mechanisms for carrying the business data in their protocol envelope. While the transport protocol envelope is different, the business data might not be in same format across these protocols.

IBM Integration Bus supplies a range of parsers to parse and write message formats, for example XMLNSC, DFDL, or DataObject:

• The XMLNSC parser is a flexible, general-purpose XML parser that offers high performance XML parsing and optional XML schema validation. The XMLNSC parser has a range of options that make it suitable for most XML processing requirements. Some of these options are only available in the XMLNSC parser. Although the XMLNSC parser can parse XML documents without an XML schema, extra features of the parser become available when it operates in model-driven mode. In model-driven mode, the XMLNSC parser is guided by an XML schema, which defines the structure of the message tree (also known as the logical model). The logical message model is equivalent to the WebSphere Enterprise Service Bus Service Message Object structure.
• The DFDL parser reads and writes a wide variety of message formats, and is intended for general text and binary message formats, including industry standards. It is not intended for parsing and writing XML or JSON formatted messages, which have their own message domains. When reading a message, the DFDL parser interprets a bit stream by using grammar that is defined in a DFDL schema file, and generates a corresponding DFDL domain logical message tree in the broker. When writing a message, the DFDL serializer generates a DFDL formatted bit stream from a DFDL domain logical message tree. Because the DFDL parser is model-driven, it can validate messages against the message model that is defined in the DFDL schema file. The level of validation that is performed by the DFDL parser is the same as the level that is defined by XML schema 1.0. For more information, see DFDL parser and domain.
• The DataObject parser reads and writes messages from Enterprise Information Systems (EIS) such as SAP, PeopleSoft, and Siebel. Such messages belong to the DataObject domain. When it receives a message from an adapter, the DataObject parser constructs a message tree from the business object that it receives from the EIS. When it writes a message, the DataObject parser creates from the message tree the business object that it sends to the EIS. The DataObject parser is always model-driven, and it is guided by the XML schemas that model the EIS business objects. The XML schemas are created automatically from the content of a message set. You use the DataObject domain to parse and write messages for WebSphere Adapters and CORBA applications.

Each parser is suited to a particular class of message format that is called a domain. For more information, see Parsers.

IBM Integration Bus parsers are analogous to the WebSphere Enterprise Service Bus data handlers, in that they are responsible for the conversion between the physical message format and a logical message structure. In WebSphere Enterprise Service Bus, the data handler is responsible for populating part of the Service Message Object (SMO) structure, while in IBM Integration Bus, the parser is responsible for populating the message tree.

In IBM Integration Bus, some message formats are self-defining and can be parsed without reference to a model. However, most message formats are not self-defining, and a parser must have access to a message model that describes the message, if it is to parse the message correctly. Message models are based on W3C XML schema. When the message format is known, IBM Integration Bus can parse an incoming message bit stream and convert it into a logical message tree for manipulation by a message flow. After the message is processed by the message flow, IBM Integration Bus converts the message tree back into a message bit stream. Conversion to and from a bit stream is performed by a parser.

You can model a wide variety of message formats by using XML schema files, DFDL schema files, and WebSphere Adapter schema files. IBM Integration Bus provides importers to read metadata such as C header files, COBOL copybooks, and EIS metadata, and to create message models from that metadata. By using these importers, you speed up the creation of message models. Additionally, predefined models are available for common industry standard message formats such as SWIFT, EDIFACT, X12, FIX, HL7, and TLOG. For more information, see Modeling different data formats.

When you create a message model, you obtain the following benefits:

• Runtime validation of the messages: A parser checks whether the input and the output messages have the correct structure and data values.
• Enhanced parsing of the XML messages: The parser is provided with the data type of data values, and can cast the data accordingly.
• Improved productivity when writing ESQL: The ESQL editor can use message models to provide code completion assistance.
• Drag-and-drop operations on message maps: When you are creating message maps, the Graphical Data Mapping uses the message model to populate its source and target views.
• Reuse of message models, in whole or in part: You can create additional messages that are based on existing messages.
• Generation of documentation.
• Provision of version control and access control for message models by storing them in a central repository.

For more information, see Constructing message models.

Data Format Description Language (DFDL)

Data Format Description Language (DFDL) is a modeling language from the Open Grid Forum that is based on XML schema 1.0. DFDL uses standard XSD model objects to express the logical structure of the data, together with DFDL annotations to describe the text or binary physical representation. You use DFDL to describe many data formats:

• Textual and binary
• Commercial record-oriented
• Scientific and numeric
• Modern and legacy
• Industry standards

You use the DFDL schema editor to create, edit, and test DFDL message models within the IBM Integration Toolkit. For more information, see DFDL schema editor.

IBM Integration Bus provides support for a DFDL domain. The DFDL domain can be used to parse and write a wide variety of message formats, and is intended for general text and binary message formats, including industry standards. It is not intended for parsing and writing XML or JSON formatted messages, which have their own message domains. For more information, see DFDL parser and domain.

Patterns framework

IBM Integration Bus provides more built-in patterns that speed up the process of implementing your integration solutions. In addition, you can create your own user-defined patterns to extend the functionality of IBM Integration Bus. IBM Integration Bus provides the capabilities to graphically define and contribute your own patterns to the Patterns Explorer.

Prebuilt patterns

A pattern is a reusable solution that encapsulates a top-down tested approach to solving a common architecture, design, or deployment task in a particular context. Patterns provide the following benefits:

• Give you guidance for the implementation of solutions.
• Increase development efficiency. Resources are generated from a set of predefined templates.
• Result in higher-quality solutions, through reuse of assets and common implementation of programming approaches, such as error handling and logging.

A number of patterns are supplied in a GitHub pattern repository, which is accessible from the IBM Integration Toolkit. For more information, see Built-in patterns. You can configure these patterns with values for use in your own environment to solve specific business problems. For more information, see Patterns.

You can create user-defined patterns to extend the functionality of IBM Integration Bus. For more information, see Creating a user-defined pattern.

User-defined patterns

User-defined patterns extend the function of IBM Integration Bus so that you can create patterns that you can reuse within your organization. For more information, see User-defined patterns.

You can package your user-defined pattern into a pattern archive so that the user-defined pattern can be distributed to pattern users by adding the pattern archive to a pattern community site. For more information, see Packaging and distributing pattern plug-ins.

Operational management

Message flow notification

The system administrator can set the notification threshold for individual message flows deployed to manage their workload. An out of range notification message is produced if the notification threshold is exceeded. A back in range notification message is produced if the notification threshold later drops back into range. For more information, see Configure a message flow to cause notification threshold messages to be published.

Message flow maximum rate

The system administrator can set the maximum rate that an individual message flow can run at. The maximum rate is specified as the total number of input messages processed every second. When set, the number of input messages that are processed across the flow is measured. This measure is irrespective of the number of additional instances that are used, the number of input nodes in the message flow, or the number of errors that occur. If necessary, a processing delay is introduced to keep the input message processing rate under the maximum flow rate setting. For more information, see Setting the maximum rate for a message flow.

Unresponsive message flows

The system administrator can specify and monitor the maximum amount of time that any message flow is allowed to process a message for, and can specify an action to be taken if the timeout is exceeded. Additionally, manual requests can be made to stop a message flow by restarting the integration server. For more information, see Unresponsive message flows.

Configure a workload management policy

In IBM Integration Bus, a configurable service can be used to configure a set of attributes that can be accessed at run time. A system administrator can change the values of the attributes for a configurable service, which then affects the behavior of an IBM Integration Bus node or message flow without the need for redeployment.

The system administrator can specify a workload management policy under a configurable service for a message flow. The workload management policy has the advantage in that it encompasses all of the properties available under workload management in one place and therefore allows for easier tuning of message flow performance. For more information, see Workload management.

.NET

.NETCompute node

You can use the .NETCompute node to route or transform messages by using any Common Language Runtime (CLR) compliant .NET programming language, such as C#, Visual Basic (VB), F# and C++/CLI (Common Language Infrastructure). By using this node, IBM Integration Bus can interact with other applications that have .NET or Component Object Model (COM) interfaces and perform tasks such as message enrichment, by obtaining data from these applications. For more information, see .NETCompute node and Using .NET.

.NET prebuilt patterns

IBM Integration Bus provides prebuilt patterns for Microsoft Dynamics CRM that allow transformation and synchronization of client data with another CRM application. These patterns enable Microsoft applications to be rapidly integrated with IBM Integration Bus integration solutions.

• A prebuilt pattern assists with the integration of Microsoft Dynamics CRM with a SAP system. Updates to customer data from create, read, update, and delete operations using a SAP BAPI can be easily reflected in a matching Dynamics CRM Account Entity. For more information, see Microsoft Dynamics CRM Account Entity output: Static BAPI input.
• A prebuilt pattern builds upon the first by allowing a Microsoft Dynamics CRM application to be updated by using various other mechanisms, including WebSphere MQ messages, HTTP, or flat files. For more information, see Microsoft Dynamics CRM Account Entity output: Dynamic transport input.

.NETInput

The .NETInput node provides an easy way to drive message flows that can receive data from other applications that have Microsoft .NET or Component Object Model (COM) interfaces. The .NETInput node provides C#, Visual Basic, and Visual F# templates for Microsoft Visual Studio, which when implemented, connect and retrieve the data for the node. For example, the .NETInput node can be used to retrieve messages from MSMQ. For more information, see .NETInput node and Using .NET.

You can also create a template from an instance of a .NETInput node in a message flow. The result is a Cloned node that has its own icon, properties and runtime implementation, which can be easily exported and shared with other IBM Integration Toolkit developers. For more information, see Cloning a .NETInput node.

File processing

In IBM Integration Bus, you use message flow nodes to handle data from files. You use a FileInput node to process messages that need to read data from files, a FileOutput node to write messages to files, and a FileRead node to read one record, or the entire contents of a file, from within a message flow.

For more information, see How the broker processes files.

WebSphere ESB conversion tool

IBM Integration Bus supplies a WebSphere ESB conversion tool that accelerates the conversion of development artifacts created for WebSphere Enterprise Service Bus in WebSphere Integration Developer or IBM Integration Designer to IBM Integration Bus development artifacts.

For more information, see The WebSphere ESB conversion tool and Converting WebSphere Enterprise Service Bus resources by using the WebSphere ESB conversion tool.

Application development differences

When you develop integration solutions in IBM Integration Bus, you must consider the following differences:

WSDL files

In IBM Integration Bus, each inline schema embedded in your WSDL file is externalized. You must create a schema (.xsd) file for each inline schema.

The WebSphere ESB conversion tool converts your WebSphere Enterprise Service Bus WSDL file and creates a schema (.xsd) file for each inline schema embedded in your WSDL file. The format of the converted WSDL file is different from the original, but it is logically equivalent.

To learn more about WSDL files in IBM Integration Bus, see Working with WSDL.

Integration solution

You convert a WebSphere Enterprise Service Bus mediation module into an IBM Integration Bus application or integration service.

For more information, see Choosing the target integration solution type.

Exports and imports

In IBM Integration Bus, you use a message flow node to re-create an export or an import. The node is specific to a type of transport. The transport binding is configured automatically and determined by the type of node that you use.

To specify the parser that you want to use to serialize or deserialize your data in an export or import, you must configure the Message Domain property.

To handle faults, you connect the Fault terminal of a node and implement its logic.

For exports that use a SOAPInput node, the function selector logic is provided by the node.

For all other exports, you must manually implement the function selector as routing logic within the message flow. This can be achieved by using the IBM Integration Bus capabilities for querying the message content and selecting the route within the flow, for example by using a JavaCompute node and a RouteToLabel node.

For more information on IBM Integration Bus built-in nodes, see Built-in nodes.

For more information, see Converting WebSphere Enterprise Service Bus resources manually.

Invocation styles to other services

In IBM Integration Bus, you define the invocation style of a call to a service provider by using a Request message flow node within a message flow. Request-response operations can be called synchronously or asynchronously. IBM Integration Bus provides a synchronous and an asynchronous implementation for Request and Response message flow nodes.

For more information, see Converting SOAP/HTTP WebSphere Enterprise Service Bus service invocation styles manually and Converting SOAP/JMS WebSphere Enterprise Service Bus service invocation styles manually.

Graphical data maps

WebSphere Enterprise Service Bus and IBM Integration Bus provide built-in transform types that you can use to construct your transformation maps. WebSphere Enterprise Service Bus group, lookup, relationship, and relationship lookup transforms do not have equivalent transform types in IBM Integration Bus. You must re-create these transforms manually.

For more information, see Mapping transform equivalents between the Graphical Data Mapping editor and the WebSphere Enterprise Service Bus Graphical Mapping editor.

Handling data from files

In WebSphere Enterprise Service Bus, you use the WebSphere Adapter for Flat Files to exchange data with the local file system. In IBM Integration Bus, you use message flow nodes to handle data from files. For this reason, capabilities to handle data from files, that you have in WebSphere Enterprise Service Bus on the WebSphere Adapter for Flat Files, are located in IBM Integration Bus on the message flow nodes.

IBM Integration Bus provides three different primitive nodes for handling data from files:

• FileInput node: Use the FileInput node to process messages that are read from files. For more information, see FileInput node.
• FileOutput node: Use the FileOutput node to write messages to files. For more information, see FileOutput node.
• FileRead node: Use the FileRead node to read one record, or the entire contents of a file, from within a message flow. For more information, see FileRead node.

Each file message flow node provides four Record detection options which describe how data should be read from a file, written to a file, and propagated through the message flow. Based on the option that you choose, you decide how much of the file content is carried through the message flow as part of a single propagation, either the whole file, a fixed length of data, a record until a delimiter is found, or a section of data which conforms to a predefined message model (parsed record sequence).

In IBM Integration Bus, the combination of a built-in parser, for example BLOB, XMLNSC, DFDL, JSON, and a Record detection option is the equivalent to the WebSphere Adapter for Flat Files built-in data handlers in WebSphere Enterprise Service Bus.

IBM Integration Bus file nodes also provide support for commands to run before or after an FTP or SFTP transfer finishes. For more information, see Local environment overrides for the remote server on the FileOutput node.

User-defined patterns

WebSphere Enterprise Service Bus and IBM Integration Bus provide built-in patterns to help you solve a specific business problem and the capability to create your own user-defined patterns. In WebSphere Enterprise Service Bus, the process is more complex and requires more coding. In IBM Integration Bus , you can define graphically your user-defined pattern and add it to the Patterns Explorer.

Terminology differences

To learn about the terminology differences between IBM Integration Bus and WebSphere Enterprise Service Bus, see WebSphere Enterprise Service Bus conversion.

DataObject

The term DataObject is used for different purposes in WebSphere Enterprise Service Bus and IBM Integration Bus. In WebSphere Enterprise Service Bus, DataObject refers to a part of the Service Data Object (SDO) API commonj.sdo.DataObject, which is used to represent structured data elements within the Service Message Object. In IBM Integration Bus, the term DataObject refers to the DataObject parser, which reads and writes messages from Enterprise Information Systems (EIS) such as SAP, PeopleSoft, and Siebel.

Runtime environment differences