HTTPRequest node

Use the HTTPRequest node to interact with a web service.

The HTTPRequest node is available in the following operation modes:
  • Developer
  • Application Integration Suite
  • Standard
  • Advanced
  • Express
  • Scale
  • Adapter
For more information, see Operation modes.

This topic contains the following sections:

Purpose

The HTTPRequest node interacts with a web service, using all or part of the input message as the request that is sent to that service. You can also configure the node to create an output message from the contents of the input message, augmented by the contents of the web service response, before you propagate the message to subsequent nodes in the message flow.

Depending on the configuration, this node constructs an HTTP or an HTTP over SSL (HTTPS) request from the specified contents of the input message, and sends this request to the web service. The node receives the response from the web service, and parses the response for inclusion in the output tree. The node generates HTTP headers if they are required by your configuration.

You can use this node in a message flow that does or does not contain an HTTPInput or HTTPReply node.

The HTTPRequest node handles messages in the following message domains:

  • DFDL
  • XMLNSC
  • JSON
  • BLOB
  • MIME
  • XMLNS
  • MRM

The HTTPRequest node is contained in the HTTP drawer of the palette, and is represented in the IBM® Integration Toolkit by the following icon:

HTTPRequest node icon

Using the HTTPRequest node to issue a request to a web service

An HTTP request has two parts:
  1. The URL of a service.
  2. A stream of data that the remote server processes, then sends back a response, which is often a SOAP or other web service message in XML.

The URL is of the format http://<address>[:<port>]/<function>; for example, http://localhost:7080/request. This URL can be specified statically in the HTTPRequest node parameters as a field in the message itself, or as a field in the local environment. The data to be sent to the web service can be the whole, or a portion of, the message tree, as specified in the HTTPRequest node properties.

The data must be in CCSID 1208 format for most requests. The reply can replace the input message, or be inserted into the message tree; the location is specified in the HTTPRequest node parameters. The domain for the reply is XMLNS. If the request is successful, the HTTPResponse is inserted into the front of the message tree, the reply placed in the specified location in the tree, and the request propagated to the Out terminal. If the HTTPRequest node is not able to issue the request, an ExceptionList is inserted into the message tree and the tree is propagated to the Failure terminal.

If the request is sent successfully by the HTTPRequest node, but the web service is not successful, the HTTPResponse is inserted into the message tree, and propagated to the Error terminal. The error message location parameter on the HTTPRequest node specifies where in the tree the response is placed, for example OutputRoot.XMLNS.error. You might have to use a Compute node to cast this response to an appropriate code page to be able to display the data, for example: 
Set OutputRoot.XMLNS.error850 = CAST(InputRoot.XMLNS.error.BLOB as CHAR CCSID 850);
For information about HTTP, see Hypertext Transfer Protocol - HTTP/1.1. For more information about HTTP return codes, see HTTP Response codes.

You can specify a timeout interval, so that if the request takes longer than the specified duration, the request is propagated to the Failure terminal with an appropriate message. For each request that the HTTPRequest node processes, it opens a connection, and then closes it when the response is returned. If the timeout interval is specified, the socket is closed after the interval. This closure ensures that a request gets only the correct response, and any response data for a request that has timed out is discarded.

You can use the HTTP proxy to route a request through an intermediate site. You can run tools as a proxy to see the request and the response, and therefore debug your flows. The HTTP destination is as seen by the proxy; if you specify the HTTP destination of localhost, and the HTTP proxy is running on a different computer, the request is routed to the remote proxy computer, not the computer from which the original request was issued.

Using the HTTPRequest node in a message flow

The HTTPRequest node can be used in any message flow that must send an HTTP request. The most common example is a message flow that calls a web service.

For more information about web services, see Processing web service messages.

An HTTPRequest node can consume a service that requires the following authentication mechanisms:

Handling errors

The node interacts directly with an external service using TCP/IP; it can, therefore, experience the following types of error:

  • Errors that are generated by TCP/IP, for example no route to host or connection refused.

    If the node detects these errors, it generates an exception, populates the exception list with the error information that is received, and routes the input message unchanged to the Failure terminal.

  • Errors that are returned by the web server. These errors are represented by HTTP status codes that are outside the range 100 - 299. If the node detects these errors, it routes the reply to the Error terminal while following the properties specified on the Error tab.

    The reply is produced as a BLOB message because the node cannot determine in what format the reply will be. If you have not configured this node to handle redirection, messages with a redirection status code (3xx) are also handled in the same way.

HTTP Response Codes

The HTTPRequest node treats the 100 series status codes as a 'continue' response, discards the current response, and waits for another response from the web server.

The 200 series status codes are treated as success, the settings on the various tabs on the node determine the format of the output message that is generated, and the response is routed to the Out terminal of the node.

The 300 series status codes are for redirection. If the Follow HTTP(s) Redirection property is selected, the node resends the request to the new destination that is specified in the response that is received. If the Follow HTTP(s) Redirection property is not selected, the codes are treated as an error, as described in Using the HTTPRequest node to issue a request to a web service. For more information about HTTP return codes, see HTTP Response codes.

The 400 and 500 series status codes are errors, and are treated as described in Using the HTTPRequest node to issue a request to a web service. For more information about HTTP return codes, see HTTP Response codes.

Manipulating headers

If you select Replace input message with web-service response or Replace input with error, the header for the input message (the header that belongs to the message when it arrives at the In terminal of the HTTPRequest node) is not propagated with the message that leaves the HTTPRequest node. However, if one of the properties that specify a location in the message tree is specified, the input message headers are propagated.

The HTTPResponse header, which contains the headers that are returned by the remote web service, is the first header in the message (after Properties) that is propagated from the node. This action is taken regardless of the options that are selected. Therefore, for the reply from the HTTPRequest node to be put to a WebSphere® MQ queue, manipulate the headers so that an MQMD is the first header (after Properties).

If you are replacing the input message with a response, you can copy the input message MQMD to the Environment tree before the HTTPRequest node, and then copy it back into the message tree after the HTTPRequest node. If you are specifying a location for the response, in order to maintain existing input message headers, you must move or remove the HTTP Response header so that the MQMD is the first header.

The following example contains ESQL that removes the HTTPHeader:
SET OutputRoot = InputRoot;
SET OutputRoot.HTTPResponseHeader = NULL;
The following example contains ESQL for moving the HTTPHeader, and therefore preserving the information that it provides:
SET OutputRoot = InputRoot;
DECLARE HTTPHeaderRef REFERENCE TO OutputRoot.HTTPResponseHeader;
DETACH HTTPHeaderRef;
ATTACH HTTPHeaderRef TO OutputRoot.MQMD AS NEXTSIBLING;

Terminals and properties

The HTTPRequest node terminals are described in the following table.

Terminal Description
In The input terminal that accepts a message for processing by the node.
Failure The output terminal to which the message is routed if a failure is detected during processing in the node.
Out The output terminal to which the message is routed if it represents successful completion of the web service request, and if further processing is required within this message flow.
Error The output terminal to which messages that include an HTTP status code that is not in the range 200 through 299, including redirection codes (3xx) if you have not set the property Follow HTTP(s) redirection property, is routed.

The following tables describe the node properties. The column headed M indicates whether the property is mandatory (marked with an asterisk on the panel if you must enter a value when no default is defined); the column headed C indicates whether the property is configurable (you can change the value when you add the message flow to the BAR file to deploy it).

The HTTPRequest node Description properties are described in the following table.

Property M C Default Description
Node name No No The node type, HTTPRequest The name of the node.
Short description No No   A brief description of the node.
Long description No No   Text that describes the purpose of the node in the message flow.

The HTTPRequest node Basic properties are described in the following table.

Property M C Default Description mqsiapplybaroverride command property
Web service URL Yes Yes   The URL for the web service. You must provide this in the form http://hostname[:port]/[path] where
  • http://hostname must be specified.
  • port has a default of 80. If you specify a value, you must include the : before the port number.
  • path has a default of /. If you specify a value, you must include the / before the path.
 URLSpecifier
Request timeout (sec) Yes Yes 120 The time in seconds that the node waits for a response from the web service. The valid range is 1 through (231)-1. You cannot enter a value that represents an unlimited wait. The timeout might take up to one second longer than the specified value. timeoutForServer

The HTTPRequest node HTTP Settings properties are described in the following table.

Property M C Default Description mqsiapplybaroverride command property
HTTP(S) proxy location No Yes   The proxy server to which requests are sent. This value must be in the form hostname:port.

If the configured proxy server requires authentication, the credentials must be set by using the mqsisetdbparms command with the resource name httpproxy, or provided in the message flow. For more information, see Routing requests through an HTTP proxy server that has authentication enabled.

 httpProxyLocation
Follow HTTP(S) redirection No No Cleared If you select the check box, redirections are followed. If you clear this check box, redirections are not followed.  
HTTP version No Yes 1.0 The HTTP version to use for requests. Valid values are 1.0 and 1.1. httpVersion
Enable HTTP/1.1 keep-alive No Yes Selected (if HTTP version is 1.1) Use HTTP/1.1 Keep-Alive. enableKeepAlive
HTTP method No No POST The HTTP method. Valid values are POST, GET, PUT, DELETE, and HEAD. By default, the HTTPRequest node uses the HTTP POST method when it connects to the remote web server. HEAD is used to determine whether a service is available - for example, by a Network Dispatcher trying to work out which servers are available - and sends back the correct headers (including content-length) but no body data.  
Use compression No Yes None This property controls whether the content of the HTTP request is compressed. You can choose a value from none, gzip, zlib (deflate), and deflate. If the request is compressed, the Content-Encoding header is set to indicate that the content is compressed.

zlib (deflate) represents RFC 1950 + RFC 1951 combined.

deflate represents RFC 1951 only.

 requestCompressionType
Attention: The HTTPRequest or HTTPAsyncRequest node always rewrites the Content-Length header, even if you have cleared Generate default HTTP headers from input or request. This action ensures that the content is correct.

The HTTPRequest node SSL properties are described in the following table.

Property M C Default Description mqsiapplybaroverride command property
Protocol No Yes TLS The SSL protocol to use when making an HTTPS request. protocol
Allowed SSL ciphers No Yes   A comma-separated list of ciphers to use when making an SSL request. The default value of an empty string means use all available ciphers. allowedCiphers
Enable SSL certificate hostname checking No Yes No This property specifies whether the host name of the server that is receiving the request must match the host name in the SSL certificate. hostnameChecking
SSL client authentication key alias No Yes (empty string) This property specifies an SSL authentication alias for the client-side of an HTTP connection. Taking the default value means that the first appropriate key is chosen for you automatically. keyAlias
Enable certificate revocation list checking No Yes Not selected This property specifies whether CRL checking should be enabled for SSL connections enableCRLCheck

The HTTPRequest node Response Message Parsing properties are described in the following table.

Property M C Default Description
Message domain No No BLOB The domain that is used to parse the message. If the field is blank then the default is BLOB.
Message model No No Cleared The name or location of the message model schema file in which the message is defined.

When you click Browse, you see a list of available message model schema files for the selected Message domain.

Message No No Cleared The name or location of the message root within your message model schema file. This list is populated with all available messages that are defined in the Message model that you have selected.
Physical format No No Cleared The name of the physical format of the message. If you are using the MRM or IDOC parser, select the physical format of the incoming message from the list. This list includes all the physical formats that you have defined for the selected message model. If you set the Message domain property to DataObject, you can set this property to XML or SAP ALE IDoc. Set this property to SAP ALE IDoc when you have to parse a bit stream from an external source and generate a message tree.

The HTTPRequest node Parser Options properties are described in the following table.

Property M C Default Description
Parse timing No No On Demand This property controls when a response message is parsed. Valid values are On Demand, Immediate, and Complete.

For a full description of this property, see Parsing on demand.
Build tree using XML schema data types No No Cleared This property controls whether the XMLNSC parser creates syntax elements in the message tree with data types taken from the XML schema. You can select this property only if you set the Validate property on the Validation tab to Content or Content and Value.
Use XMLNSC compact parser for XMLNS domain No No Cleared This property controls whether the XMLNSC Compact Parser is used for messages in the XMLNS Domain. If you set this property, the response message data is displayed under XMLNSC in nodes that are connected to the output terminal when the input MQRFH2 header or Response Message Parsing properties Domain is XMLNS.
Retain mixed content No No Cleared This property controls whether the XMLNSC parser creates elements in the message tree when it encounters mixed text in a response message. If you select the check box, elements are created for mixed text. If you clear the check box, mixed text is ignored and no elements are created.
Retain comments No No Cleared This property controls whether the XMLNSC parser creates elements in the message tree when it encounters comments in a response message. If you select the check box, elements are created for comments. If you clear the check box, comments are ignored and no elements are created.
Retain processing instructions No No Cleared This property controls whether the XMLNSC parser creates elements in the message tree when it encounters processing instructions in a response message. If you select the check box, elements are created for processing instructions. If you clear the check box, processing instructions are ignored and no elements are created.
Opaque elements No No Blank This property is used to specify a list of elements in the response message that are to be opaquely parsed by the XMLNSC parser. Opaque parsing is performed only if validation is not enabled (that is, if the Validate property is set to None); entries that are specified in Opaque Elements are ignored if validation is enabled.

The HTTPRequest node Error Handling properties are described in the following table.

Property M C Default Description
Replace input with error No No Selected If you select this check box, the input message content is replaced by the error message content. If you clear this check box, you must specify Error message location.
Error message location Yes No OutputRoot The start location at which the parsed elements from the web service error bit stream are stored. This property takes the form of an ESQL field reference.

The HTTPRequest node Advanced properties are described in the following table.

Property M C Default Description mqsiapplybaroverride command property
Use whole input message as request No No Selected If you select this check box, the whole input message body is to be passed to the web service. If you clear this check box, you must select Request message location in tree.  
Request message location in tree Yes No InputRoot The start location from which the bit stream is created for sending to the web service. This property takes the form of an ESQL field reference.  
Replace input message with web-service response No No Selected If you select this check box, the web service response message replaces the copy of the input message as the content of the output message that is created. If you clear this check box, you must select Response message location in tree.  
Response message location in tree Yes No OutputRoot The start location at which the parsed elements from the web service response bit stream are stored. This property takes the form of an ESQL field reference.  
Generate default HTTP headers from input No No Selected If you select this check box, an HTTPRequestHeader is generated. If you clear this check box, a valid HTTPRequestHeader must exist in the input message.  
Accept compressed responses by default No Yes Cleared This property indicates whether the request node handles compressed responses by default. If the request header does not contain an Accept-Encoding header and this option is selected, the node sets the Accept-Encoding header to "gzip, deflate", and any compressed response that is received is decompressed by the node.

If the message propagated to the Request node includes an Accept-Encoding header, the message flow or client application should handle any compressed response. Therefore selecting this option has no effect in that case.

 acceptCompressedResponses

The HTTPRequest node Validation properties are described in the following table.

For a full description of these properties see Validation properties.

Property M C Default Description mqsiapplybaroverride command property
Validate No Yes None This property controls whether validation takes place. Valid values are None, Content and Value, Content, and Inherit. validateMaster
Failure action No No Exception This property controls what happens if validation fails. You can set this property only if you set Validate to Content or Content and Value. Valid values are User Trace, Local Error Log, Exception, and Exception List.  
The Monitoring properties of the node are described in the following table.
Property M C Default Description
Events No No None Events that you have defined for the node are displayed on this tab. By default, no monitoring events are defined on any node in a message flow. Use Add, Edit, and Delete to create, change or delete monitoring events for the node; see Configuring monitoring event sources by using monitoring properties for details.

You can enable and disable events that are shown here by selecting or clearing the Enabled check box.

Local environment overrides

You can dynamically override set values in the local environment in the same way as setting values in other elements of a message. The following values can be set under LocalEnvironment.Destination.HTTP.
Setting Description
RequestURL Overrides the Web service URL property on the node. For example:
SET OutputLocalEnvironment.Destination.HTTP.RequestURL = 'http://ibm.com/abc/';
RequestURLDoNotPercentEncodeCharacters Specifies that the characters in RequestURL are excluded from percent encoding. The default behavior of the HTTPRequest node is to percent encode all non-reserved characters in RequestURL. Reserved characters are, by default, excluded from percent encoding. The reserved characters are the following characters: : / ? # [ ] @ ! $ & ' ( ) * + , ; = < > %
Timeout Overrides the Request timeout (sec) property on the node. For example:
SET OutputLocalEnvironment.Destination.HTTP.Timeout = 42;
TimeoutMillis Overrides the Request timeout (sec) property on the node. For example: 
SET OutputLocalEnvironment.Destination.HTTP.TimeoutMillis = 5000;
This property defines the timeout in milliseconds. The value of TimeoutMillis overrides the value for Timeout if both values are set.
ProxyURL Overrides the HTTP(S) proxy location property on the node. For example:
SET OutputLocalEnvironment.Destination.HTTP.ProxyURL = 'my.proxy';
RequestLine.RequestURI Overrides the RequestURI, which is the path after the URL and port. For example:
SET OutputLocalEnvironment.Destination.HTTP.RequestLine.RequestURI = '/abc/def';
RequestLine.HTTPVersion Overrides the HTTP version property on the node. For example:
SET OutputLocalEnvironment.Destination.HTTP.RequestLine.HTTPVersion = 'HTTP/1.1';
KeepAlive Overrides the Enable HTTP/1.1 keep-alive property on the node. For example:
SET OutputLocalEnvironment.Destination.HTTP.KeepAlive = TRUE;
RequestLine.Method Overrides the HTTP method property on the node. For example:
SET OutputLocalEnvironment.Destination.HTTP.RequestLine.Method = 'GET';
SSLProtocol Overrides the SSLProtocol. For example:
SET OutputLocalEnvironment.Destination.HTTP.SSLProtocol = 'TLS';
Valid values are: SSL, SSLv3, TLS, TLSv1, TLSv1.1, TLSv1.2, SSL_TLS, and SSL_TLSv2
Note: SSLv3 is disabled by default in IBM Integration Bus Version 10.0, because SSLv3 is no longer considered secure; see Migrating a flow that uses SSLv3 for more information.
SSLCiphers Overrides the Allowed SSL Ciphers property on the node. For example:
SET OutputLocalEnvironment.Destination.HTTP.SSLCiphers =
 'SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA';
ProxyConnectHeaders Specifies additional headers that are used if the outbound request is an SSL connection through a proxy. These additional headers are sent with the initial CONNECT request to the proxy. For example, you can send proxy authentication information to a proxy server when you are using SSL. You can send multiple headers but each one must be separated by a carriage return and a line feed (ASCII 0x0D 0x0A), in accordance with RFC2616; for example:
DECLARE CRLF CHAR CAST(X'0D0A' AS CHAR CCSID 1208);
SET OutputLocalEnvironment.Destination.HTTP.ProxyConnectHeaders =
'Proxy-Authorization: Basic Zm5lcmJsZTpwYXNzd29yZA==' || CRLF || 
'Proxy-Connection: Keep-Alive' || CRLF;
This setting is used only if the request is an SSL request through a proxy server. To send proxy authentication information for a non-SSL request, specify the individual headers in the HTTPRequestHeader folder, as shown in the following example:
SET OutputRoot.HTTPRequestHeader."Proxy-Authorization" =
 'Basic Zm5lcmJsZTpwYXNzd29yZA==';
SET OutputRoot.HTTPRequestHeader."Proxy-Connection" = 'Keep-Alive';
UseFolderMode Sets the UseFolderMode. Use for bitstream generation; for certain parsers this changes the output bitstream. For example:
SET OutputLocalEnvironment.Destination.HTTP.UseFolderMode = TRUE;
QueryString Allows the setting of outbound query string parameters. Each parameter must be set individually. For example:
SET OutputLocalEnvironment.Destination.HTTP.QueryString.param1 = 'my"Value"1';
SET OutputLocalEnvironment.Destination.HTTP.QueryString.param2 = 'my"Value"2';
The above ESQL results in the following query string being encoded (according to http://tools.ietf.org/html/rfc3986) and sent with the outbound request:
?param1=my%22Value%221&param2= my%22Value%222
If the destination URL already has one or more query parameters, additional parameters specified here are appended to the existing list.
QueryStringCCSID Specifies that, before encoding, the query string parameters must be converted into a character set other than the default, which is UTF-8. Any query string parameters are first converted into the specified CCSID before the resulting string is encoded, according to RFC3986. For example:
SET OutputLocalEnvironment.Destination.HTTP.QueryStringCCSID = 943;
The above ESQL results in any QueryString parameters being converted to the 943 code page before they are encoded. Note: Any query string parameters must contain the data in unicode.
QueryStringPercentEncodeSpace Specifies that any space character in the QueryString be percent encoded and then sent with outbound request. By default any space character in the QueryString is converted to "+" character. For example:
SET OutputLocalEnvironment.Destination.HTTP.QueryStringPercentEncodeSpace = TRUE;
QueryStringSendWithRedirect Specifies that the QueryString be sent with HTTP(S) redirect. For example:
SET OutputLocalEnvironment.Destination.HTTP.QueryStringSendWithRedirect = TRUE;
QueryStringDoNotPercentEncodeCharacters Specifies any characters in the QueryString that are not to be percent encoded while the URI is built. For example:
SET OutputLocalEnvironment.Destination.HTTP.QueryStringDoNotPercentEncodeCharacters = 'ch"ar';
QueryStringSemicolonSeparator Specifies that the semicolon be used as a separator between query strings instead of "&". For example:
SET OutputLocalEnvironment.Destination.HTTP.QueryStringSemicolonSeparator = TRUE;
Compression Overrides the Use compression property on the node. For example:
SET OutputLocalEnvironment.Destination.HTTP.Compression =
 'gzip';
To set a minimum size (in bytes) at which compression is applied, use the following override: 

SET OutputLocalEnvironment.Destination.HTTP.MinimumCompressionSize = 1048576;
SSL authentication alias Overrides the SSL authentication alias property for the client-side of an HTTP connection on the node. For example:
SET OutputLocalEnvironment.Destination.HTTP.KeyAlias = 'Key1Alias';
EnableCRLCheck Overrides the Enable Certificate Revocation List checking property on the node. For example:
SET OutputLocalEnvironment.Destination.HTTP.EnableCRLCheck = 'true';
ServicePrincipalName Specifies the Service Principal Name (SPN) to use when the integration node negotiates the Kerberos security protocol. For example:
SET OutputLocalEnvironment.Destination.HTTP.ServicePrincipalName = 
'HTTP/iib.iibservice2.com:7800';
Start of changeFollowRedirectionEnd of change Start of changeOverrides the Follow HTTP(S) redirection property on the node. For example:
SET OutputLocalEnvironment.Destination.HTTP.FollowRedirection = TRUE;
End of change

Working with WrittenDestination data

After the request has been made, the WrittenDestination folder in the local environment is updated with the URI to which the request was sent and compression details (if used). A WrittenDestination for an HTTPRequest node has the following format, with Compression present only if it is used:
WrittenDestination = (
   HTTP  = (
      RequestURL = 'http://127.0.0.1:7800/HTTPFLOW' (CHARACTER)
      Compression = (
        OriginalSize = 53 (INTEGER)
        CompressedSize = 71 (INTEGER)
      )
   )
)