What's new in JSON, Version 12.10

This publication includes information about new features and changes in existing functionality.

For a complete list of what's new in this release, go to What's new in Informix®.

Table 1. What's new in JSON for IBM Informix Version 12.10.xC12
Overview Reference
Support for Mongo Compass. Command utilities and tools
Table 2. What's new in JSON for IBM Informix Version 12.10.xC11
Overview Reference
Upgrade to BSON 3.5 library.  
MQTT listener support for publishing to non-BSON timeseries tables. Loading time series data with the MQTT protocol
Table 3. What's new in JSON for IBM Informix Version 12.10.xC10
Overview Reference
Listener support for creating array indexes. Index creation
Listener support for $bigint as an index type. Index creation
Table 4. What's new in JSON for IBM Informix Version 12.10.xC9
Overview Reference
Ability to configure the listener to automatically retry a statement on certain SQL error codes. The wire listener configuration file
Table 5. What's new in JSON for IBM Informix Version 12.10.xC8
Overview Reference
Consistent sharded insert, update, and delete operations

When you run sharded operations that insert, update, or delete data, the transactions are now applied with the two-phase commit protocol instead of being eventually consistent. Data is moved to the appropriate shard server before the transaction is committed.

For sharding with MongoDB commands, you must set the sharding.enable wire listener configuration parameter to enable consistent sharded insert, update, and delete operations.

Preparing shard servers
Complex text search with regular expressions

The JSON wire listener now supports searching with regular expressions with the $regex operator in MongoDB commands.

Query and projection operators
JSON compatibility enhancements

JSON compatibility is enhanced in the following ways:

  • Run MongoDB or REST API queries with native cursors

    By default, the wire listener separates MongoDB and REST API queries into multiple queries that each return up to the number documents set by the response.documents.count.default property. You can now also enable a native cursor for a MongoDB or REST API query with the new $nativeCursor or &nativeCursor query modifier. A native cursor holds open a true cursor on the Informix database server. A native cursor requires more listener resources because connections and result set objects are tied to a single session, but the cursor guarantees consistent query results. For REST API queries, use the new killCursors command to close the cursor.

  • Cursor idle timeout

    You can control the cursor idle timeout with the new cursor.idle.timeout wire listener property.

  • Client connections to the wire listener are now timed out by default

    The new default value for the listener.idle.timeout parameter is 300000. This parameter sets the amount of time, in milliseconds, that a client connection to the wire listener can idle before it is forcibly closed. The value 0, which was the previous default value, indicates that client connections are never timed out.

  • Wire listener HTTP request headers have a size limit by default

    The new listener.http.headers.size.maximum parameter specifies the maximum size of incoming HTTP request headers. The default value is 8192 bytes.

query operators and modifier

The wire listener configuration file

Table 6. What's new in JSON for IBM Informix Version 12.10.xC7
Overview Reference
JSON compatibility enhancements

JSON compatibility is improved in the following ways:

  • Faster updates of JSON documents

    Updates that you make to JSON documents now run faster because they are run in the database server instead of in the wire listener. When you run a MongoDB update operation on a document in an Informix database, the database server internally runs the update operation with the BSON_UPDATE function. The wire listener now sends document updates to the database server first by default. Previously, the wire listener processed document updates by default. The default setting of the update.mode parameter in the wire listener configuration file is now mixed instead of client.

  • More options for SQL statements and transactions

    Bind host variables to SQL statements: You can now include host variables when you run SQL statements through the MongoDB API. The ability to run SQL statements must be enabled by the security.sql.passthrough=true wire listener configuration parameter.

    Securely run SQL functions and stored procedures: You can now run an SQL function or stored procedure even when the ability to run other SQL statements is disabled by the security.sql.passthrough=false wire listener configuration parameter. Include the Informix JSON command runFunction or runProcedure in a command document for the db.runCommand command or the GET REST API method.

    Run batch transactions: You can now run multiple command documents as a single transaction. Include command documents with the execute parameter of the Informix JSON transaction command.

  • MongoDB compatibility

    MongoDB shell version 3.2 support: You can now run commands with version 3.2 of the MongoDB shell.

    GridFS support: You can now use the GridFS functionality that is available in the MongoDB drivers for storing and retrieving BSON documents that are larger than 16 MB.

  • Aggregate or slice time series data

    You can now use the MongoDB aggregation framework pipeline to aggregate time series data or return the time series element in a time range.

The wire listener configuration file

Running SQL commands by using the MongoDB API

Informix JSON commands

Database commands

Aggregate or slice time series data

Load JSON documents with the MQTT protocol

You can now load JSON documents into Informix databases with the MQTT protocol by defining an MQTT wire listener. The MQTT protocol is a light-weight messaging protocol that you can use to load data from devices or sensors. For example, you can use the MQTT protocol to publish data from sensors into a time series that contains a BSON column. You can also use the MQTT protocol to publish data to relational tables or JSON collections.

You configure an MQTT wire listener by setting the listener.type=mqtt parameter in the wire listener configuration file. From an MQTT client, you send PUBLISH packets to insert data. You cannot, however, query data from MQTT clients with SUBSCRIBE packets. You can authenticate MQTT client users through the wire listener.

MQTT protocol

Loading time series data with the MQTT protocol

Table 7. What's new in JSON for IBM Informix Version 12.10.xC6
Overview Reference
Parallel sharded queries

You can now run SELECT statements in sharded queries in parallel instead of serially on each shard. Parallel sharded queries return results faster, but also have the following benefits:

  • Reduced memory consumption: Table consistency is enforced on the shard servers, which eliminates the processing of data dictionary information among the shard servers.
  • Reduced network traffic: Client connections are multiplexed over a common pipe instead of being created individual connections between each client and every shard server. Client connections are authenticated on only one shard server instead of on every shard server. Network traffic to check table consistency is eliminated.

To enable parallel sharded queries, set the new SHARD_ID configuration parameter in the onconfig file to a unique value on each shard server in the shard cluster. Also set the new sharding.parallel.query.enable=true and sharding.enable=true parameters in the wire listener configuration file for each shard server. You can customize how shared memory is allocated for parallel sharded queries on each shard server by setting the new SHARD_MEM configuration parameter. You can reduce latency between shard servers by increasing the number of pipes for SMX connections with the new SMX_NUMPIPES configuration parameter.

If you plan to upgrade your existing shard cluster from a previous version of Informix 12.10, upgrade and set the SHARD_ID configuration parameter on all the shard servers to enable parallel sharded queries.

JSON data sharding
MongoDB 2.6 and 3.0 compatibility

Informix now supports the following MongoDB commands:

  • The following database management commands:
    • The query and write operation commands insert, update, and delete.
    • The instance administration commands createIndexes, listCollections, and listIndexes.
    • The user management commands, for MongoDB 2.6 and later, createUser, updateUser, dropUser, dropAllUserFromDatabase, grantRolesToUser, revokeRolesFromUser, and usersInfo.
    • The role management commands: createRole, updateRole, dropRole, dropAllRolesFromDatabase, grantPrivilegesToRole, revokePrivilegesFromRole, grantRolesToRole, revokeRolesFromRole, and rolesInfo.
  • The query and projection command $eq.
  • The field update operators $mul, $min, $max, and $currentDate.
  • The pipeline aggregation operator $out.

You can authenticate MongoDB clients with the MongoDB 3.0 SCRAM-SHA-1 authentication method. You must upgrade the user schema for existing users.

You upgrade to MongoDB 3.0 by setting the new mongo.api.version parameter to 3.0 in the wire listener configuration file.

Query and projection operators

Database commands

Aggregation framework operators

Configuring MongoDB authentication

Wire listener enhancements

The wire listener has the following new parameters that you can set to customize the wire listener.

MongoDB compatibility
Specify the version of MongoDB API compatibility with the mongo.api.version parameter.
Security
Disable commands with the command.blacklist parameter.
Specify the authentication type with the db.authentication parameter.
Specify an IP address as the administrative host with the listener.admin.ipAddress parameter.
Set authentication timeout with the listener.authentication.timeout parameter.
Add information to HTTP headers with the listener.http.headers parameter.
Resource management
Configure a memory monitor to reduce resource usage with the listener.memoryMonitor parameters.
Create a separate thread pool for administrative connections with the listener.pool.admin.enable parameter.
Specify the timeout periods for socket connections with the listener.socket.accept.timeout and listener.socket.read.timeout parameters.
Suppress pooled connection checking with the pool.lenient.return.enable and the pool.lenient.dispose.enable parameters.
Specify the number of maintenance threads for connection pools with the pool.service.threads parameter.
The wire listener configuration file
Authenticate wire listener connections with Informix

You can configure the database server to authenticate MongoDB client users, who connect through the wire listener, with a pluggable authentication module (PAM). Because you administer user accounts through the database server, you can audit user activities and configure fine-grained access control. In contrast, if you use MongoDB authentication, MongoDB clients connect to the database server as the wire listener user that is specified by the url parameter.

Configuring database server authentication with PAM (UNIX, Linux)
Starting the wire listener for the REST API

You no longer need to provide the path to tomcat when you start the wire listener for the REST API.

Starting the wire listener
Table 8. What's new in JSON for IBM Informix Version 12.10.xC5
Overview Reference
High availability for MongoDB and REST clients

You can provide high availability to MongoDB and REST clients by running a wire listener on each server in your Informix high-availability cluster.

You can also provide high availability between the wire listener and the Informix database server. Connect the wire listener to the database server through the Connection Manager or specify an sqlhosts file in the url parameter in the wire listener properties file.

High availability support in the wire listener
Wire listener configuration enhancements

You can set these new or updated parameters in the wire listener properties file:

  • url parameter can include JDBC environment variables.
  • New: listener.hostName parameter can specify the listener host name to control the network adapter or interface to which the wire listener connects.
  • New: collection.informix.options parameter can specify table options to automatically add shadow columns or enable auditing when you create a JSON collection.
  • New: command.listDatabases.sizeStrategy parameter can specify a strategy for computing the size of a database when you run the MongoDB listDatabases command.
  • New: fragment.count parameter can specify the number of fragments to create for a collection.
  • New: jdbc.afterNewConnectionCreation parameter can specify SQL statements, for example, SET ENVIRONMENT statements, to run after connecting to the database server.
The wire listener configuration file
Wire listener query support

The wire listener now supports these types of queries:

  • Join queries on JSON data, relational data, or both JSON and relational data.
  • Array queries on JSON data with the MongoDB $elemMatch query operator.
Running join queries by using the wire listener

Query and projection operators

Enhanced account management through the wire listener

You can control user authorization to Informix databases through the wire listener by locking and unlocking user accounts or individual databases with the new Informix JSON lockAccount and unlockAccounts commands.

Informix JSON commands
Manipulate JSON and BSON data with SQL statements

You can use SQL statements to manipulate BSON data. You can create BSON columns with the SQL CREATE TABLE statement. You can manipulate BSON data in a collection that was created by a MongoDB API command. You can use the CREATE INDEX statement to create an index on a field in a BSON column. You can insert data with SQL statements or Informix utilities. You can view BSON data by casting the data to JSON format or running the new BSON value functions to convert BSON field values into standard SQL data types, such as INTEGER and LVARCHAR. You can use the new BSON_GET and BSON_UPDATE functions to operate on field-value pairs.

Manipulate BSON data with SQL statements
Table 9. What's new in JSON for IBM Informix Version 12.10.xC4W1
Overview Reference
Support for CORS requests in the REST API (12.10.xC4W1)

You can now set up cross-origin resource sharing (CORS) with the REST API. To do so, set the following optional parameters that were added to the jsonListener.properties file:

  • listener.http.accessControlAllowCredentials
  • listener.http.accessControlAllowHeaders
  • listener.http.accessControlAllowMethods
  • listener.http.accessControlAllowOrigin
  • listener.http.accessControlExposeHeaders
  • listener.http.accessControlMaxAge

Use these parameters to configure the HTTP headers of all responses. The HTTP headers provide access to JSON fields that are required by synchronous JavaScript + XML (AJAX) applications in a web browser when these applications access the REST listener.

The wire listener configuration file

Table 10. What's new in JSON for IBM Informix Version 12.10.xC4
Overview Reference
Basic text searching support for JSON and BSON data

You can now create a basic text search index on columns that have JSON or BSON data types. You can create the basic text search index on JSON or BSON data types through SQL with the CREATE INDEX statement or on BSON data types through the Informix extension to MongoDB with the createTextIndex command. You can control how JSON and BSON columns are indexed by including JSON index parameters when you create the basic text search index. You can run a basic text query on JSON or BSON data with the bts_contains() search predicate in SQL queries or the $ifxtext query operator in JSON queries.

Informix JSON commands

query operators and modifier

Enhanced JSON compatibility

Informix now supports the following MongoDB 2.4 features:

  • Cursor support so that you can query large volumes of data.
  • Text search of string content in collections and tables.
  • Geospatial indexes and queries.
  • Pipeline aggregation operators.
  • The array update modifiers: $each, $slice, $sort.

You can perform the following new tasks that extend MongoDB functionality in your JSON application:

  • Import and export data directly with the wire listener by using the Informix JSON commands exportCollection and importCollection.
  • Configure a strategy for calculating the size of your database by using the Informix extension to the MongoDB listDatabases command: sizeStrategy option or command.listDatabases.sizeStrategy property.

You can customize the behavior of the wire listener by setting new properties. For example, you can control logging, caching, timeout, memory pools, and the maximum size of documents.

Database commands

Query and projection operators

Update operators

Aggregation framework operators

Informix JSON commands

The wire listener configuration file

Access Informix from REST API clients

You can now directly connect applications or devices that communicate through the REST API to Informix. You create connections by configuring the wire listener for the REST API. With the REST API, you can use MongoDB and SQL queries against JSON and BSON document collections, traditional relational tables, and time series data. The REST API uses MongoDB syntax and returns JSON documents.

REST API
Create a time series with the REST API or the MongoDB API

If you have applications that handle time series data, you can now create and manage a time series with the REST API or the MongoDB API. Previously, you created a time series by running SQL statements. For example, you can program sensor devices that do not have client drivers to load time series data directly into the database with HTTP commands from the REST API.

You create time series objects by adding definitions to time series collections. You interact with time series data through a virtual table.

Manage time series through the wire listener
Table 11. What's new in JSON for IBM Informix Version 12.10.xC3
Overview Reference
Use the Mongo API to access relational data

You can write a hybrid MongoDB application that can access both relational data and JSON collections that are stored in Informix. You can work with records in SQL tables as though they were documents in JSON collections by either referencing the tables as you would collections, or by using the $sql operator on an abstract collection.

About the JSON compatibility

Running SQL commands by using the MongoDB API

Running MongoDB operations on relational tables

Improved JSON compatibility
Informix now supports the following MongoDB features:
  • The findAndModify command, which performs multiple operations at the same time.
  • The MongoDB authentication methods for adding users and authenticating basic roles, such as read and write permissions for database and system level users.
Collection methods

Database commands

The wire listener configuration file


Copyright© 2018 HCL Technologies Limited