IBM Support

Release Notes - IBM Watson Explorer Analytical Components, Version 11.0

Release Notes


Abstract

This document describes known issues and workarounds for using the analytical components of IBM Watson Explorer Version 11.0 and subsequent fix packs.

Content


Release Notes for Version 11.0.0.3


System requirements in Fix Pack 3

This section summarizes the changes in system requirements since the release of Watson Explorer Version 11.0.0.1. For the most current information about all system requirements and supported data sources, including requirements for Content Analytics Studio, see Watson Explorer system requirements.



Watson Explorer Content Analytics Version 11.0.0.3 extends support to the following resources:
Operating systemsWindows Server 2012 Enterprise Edition Failover Cluster, as a high availability platform
Integrated application platformsIBM BigInsights for Apache Hadoop Version 4.1. For setup instructions, see Support IBM Open Platform with Apache Hadoop.
JavaIBM Java 8.0 SR2 FP10
Web browsersMicrosoft Edge Version 20 and future fix packs

What's new in Fix Pack 3?

This section summarizes enhancements in Version 11.0.0.3 and provides links to where you can find additional information.



Facet tree sorting in content analytics miner
You can now specify a default sort order for how facets are displayed in the facet tree. For details about this enhancement, see Facet tree sorting in content analytics miner.

Setting the maximum number of results in reports
In the Create a Report dialog, you can now set the maximum number of results to include in a report, for both CSV and IBM Cognos Business Intelligence (Cognos BI) reports. When requesting a report, you can choose a number from the drop-down list or enter an arbitrary number in the combo box. The drop-down candidate numbers and the default selected number can be configured on the Report pane of the analytics application customizer.

IBM BigInsights for Apache Hadoop Version 4.1


To configure support for Version 4.1:
1. Enter this command to stop Watson Explorer Content Analytics: esadmin system stopall
2. Add the following lines to the ES_NODE_ROOT/master_config/hadoop.xml file:
  <property>
    <name>iop.version</name>
    <value>4.1.0.0</value>
  </property>
3. Enter this command to restart Watson Explorer Content Analytics: esadmin system startall

Teradata FastLoad support
The export of data to Teradata relational databases was enhanced to support FastLoad technology.

Installing Fix Pack 3

For information about installing Version 11.0.0.3, download the readme file from Fix Central. For information about connecting to Fix Central to obtain the readme file or fix pack software, see the download document.



In addition to the following issues, installation issues in Version 11.0.0.1 also apply in Version 11.0.0.3. Review all issues before you install or uninstall the fix pack.

Rebuild Content Analytics Studio resources
After installing Content Analytics Studio Version 11.0.0.3, you must upgrade the workspace and rebuild all resources. Otherwise you may see invalid type priorities error when you analyze documents.

Ensure that the First Steps program is not running
You must stop the First Steps program before you run the fix pack installation program. When the program is run, some files are moved. If Watson Explorer Content Analytics processes are running, some files might be locked and prevented from being moved. The program can detect some processes and stop them as necessary, but the program cannot stop the First Steps program.

Remove an unnecessary cluster member from a distributed server installation on WebSphere Application Server Network Deployment
If you install Watson Explorer Content Analytics as a distributed server installation on WebSphere Application Server Network Deployment (cluster), an unnecessary "ICAMember-XX" cluster member is generated on the master node. This situation might cause connection errors when you use the search application or content analytics miner. To avoid connection errors, remove the "ICAMember-XX" member that is associated with the master node by using the WebSphere Application Server administrative console.

Migrate existing RDB and CSV export mapping files
If the following conditions are true, existing RDB and CSV export mapping files must be migrated:

- The system is upgraded to Version 11.0.0.3 from Watson Explorer Content Analytics Version 10.0.x or Watson Content Analytics Version 3.5 or earlier
- An existing collection is configured to support RDB or CSV export capabilities

To migrate a mapping file:
  1. Create a backup copy of the
  2. ES_NODE_ROOT/master_config/<collection_id>.indexservice/export/<export_point>_export_<type>_mapping.xml file.
  3. Copy the ES_NODE_ROOT/master_config/<collection_id>.indexservice/export/<export_point>_export_<type>_mapping.xml file to ES_NODE_ROOT/master_config/<collection_id>.indexservice/export/<export_point>_<type>_export_mapping.xml.
  4. Run the "esadmin config sync" command.

    Usage:
    <export_point> can be "crawled", "analyzed", or "searched"
    <type> can be "rdb" or "csv"

    For example, "analyzed_rdb_export_mapping.xml".

Known issues and workarounds in Fix Pack 3

In addition to the following issue, known issues in Version 11.0.0.1 also apply in Version 11.0.0.3.



Connection timeout issue when starting the Application Customizer in Firefox 38 ESR
When you access the Application Customizer with Firefox 38 ESR from the content analytics administration console, the interface might take a long time to load and you might see a connection error that says a timeout occurred. If you experience this issue, reload the Firefox browser.


Release Notes for Version 11.0.0.2


There are no fixes or enhancements for the analytical components in IBM Watson Explorer Version 11.0.0.2.



Release Notes for Version 11.0.0.1


System requirements in Fix Pack 1

This section summarizes the changes in system requirements since the release of Watson Explorer Version 11.0.0.1. For the most current information about all system requirements and supported data sources, including requirements for Content Analytics Studio, see Watson Explorer system requirements.



Version 11.0.0.1 extends support to the following resources:
  • Java update: IBM Java 8.0 SR1 FP10
  • Microsoft Windows 2008 R2 Distributed File System, including support for secure search. For details, see Support for DFS.
  • Database sources through the JDBC database crawler. To obtain the drivers, you must download them from the Cloudera website. For information about using these drivers, see Support for Cloudera.
    • Apache Hive, through Cloudera JDBC Driver for Apache Hive 2.5.15 or later
    • Cloudera Impala, through Cloudera JDBC Driver for Impala 2.5.28 or later


What's new in Fix Pack 1?

This section summarizes enhancements in Version 11.0.0.1 and provides links to where you can find additional information.



Additional language support
  • Slovak is now supported in search collections, analytics collections, and Content Analytics Studio.
  • Hebrew is now supported for Named Entity Recognition (NER) in search collections and analytics collections.

Support for Cloudera


Watson Explorer Version 11.0.0.1 extends support to Cloudera Distribution of Apache Hadoop (CDH) Version 5.4.8. Through Cloudera JDBC drivers for Apache Hive and Cloudera Impala, files hosted by a Hadoop server can be loaded as JDBC tables into a Hadoop-enabled collection. For information about using Cloudera drivers with Hadoop collections, see http://www.ibm.com/support/docview.wss?uid=swg27047250.

Toolkit for importing files into Hadoop-enabled collections
A new toolkit allows you to generate raw data store (RDS) files from various inputs and import them into a Hadoop-enabled collection. For details, see http://www.ibm.com/support/docview.wss?uid=swg27047247.

Date facets automatically created from index fields
When you create an index field, or create an index field for a column in a JDBC database table, the index field can automatically become a date facet for use in content analytics miner. For details, see http://www.ibm.com/support/docview.wss?uid=swg27047241.

Automatic index fields for JDBC database columns
You can specify an option to automatically generate index fields from column names in JDBC database tables. For details, see http://www.ibm.com/support/docview.wss?uid=swg27047239.

Importing the facet tree and index fields used by a system text analysis engine into another collection
To help you quickly set up a new content analytics collection to process PEAR analytics, you can export the facet tree and index field configuration from a system text analysis engine in one content analytics collection, and then import the configuration into another collection. For details, see http://www.ibm.com/support/docview.wss?uid=swg27047261.

Erasing collection data
You can run a command to completely erase collection data, including RDS, HDFS, and index files, and the query and import history. For details, see http://www.ibm.com/support/docview.wss?uid=swg27047243.

Support for DFS


Beginning with Version 11.0.0.1, you can configure your Content Analytics system to support secure search of Microsoft Windows Distributed File System (DFS) files.

Prerequisite setup
  • The DFS environment must consist of a Domain Controller server (Windows 2008 R2 server) and at least one member server (Windows 2008 R2).
  • All servers must be configured with DFS and a common share directory, such as \\example.server.com\<main_shares>.
  • All domain users and groups must have permission to access the DFS share directories.
  • The Content Analytics server must be a domain member in the same domain.


Configure Watson Explorer Content Analytics 11.0 Fix Pack 1 by using WebSphere Liberty Profile
    1. Install Watson Explorer Content Analytics V11.0, if it is not already installed.
    2. Install Watson Explorer Content Analytics V11.0 Fix Pack 1.
    3. Configure LDAP for Watson Explorer Content Analytics. Configure the system to use Active Directory LDAP (DC).
    4. Create a collection and enable security for the collection. Create crawlers to crawl DFS files by specifying the DFS UNC path name and the credentials for a domain user that has permission to access the share files.
    5. To test secure search, search the secure collection by logging into the user application as various domain users.

Notes

Installing Fix Pack 1

For information about installing Version 11.0.0.1, download the readme file from Fix Central. For information about connecting to Fix Central to obtain the readme file or fix pack, see the download document.



Review the following known issues before you install or uninstall the fix pack.

Rebuild Content Analytics Studio resources
After installing Content Analytics Studio Version 11.0.0.1, you must upgrade the workspace and rebuild all resources. Otherwise you may see invalid type priorities error when you analyze documents.

Ensure that the First Steps program is not running
You must stop the First Steps program before you run the fix pack installation program. When the program is run, some files are moved. If Watson Explorer Content Analytics processes are running, some files might be locked and prevented from being moved. The program can detect some processes and stop them as necessary, but the program cannot stop the First Steps program.

Installing or upgrading in a WebSphere Application Server Network Deployment environment
If you install Watson Explorer Content Analytics as a non-root user and select WebSphere Application Server Network Deployment as the application server, the Watson Explorer Content Analytics applications might not be available because the WebSphere Application Server cluster, ICACluster, has no members. If this situation occurs, run following commands to add members to ICACluster and make the applications available:

startccl.sh (UNIX) or startccl.exe (Windows)
esadmin configmanager start
esadmin configmanager WASNDaddClusterMember

Upgrading in a multiple server WebSphere Application Server Network Deployment environment
If you installed Watson Explorer Content Analytics in a distributed server configuration or in an all-on-one configuration that includes additional servers, and you use WebSphere Application Server Network Deployment Version 8.0, 8.5, or 8.5.5 as the application server, the fix pack installation program does not upgrade the search and analytics applications automatically. Additional problems affect your ability to export documents from search results.

To resolve these problems:

1. Install the fix pack on each server. First upgrade the master node and then upgrade the additional nodes.
2. Start Deployment Manager on the master node.
3. Log in to the WebSphere Application Server administrative console on the master node.
4. On the All applications page, click the ESAdminRestServer20 application. Click the Manage Modules link on the application.
5. Review the ESAdminRestServer module's mapped servers. If the module is mapped incorrectly to ESAdminServer and some web server on a node that is configured to use the search role (for example, "WebSphere:cell=kn001Cell01,node=nodeSearch,server=webserver1" and "WebSphere:cell=kn001Cell01,node=nodeMaster,server=ESAdminServer"), fix the mapping by following these steps:
      a) Click the check box for the ESAdminRestServer module.
      b) Select ESAdminServer and the web server on the master node, for example,
      "WebSphere:cell=wca001Cell01,node=nodeMaster,server=webserver1" and "WebSphere:cell=wca001Cell01,node=nodeMaster,server=ESAdminServer"
      c) Click Apply. Make sure that the selected servers are correctly mapped to the module.
6. After saving these changes, restart the ESAdminServer instance in the WebSphere Application Server administrative console.

Uninstalling the Agent for Windows File Systems Fix Pack
If you uninstall the Agent for Windows File Systems fix pack, you might see service errors or other problems when you attempt to run or stop Version 11.0 of the Agent for Windows file systems crawler. To resolve this problem, enter the command appropriate for your installation.
  • For a local user, enter the following command:
    sc create "CCA_Agent_Service" binpath= "AGENT_INSTALLATION_DIRECTORY/bin/esagentservice.exe" start= auto error= normal obj= ".\WIN_USER_NAME" password= PASSWORD displayname= "IBM Watson Explorer Content Analytics Agent for Windows File Systems"
  • For a domain user, enter the following command:
    sc create "CCA_Agent_Service" binpath= "AGENT_INSTALLATION_DIRECTORY/bin/esagentservice.exe" start= auto error= normal obj= "WIN_DOMAIN_NAME_\WIN_USER_NAME" password= PASSWORD displayname= "IBM Watson Explorer Content Analytics Agent for Windows File Systems"

    Where:

  • AGENT_INSTALLATION_DIRECTORY is the directory where Agent crawler installed.
    WIN_USER_NAME is the Windows user name that was used to install the Agent crawler.
    WIN_DOMAIN_NAME is the Windows domain name of the Windows user name.
    PASSWORD is password of the Windows user.

Known issues and workarounds in Fix Pack 1

Unless otherwise noted, known issues in Version 11.0 also apply in Version 11.0.0.1.



Apache Commons update required for sample search portlet
If you want to build a search portlet from sample code after you install Version 11.0.0.1, you must replace an Apache Commons JAR file because of a security vulnerability. For details, see http://www.ibm.com/support/docview.wss?uid=swg27047264.

IBM BigInsights for Apache Hadoop
When you configure Watson Explorer Content Analytics to use IBM BigInsights for Apache Hadoop 4.0, you no longer need to set the maximum memory size, which was a known issue in Version 11.0.0.0.

WebSphere Portal search bar integration
The WebSphere Portal search bar integration function is not supported if the Watson Explorer Content Analytics portlet is deployed to a page under some Virtual Portal.

List of collections to search is not displayed
If you are not logged in to the search application or content analytics miner and there is one or more secure data sources in a secure collection, no collections are shown in the list of collections that are available to search. The workaround for this issue is to enable login security for the application:
  • If you use the embedded web application server, enable login security in the Watson Explorer Content Analytics administration console.
  • If you use WebSphere Application Server, enable application security in the WebSphere Application Server administrative console.

Issues with copying the Admin REST API security token from the administration console
Problem: The administrator cannot access the Security > Token roles > Edit a Token User page, and thus cannot copy the Admin REST security token. The security token is needed, for example, to use the IBM Content Navigator plug-in.

Workaround: Use one of the following methods:
  • Instead of accessing the Edit a Token User page, add a new security token on the Add a Token User page.
  • Edit the ES_NODE_ROOT/master_config/admin/tokenUser.properties file, and specify the token value.

Using the Notes crawler with NRPC to search secure collections
By default, search applications and the content analytics miner are configured to use DIIOP mode, which can process secure searches against crawled documents by using the Notes DIIOP protocol. If your secure collection uses the NRPC protocol to crawl Notes documents, an error message similar to the following message occurs:

FFQEP0002E An error occurred when processing a remote API. The reason is : FFQEP1101E The identity management component failed to validate the specified user user_name.

If you want to enable secure search for NRPC sources, follow these steps to modify the configuration:
  1. Open the config.properties file for the application in a text editor. The paths for the default applications are ES_NODE_ROOT/master_config/searchserver/repo/search/default and ES_NODE_ROOT/master_config/searchserver/repo/analytics/default. To specify a custom application, replace "default" in the path with the name of your application directory.
  2. Insert the line "preferences.useNRPCForSecureSearch=true" into the file and save the change. For example, add the line at the end of the file.
  3. Restart the Watson Explorer Content Analytics system.


Release Notes for Version 11.0.0.0



System requirements in Version 11.0.0.0

This section summarizes the changes in system requirements since the release of Watson Explorer Version 10.0. For the most current information about all system requirements and supported data sources, including requirements for Content Analytics Studio, see Watson Explorer Version 11.0 system requirements.



Watson Explorer Content Analytics Version 11.0 extends support to the following operating systems, integrated application platforms, and web browsers.
Operating systems
  • Red Hat Enterprise Linux (RHEL) Server 7, x86-64
  • Red Hat Enterprise Linux (RHEL) Server 7, IBM z Systems
  • SUSE Linux Enterprise Server (SLES) 12, x86-64
  • SUSE Linux Enterprise Server (SLES) 12, IBM z Systems
Integrated application platforms
  • IBM Open Platform with Apache Hadoop
  • IBM BigInsights for Apache Hadoop
  • IBM WebSphere Portal Server 8.0 and future fix packs
Web browsers
  • Mozilla Firefox ESR 31 and future fix packs

The following operating systems, integrated application platforms, data sources, and web browsers are no longer supported in Watson Explorer Content Analytics Version 11.0. Support for later versions of most of these products is provided.
Integrated application platforms
  • IBM InfoSphere BigInsights 1.3 and subsequent fix packs
  • IBM InfoSphere BigInsights 1.4 and subsequent fix packs
  • IBM InfoSphere BigInsights Enterprise Edition 2.1 and subsequent fix packs
  • IBM WebSphere Portal Server 7.0 and subsequent fix packs
Data sources
  • IBM DB2 for IBM i - Release Overview 6.1 and subsequent fix packs
  • IBM DB2 for IBM i - Release Overview 7.1 and subsequent fix packs
  • IBM Lotus Quickr 8.2 and subsequent fix packs
  • IBM Lotus Quickr for Domino 8.5 and subsequent fix packs
  • IBM Lotus Quickr for WebSphere Portal 8.5 and subsequent fix packs
  • IBM WebSphere Portal Server 7.0 and subsequent fix packs
  • IBM Workplace Web Content Management 6.0 and subsequent fix packs
  • Lotus Web Content Management 7.0 and subsequent fix packs
  • Microsoft Exchange Server 2003 and subsequent fix packs
  • Microsoft Exchange Server 2007 and future fix packs, including SP1
  • Microsoft SQL Server Enterprise Edition 2008
  • Microsoft SQL Server Enterprise Edition 2008 R2
  • Microsoft Windows SharePoint Services 3.0 and subsequent fix packs
  • Microsoft SharePoint Server 2007 and subsequent fix packs
Web browsers
  • Microsoft Internet Explorer 8.0 and subsequent fix packs (also no longer supported by Content Analytics Studio)
  • Mozilla Firefox ESR 24 and subsequent fix packs (also no longer supported by Content Analytics Studio)

What's new?


This section describes functions that are new in Version 11.0.0.0.

Ability to scale content mining through native integration with Apache Hadoop
Watson Explorer content mining capabilities are no longer dependent on proprietary components of the IBM BigInsights Hadoop distribution. Watson Explorer Content Analytics is now certified to work with IBM BigInsights for Apache Hadoop and IBM Open Platform with Apache Hadoop.

Ability to generate queries from an RDF graph
You can now generate queries from a Resource Description Framework (RDF) graph. Step 3 in the Building dictionaries and facet trees from RDF files topic tells you how to manually specify queries. Instead of specifing queries manually, you can now right-click a resource node in the RDF graph and select Generate default query from the menu. The full ontology query with default predicates will be generated and the corresponding fields will be populated for you.

Ability to publish visualizations from content analytics miner to an Application Builder page
When you customize layouts in the content analytics miner, you can select a widget and publish it to Application Builder page by clicking a Publish button. In Application Builder, drag the widget to your preferred position on the page. The widget dynamically pulls data from content analytics miner to show the most current analysis results, such as trends, word clouds, or sentiment. This integration lets users interact with content mining visualizations without leaving Application Builder. For more information, see Showing analysis results from Content Analytics Miner in an i-Frame widget.

Enhanced user interface
Visual and other design changes in content analytics miner improve your ability to perform hypothesis-free, unstructured data analysis. Enhancements include:
  • Banners, colors, and icons are now consistent with Application Builder and other Watson technologies.
  • By default, the Facets view shows the top 50 facet values. A new Load more button lets you easily explore additional facet values.
  • By default, the Facets view sorts values by character code, which means that upper-case values are listed higher than lower-case values. A new case-insensitive sort option lets you alter this behavior.

Ability to hide facet values in content analytics miner
In the content analytics miner, you can now hide specific facet values. For example, if you configure the application to hide the word "Car" in the Part of Speech facet, and the word "Car" is indexed in the collection, "Car" is not displayed as a facet value in any of the content analytics miner views. If you load the top 50 facet values in the Facets view, but one of the facet values is "Car", the Facets view shows only 49 values.

To configure the facet values that you want to hide:
  1. Log in as the content analytics administrator on the master server.
  2. Edit the config.properties file in the master_config/searchserver/repo/analytics/APP_NAME directory.
  3. Add the facet values that you want to hide. The following example shows the required format to hide the values "brake" and "drive" in the Part of Speech facet and hide the values "Maker1"and "Maker2" in the "Maker" facet:

    facet.ignoreFacetValues={"exactMatchPattern":[{"path":["keyword","Part of Speech"],"values":["brake","drive"]},{"path":["keyword","Maker"],"values":["Maker1","Maker2"]}]}

  4. Restart the server.


New embedded AlchemyAPI solution packages
Two new solution packages provide support for using AlchemyAPI APIs. Each solution package includes a PEAR file (a UIMA annotator and its configuration files) to call AlchemyAPI services during the document processing. Fields and facets to store the analysis results are predefined in the solution packages. The following table provides guidelines for using the solution packages:

Setup requirementsYou must do the following steps before you create a collection that is based on an AlchemyAPI solution package:
  1. Visit this website to obtain an API key: http://www.alchemyapi.com/developers.
  2. Open the ES_NODE_ROOT/master_config/alchemy.properties file with a text editor and add the API key. For example: alchemy.api.key=XXX.
  3. After saving your changes to the ES_NODE_ROOT/master_config/alchemy.properties file, restart the Content Analytics system.
AlchemyLanguageAPI solution packageThe annotator in the AlchemyLanguageAPI solution package calls the Taxonomy API (http://www.alchemyapi.com/products/alchemylanguage/taxonomy). Each document is sent to the Alchemy Taxonomy API service over the Internet, and taxonomy information derived from the document is returned. The results are mapped to a field and facet that are predefined in the solution package.

One use case for the AlchemyLanguageAPI is to classify documents. For example, a finance company wants to analyze their customer inquiries. With this solution package, the inquiry documents may have "/finance/personal finance/lending/home financing" or "/finance/personal finance/lending/vehicle financing" facets. In content analytics miner, you can explore the results, such as checking Trends on each taxonomy separately.
AlchemyImageAPI solution packageThe annotator in the AlchemyImageAPI solution package calls the Image Tagging API (http://www.alchemyapi.com/products/alchemyvision/image-tagging).
The annotator extracts the URLs of image files from document content, and then sends the URLs to the Image Tagging API over the Internet. The Image Tagging API fetches the image at the URL (so the image should be publicly accessible), analyzes the image, and returns tag information that describes the image. The results are mapped to a field and facet that are predefined in the solution package.

One use case for the AlchemyImageAPI is Multi-Modal-Mining. For example, social networking services often contain a message and a link to an image, such as "This looks gorgeous http://example.com/001.jpg&quot; or "I really want to buy this http://example.com/zzz.jpg&quot;. With only text, those messages are not adequate for mining.

With this solution package, the annotator extracts the URL and the Alchemy Image Tagging API analyzes the JPG image files. If an "iPhone" tag is returned and the tag is stored as a facet, then the messages will be available for mining in content analytics miner.
Sample source codeThe source code for the annotators is available in the ES_INSTALL_ROOT/samples/annotator/AlchemyAPIAnnotator-eclipse-project.zip as an Eclipse project. You can modify the code to call other AlchemyAPI APIs and use the results for content mining.

New downloadable Concept Discovery solution package
After you install Version 11.0, you can configure Watson Explorer Content Analytics to use the AlchemyAPI Concept Tagging API to automatically identify concepts in text. A new Concepts via Alchemy view in content analytics miner can help you focus on the most highly ranked concepts or focus on concepts that are of greatest interest regardless of how they are ranked. Like other visualizations, the Concepts via Alchemy view uses yellow, orange, and red to identify values that might require investigation, such as documents that include many more occurrences of a particular concept compared to other documents in the collection.

To download and learn how to apply and use the Concept Discovery solution package that supports this view, see the Text Analytics Catalog for IBM Watson Explorer.

Deprecated functions

This section describes functions that are deprecated in Version 11.0.0.0.



RDF stores in IBM DB2
If you configure Watson Explorer Content Analytics to store data as triples in an RDF store, you must store the triples in the embedded triplestore database. Because of changes made in IBM DB2, you can no longer store the triples in a DB2 database. In the RDF stores topic in IBM Knowledge Center, disregard the discussion of how to use a DB2 database as the RDF store.

Custom global analysis
The custom global analysis function required an IBM InfoSphere BigInsights server where Jaql was installed. Because BigInsights for Apache Hadoop does not support Jaql, support for custom global analysis was removed from Watson Explorer Content Analytics. Disregard the discussions about custom global analysis in the following IBM Knowledge Center topics:

Installation

This section describes known issues and workarounds for installing Watson Explorer Content Analytics.



Upgrading to Version 11.0
For information about the supported upgrade paths, data that is migrated automatically, and upgrade procedures, see Upgrading the Watson Explorer Analytical Components to Version 11.0.

Possible "required system services are not running" error
If you do a new (fresh) installation of Watson Explorer Content Analytics Version 11.0 and select the option to use WebSphere Application Server instead of the embedded web application server, the "esadmin system startall" command might return the following error, even if the Watson Explorer Content Analytics system starts successfully: "One or more of the required system services are not running. ..." If this error occurs, access http://host_name:port/ui/analytics to verify that the application was started.

Manually creating the CCL service after installation on Windows 2012
A known issue can prevent the common communication layer (CCL) service from being created on Windows 2012, which means that the system components cannot be started correctly.  To prevent this problem, run the installation program as a Windows administrator: right-click the launchpad.exe executable and select Run as administrator.

If you discover the problem after you install Watson Explorer Content Analytics on a Windows 2012 machine, run the following command. In this command, "esadmin" is the default Content Analytics administrator ID. Be sure to specify the Content Analytics administrator ID and password that is configured for your system.

EXECUTE:"C:\Program Files\IBM\es\bin\install_service.bat" C:\\PROGRA˜1\\IBM\\es "C:\Program Files\IBM\es\esadmin" ICAService "IBM Watson Explorer Content Analytics" .\esadmin esadmin_password

Changes in RHEL affect how the system starts
If you upgrade to Watson Explorer Content Analytics V11.0 and use Red Hat Enterprise Linux Server 6.6 or 7, you must log in as the root user and run the following command after you run the installation program:

# ES_INSTALL_ROOT/bin/setinittab.sh

Optional: Remove the following line from the /etc/inittab file:
ffq:2345:once:ES_INSTALL_ROOT/bin/startcclinit.sh #ES CCL

Removing the data directory during silent uninstallation
If you want to delete the data directory when you run the uninstall program silently (such as /home/esadmin/esdata on UNIX or C:\Program Files\IBM\es\esadmin on Windows), you must add the following property to the response file:

DELETE_DATA_DIRECTORY=true

Then, run the uninstall program by entering the following command:

UNIX: ./uninstall_11.0.0.0 -i silent -f <path_to_response_file>
Windows: uninstall_11.0.0.0.exe -i silent -f <path_to_response_file>

Administration

This section describes known issues and workarounds for administering Watson Explorer Content Analytics.



Syncing changes to configuration files
If you manually edit configuration files under the ES_NODE_ROOT/master_config directory, you must run the "esadmin configmanager sync" command to apply the changes across your content analytics system. For details, see Run "esadmin configmanager sync" after manually editing configuration files.

Setting the maximum memory size for IBM BigInsights for Apache Hadoop
When you configure Watson Explorer Content Analytics to use IBM BigInsights for Apache Hadoop 4.0, you must set the Maximum memory size field on the Configure IBM InfoSphere BigInsights Server page (in the Content Analytics administration console) to a value that is equal to or larger than the values in the Default virtual memory for a job's map-task and Default virtual memory for a job's reduce-task fields in your BigInsights MapReduce2 configuration (in the BigInsights administration console).

Customizing JVM properties for specific sessions
If you add the line "sessionX.JVMOptions=..." to the ES_NODE_ROOT/master_config/colID_config.ini file to specify custom JVM properties for specific sessions, you must edit the line to add the system default JVM properties that are specified in the ES_INSTALL_ROOT/configurations/interfaces/default.ini file. This is required because JVMOptions in the colID_config.ini file replaces JVMOptions in the default.ini file.

Note: $v$d is special syntax in the default.ini file. It should be replaced by specific values, if any, in the colID_config.ini file.

Watson Explorer Content Analytics V11.0 adds a new system default JVM property:
-Duima.disable_enhanced_check_wrong_add_to_index

If the colID_config.ini file replaces JVMOptions and thus removes this property, some UIMA- related sessions will cause errors, such as:
org.apache.uima.cas.CASRuntimeException: No sofaFS for specified sofaRef found.

Enabling support for thumbnail images on Linux
Enabling thumbnails for collections on a Linux system requires two libraries to be installed: libXm and libXt. To locate the libraries so that you can install them, use the yum command:
    yum install *libXm*.*
    yum install *libXt*.*

Integration

No issues relating to the integration of Watson Explorer Content Analytics with other products have been reported.


Application development

This section describes known issues and workarounds for using the application programming interfaces.



Including custom views in a solution package
If you create custom views for a user application, an administrator can include the custom view plug-ins in a solution package. To use this feature, the plug-in directory structure must be organized in a specific way. To learn more about this feature and the directory structure requirements, see Including custom view plug-ins in a solution package.

User applications

This section describes known issues and workarounds for using enterprise search applications and the content analytics miner. No issues have been reported.


IBM Watson Explorer Content Analytics Studio (Content Analytics Studio)

This section describes known issues and workarounds for using Content Analytics Studio.



Error message when using RTC client version 5.0.2 to check in a workspace
If you use RTC 5.0.2 or later as your source control client for managing a Content Analytics Studio workspace, you might see a message about inconsistent line delimiters when you when check in a workspace. For details, see Problems occur when checking in a Content Analytics Studio workspace with RTC 5.0.2.

Important: All of the known issues, limitations, and workarounds that were documented for Content Analytics Studio in Version 10 still apply in Version 11. For details, see Release Notes - IBM Watson Explorer Analytical Components, Version 10.0.

Documentation updates

This section describes corrections to the Watson Explorer Content Analytics documentation. Corrections are also embedded as comments to topics in IBM Knowledge Center.



Exporting a UIMA pipeline for domain adaptive search
Two issues concern the "Other parameters" section in the Exporting a UIMA pipeline for domain adaptive search topic:
  • The reference to DisableStopWord is incorrect. The correct spelling is DisableStopword. This correction also applies when you click F2 on the Domain Adaptive Search window to view help in Content Analytics Studio.
  • The following statement is incorrect: "You can also specify whether queries from the same UIMA type are merged to a single query, and whether the UIMA type is enabled only when the original query is a plain text query that does not contain any special characters." Actually, the UIMA type is enabled when the original query is a plain text query without special syntax , which includes query syntax like '|' ("OR"), "IN',"WITHIN","INORDER","ANY", "samegroupas:", and so on, not just special characters like +,-,~,"", and so on.

BoardReader crawler help


Four help files about monitoring the BoardReader crawler and LDAP Entity Resolver were omitted from IBM Knowledge Center. You can read the content of the help files in this technote.

Configuring application user privileges
Disregard the information in the Configuring application user privileges topic. User privileges are configured through application customizers, not the administration console.

Content Analytics Studio
  • The Installing Content Analytics Studio topic states: "If you plan to install Content Analytics Studio with Studio Demonstrator, and if there is an existing studioworkspace folder in the Users directory, rename the folder before you install Content Analytics Studio. If the studioworkspace folder is in the Users directory, the installer program automatically renames it to studioworkspace_bak30." For Content Analytics Studio V11.0, the installer program automatically renames the studioworkplace folder to studioworkspace_bak11.
  • In V11.0, several changes affect the Configuring source control to manage Content Analytics Studio resources topic:
    • In the About this task section, the source control repositories were tested with Rational Team Concert 5.0.2, not 4.0.3.
    • In Step 3, the web address for obtaining for the Eclipse client used by Content Analytics Studio changed from http://download.eclipse.org/releases/kepler/ to http://download.eclipse.org/releases/luna/.
    • In Step 4, the path to the installation package directory changed from file:/C:/Install/RTC/403 to file:/C:/Install/RTC/502 .

Integrating with IBM BigInsights for Apache Hadoop Version 4.0


If you use IBM BigInsights for Apache Hadoop Version 4.0, disregard references to InfoSphere and Jaql in the Integrating with IBM InfoSphere BigInsights topic. In addition, disregard Step 3c in this procedure. There is no need to configure the JobTracker node for BigInsights V4.0, and the fields for specifying JobTracker information are no longer available in the Content Analytics administration console.

Installing an agent server
In the Installing an agent server topic, disregard the first paragraph under Step 4b. There is no need to specify the user name and password when you configure the agent server in the Content Analytics administration console.

Ambiguous references to high availability servers
The documentation uses the term "high availability" to describe servers that are configured to serve as backup servers for disaster recovery and to describe servers that are added to a system to provide scalability and failover support.

In a single-server installation, you can install one high availability server to support disaster recovery on Windows or AIX. In a distributed server installation, you can install two high availability servers to support disaster recovery on Windows or AIX; one server is dedicated to crawling and the other is dedicated to parsing and indexing.

All other servers that you add to a Watson Explorer Content Analytics system are designed to support increased scalability, throughput, and failover support. These additional servers, which are dedicated to search, document processing, or indexing, are not high availability servers for backup or disaster recovery purposes, and they can be installed on any of the supported operating systems.

Important: The documentation in IBM Knowledge Center about configuring support for high availability was originally written for the Windows Server 2008 environment. For current procedures, see Configuring high availability in a Windows Server 2012 environment.

[{"Product":{"code":"SS8NLW","label":"IBM Watson Explorer"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"--","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"},{"code":"PF033","label":"Windows"}],"Version":"11.0.0;11.0.0.1;11.0.0.2;11.0.0.3","Edition":"Advanced","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
17 June 2018

UID

swg27045730