IBM Support

Securing the InfoSphere Information Server Zookeeper and Kafka services

Question & Answer


Question

How can I secure Apache Zookeeper, Kafka, and Solr services installed by the InfoSphere Information Server?

Cause

After installing Governance Rollup Patch 1 on InfoSphere Information Server 11.5 or after installing InfoSphere Information Server, 11.5.0.1 or later versions, the newly installed Apache Zookeeper, Kafka, and Solr cloud services in Services tier are running without security scheme. Each user who knows the hostname and the port number of the services is allowed to connect to them, add, update, or delete data and metadata.

Answer

Prerequisite: This document applies to InfoSphere Information Server version 11.5 with Governance Rollup Patch 1, and to later versions.

Conventions:

  • Symbols in italic and in angle brackets are placeholders. Change them with the appropriate values of your configuration.
  • If not otherwise indicated, all listed paths are relative paths from the directory where the Shared Open Source patch is installed. When needed, this path is indicated as <shared-open-source>. The default installation path is:
    AIX or Linux: 
    cd /opt/IBM/InformationServer/shared-open-source

    Windows: 
    cd C:\IBM\InformationServer\shared-open-source
  • In Windows, use CMD window to run commands, not PowerShell.

Firewall

To prevent unallowed access to the Apache Zookeeper, Kafka, and Solr cloud services, we recommend protecting the access to the service ports through your firewall configuration. The configuration depends on the firewall you use and is not explained here.

You can find the port numbers in the configuration files of the services listed here:

  • If you have InfoSphere Information Server version 11.5 with Governance Rollup Patch 1 or InfoSphere Information Server version 11.5.0.1:
    - property port=59092 in shared-open-source/kafka/conf/server.properties
    - option -p 58983 in shared-open-source/solr/start-solr.*
    - property clientPort=52181 in shared-open-source/zookeeper/conf/zookeeper-config.cfg
  • If you have InfoSphere Information Server version 11.5.0.1 with Governance Rollup Patch 2 or later, all ports are in the single configuration file conf/env.sh on Unix or conf\env.cmd on Windows. Refer to the values of variables KAFKA_PORT, SOLR_PORT and ZOOKEEPER_CLIENT_PORT.
 

Kafka


Starting with RUP7, Information Server uses Kafka 0.10.0.1 that provides security features such as SSL over the wire.

The following procedure describes how to enable SSL secured client to broker communication as well as how to enable SSL for Information Server Kafka events. In this example, self-signed certificates are used.

Note: This example is limited to a single node Kafka configuration. The Kafka broker configuration (server.properties) is overwritten by any subsequent RUP installation.
 

Procedure for enabling the Kafka broker SSL feature

Generate SSL key and SSL certificate for the broker.

  1. Make sure you have at least OpenSSL 1.0.2 installed and in the path. In order to check the version, use the following command: 
    openssl version
  2. Generate the key in a temporary keystore in the wanted location by using Java. In this example, the keystore is created in the current directory. The key is valid for 10 years. 
    keytool -keystore kafka.server.keystore.jks -alias localhost -validity 3650 -genkey -keypass keypass -storepass storepass -dname "CN=mdmdemowin, OU=Data Quality, O=IBM, L=Boeblingen, S=BW, C=DE"
  3. Generate a certificate authority (CA) that is later used to sign certificates. 
    openssl req -new -x509 -keyout ca-key -out ca-cert -days 3650 -passout pass:ca-password -subj "/C=DE/ST=BW/L=Boeblingen/O=IBM/OU=Data Quality/CN=mdmdemowin"
  4. Add the generated CA to the truststore. 
    keytool -keystore kafka.server.truststore.jks -alias CARoot -import -file ca-cert -storepass storepass
  5. Sign the certificates in the keystore by using the CA generated before.
    • Export the certificate from the keystore. 
      keytool -keystore kafka.server.keystore.jks -alias localhost -certreq -file cert-file -storepass storepass -keypass keypass
    • Sign the certificate. 
      openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 3650 -CAcreateserial -passin pass:ca-password
  6. Import both the certificate of the CA and the signed certificate into the keystore.
    • Import the CA certificate. 
      keytool -keystore kafka.server.keystore.jks -alias CARoot -import -file ca-cert -storepass storepass
    • Import the signed certificate. 
      keytool -keystore kafka.server.keystore.jks -alias localhost -import -file cert-signed -storepass storepass -keypass keypass
  7. Move the keystore and truststore files to a location of your choice. 
    move *store.jks C:\temp
  8. Configure the Kafka broker for secure client to broker communication.

    Update the server.properties file. The following example shows the default name and location.
    \InformationServer\shared-open-source\kafka\conf\server1.properties
    • Comment out or remove the port information:
      Change
      port=59092
      to
      #port=59092
    • Add the keystore and truststore information as well as the port information. In this example, the SSL port is 59093. Adjust the passwords and file location as needed.
        
      listeners=PLAINTEXT://localhost:59092,SSL://localhost:59093
ssl.keystore.location=C:/temp/kafka.server.keystore.jks
ssl.keystore.password=storepass
ssl.key.password=keypass
ssl.truststore.location=C:/temp/kafka.server.truststore.jks
ssl.truststore.password=storepass
    • Restart the Kafka service. To check whether the configuration was successful, use the following command: 
      openssl s_client -connect localhost:59093 -tls1


      The output shows a successful connection and certificate details as provided during certificate creation.

      Example output: 
      Certificate chain
0 s:/C=DE/ST=BW/L=Boeblingen/O=IBM/OU=Data Quality/CN=mdmdemowin
i:/C=DE/ST=BW/L=Boeblingen/O=IBM/OU=Data Quality/CN=mdmdemowin
All clients can now connect to port 59093 using SSL as protocol. A truststore with the self-signed certificate needs to be present on the client. The following examples show how to connect using SSL.


The following examples show how Kafka clients can connect using secured client to broker communication.

Command line producer example:

The producer.config file needs to be enhanced as follows:

security.protocol=SSL
ssl.truststore.location=C:/temp/kafka.server.truststore.jks
ssl.truststore.password=storepass



 

kafka-console-producer --broker-list localhost:59093 --topic test --producer.config ..\..\config\producer.properties

Java client connection example:

Properties props = new Properties();



props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, “brokerHost:brokerPort");
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,StringSerializer.class.getName());
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,StringSerializer.class.getName());
//SSL
props.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, SecurityProtocol.SSL.name());
props.put("ssl.truststore.location","C:/temp/kafka.server.truststore.jks");
props.put("ssl.truststore.password","storepass");

KafkaProducer<String,String> kafkaProducer = new KafkaProducer<String,String>(props);

Procedure for enabling SSL for Information Server Kafka events

Note: This feature is available in Rollup Patch 8, and later.

  1. Upload the kafka.server.truststore.jks that was created before to Zookeeper. The default Zookeeper node name is '/kafka.server.truststore.jks'. You can specify another location by setting the Information Server key 'com.ibm.iis.events.kafka.truststoreZookeeperURI'. 
    shared-open-source\solr\install\server\scripts\cloud-scripts\zkcli -zkhost localhost:52181 -cmd putfile /kafka.server.truststore.jks C:\temp\kafka.server.truststore.jks
  2. Set up the Kafka security protocol (SSL) Information Server key:
    AIX or Linux: 
    ASBServer/bin/iisAdmin.sh -s -k com.ibm.iis.events.kafka.securityProtocol -value SSL

    Windows: 
    ASBServer\bin\iisAdmin.bat -s -k com.ibm.iis.events.kafka.securityProtocol -value SSL
  3. Encrypt the truststore password for Information Server:
    AIX or Linux: 
    ASBServer/bin/encrypt.sh storepass
{iisenc}pXOSozvaBVpxwcWJmG6Dnw==

    Windows: 
    ASBServer\bin\encrypt.bat storepass
{iisenc}pXOSozvaBVpxwcWJmG6Dnw==
  4. Set up the truststore password Information Server key to the just encrypted password value:
    AIX or Linux: 
    ASBServer/bin/iisAdmin.sh -s -k com.ibm.iis.events.kafka.truststorePassword -value {iisenc}pXOSozvaBVpxwcWJmG6Dnw==

    Windows: 
    ASBServer\bin\iisAdmin.bat -s -k com.ibm.iis.events.kafka.truststorePassword -value {iisenc}pXOSozvaBVpxwcWJmG6Dnw==
  5. Restart Information Server.
Solr

SolrCloud (collections authentication)

If you have InfoSphere Information Server version 11.5.0.1 with Governance Rollup Patch 3 or later, the Information Server components that use Solr for indexing their data support authentication. We recommend protecting the Solr collections as follows.


How to:
  1. Enable authentication and authorization for Solr, define an admin user and a non-admin user, and restrict security access to the admin user and collections access to both users.
    1. Create a file named security.json with the following content. You can save this file in /tmp on AIX or Linux, or in c:\temp on Windows.
      This file specifies an admin user "solr" with password "SolrRocks".
      You can use other user name than "solr" in line 3 and 7. You can change this password later by using the Solr REST API "set-user", refer to the sample command in step c. 
      { "authentication":
  { "class": "solr.BasicAuthPlugin",
    "credentials": { "solr": "IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
  },
  "authorization":
  { "class": "solr.RuleBasedAuthorizationPlugin",
    "user-role": { "solr": ["admin", "user"] },
    "permissions": [ { "name": "security-edit", "role": "admin" } ]
  }
}
    2. Load the file security.json above into zookeeper to zNode /security.json. All solr nodes will instantly pick up the file and enable security. We assume that the file created in item 1 is located in /tmp on AIX or Linux, and in c:\temp on Windows. If not, modify the command to refer to the actual path.
      AIX or Linux: 
      # Set JAVA_HOME to the JRE used for running Solr
### For for Information Server 11.7.1.4 or lower version
JAVA_HOME=/opt/IBM/InformationServer/jdk/jre

### For Information Server 11.7.1.4 sp1 or higher
JAVA_HOME=/opt/IBM/InformationServer/jdk-11

export JAVA_HOME

shared-open-source/solr/install/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:52181 -cmd putfile /security.json /tmp/security.json

      Windows: 
      REM Add the directory including java used for running Solr to the PATH
REM ### For Information Server 11.7.1.4 or lower
set PATH=C:\IBM\InformationServer\jdk32\jre\bin;%PATH%

REM ### For Information Server 11.7.1.4 sp1 or higher
set PATH=C:\IBM\InformationServer\jdk-11\bin;%PATH%

shared-open-source\solr\install\server\scripts\cloud-scripts\zkcli.bat -zkhost localhost:52181 -cmd putfile /security.json c:\temp\security.json

      Note: you can check if the zookeeper file /security.json created correctly using "zkcli.sh -cmd get /security.json -zkhost localhost:52181" command.
    3. If you need to use other password than "SolrRocks", change the user's password using following command. 
      curl -X POST -H "Content-Type: application/json" -u <solr-admin-username>:SolrRocks -d "{ \"set-user\": { \"<solr-admin-username>\": \"<solr-admin-password>\" } }" localhost:58983/solr/admin/authentication
      For example, if you used "solradmin" as the user name in step a and want to use "secret!" as the user's password, run 
      curl -X POST -H "Content-Type: application/json" -u solradmin:SolrRocks -d "{ \"set-user\": { \"solradmin\": \"secret!\" } }" localhost:58983/solr/admin/authentication
      Note:
      If curl is not installed in Information Server machine, you can install it or use any other REST client of your choice to post the indicated messages to the Solr REST API.
      Do not forget to specify credential "<solr-admin-username>:<solr-admin-password>" in the POST request because the Solr security-edit authorization is already enabled. 
      Optionally, you can create an additional user without admin privileges as shown in step d and e. 
    4. Add a user "solruser" with password "infosphere" and give it role "user". 
      curl -X POST -H "Content-Type: application/json" -u <solr-admin-username>:<solr-admin-password> -d "{ \"set-user\": { \"solruser\": \"infosphere\" } }" localhost:58983/solr/admin/authentication
curl -X POST -H "Content-Type: application/json" -u <solr-admin-username>:<solr-admin-password> -d "{ \"set-user-role\": { \"solruser\": \"user\" } }" localhost:58983/solr/admin/authorization
    5. Restrict access to collections dqecExceptionSets and da-datasets to users having role "user". The sample commands use executable curl that is available on Linux by default. It can be installed on AIX and Windows too. You can of course use any other REST client of your choice to post the indicated messages to the Solr REST API. Do not forget to specify credentials "solr:SolrRocks" in the POST request because the Solr security-edit authorization is already enabled. 
      curl -X POST -H "Content-Type: application/json" -u <solr-admin-username>:<solr-admin-password> -d "{ \"set-permission\": { \"name\": \"collections\", \"collection\": [\"dqecExceptionSets\", \"da-datasets\"], \"path\": \"/*\", \"role\": \"user\" } }" localhost:58983/solr/admin/authorization
  2. Declare into Information Server the non-admin user that is allowed to access the Solr collections.
    1. Encrypt the Solr authentication password for Information Server:
      AIX or Linux: 
      ASBServer/bin/encrypt.sh infosphere
{iisenc}3Jz+rq7CxMI57umgoCHPEw==

      Windows: 
      ASBServer\bin\encrypt.bat infosphere
{iisenc}3Jz+rq7CxMI57umgoCHPEw==
    2. Set up the Solr authentication user and password (encrypted) in Information Server keys:
      AIX or Linux: 
      ASBServer/bin/iisAdmin.sh -s -k com.ibm.iis.solr.search.user -value solruser
ASBServer/bin/iisAdmin.sh -s -k com.ibm.iis.solr.search.password -value "{iisenc}3Jz+rq7CxMI57umgoCHPEw=="

      Windows: 
      ASBServer\bin\iisAdmin.bat -s -k com.ibm.iis.solr.search.user -value solruser
ASBServer\bin\iisAdmin.bat -s -k com.ibm.iis.solr.search.password -value "{iisenc}3Jz+rq7CxMI57umgoCHPEw=="

SolrCloud (admin UI authentication)

If you have InfoSphere Information Server version 11.5 with Governance Rollup Patch 1 installed, or if you have InfoSphere Information Server version 11.5.0.1 but not yet Governance Rollup Patch 3 installed, the Information Server components that use Solr for indexing their data do not support authentication. Hence, we recommend protecting the access to the Solr cloud admin UI so that creating, modifying, or deleting collections is password-protected.

Even if you have InfoSphere Information Server version 11.5.0.1 with Governance Rollup Patch 3 installed, or a later version, consider protecting the access to the Solr cloud admin UI in addition to protect the Solr collections as explained before. The parts of the admin UI that display or modify collections are password-protected as soon as the above Solr collections authentication is in place. But other parts of the admin UI might display information that you might want to protect too.


How to:
  1. Append the following content to the Solr jetty context file shared-open-source/solr/install/server/contexts/solr-jetty-context.xml, at the end but inside of the <Configure> element: 
            <!-- security handling -->
        <Get name="securityHandler">
          <Set name="loginService">
            <New class="org.eclipse.jetty.security.HashLoginService">
               <Set name="name">sos-realm</Set>
               <Set name="config"><SystemProperty name="jetty.base" default="."/>/resources/sos-realm.properties</Set>
            </New>
          </Set>
        </Get>
  2. Create a file shared-open-source/solr/install/server/resources/sos-realm.properties containing one line per user with the user name, the obfuscated password of the user, and the list of roles of this user in followng format. 
    <solr-admin-username>: <obfuscated-solr-admin-password>,<role>
    For example, if basic authentication configured for admin user "solr" with password "SolrRocks",
    (1) obfuscate the password using "java -cp /path/to/jetty-util-<version>.jar org.eclipse.jetty.util.security.Password <password>" command, 
    # pwd
/opt/IBM/InformationServer
# ls -1 shared-open-source/solr/install/server/lib/jetty-util*.jar
shared-open-source/solr/install/server/lib/jetty-util-9.4.19.v20190610.jar
shared-open-source/solr/install/server/lib/jetty-util-9.4.44.v20210927.jar
# jdk/jre/bin/java -cp shared-open-source/solr/install/server/lib/jetty-util-9.4.44.v20210927.jar org.eclipse.jetty.util.security.Password SolrRocks
2023-11-13 21:48:20.457:INFO::main: Logging initialized @211ms to org.eclipse.jetty.util.log.StdErrLog
SolrRocks
OBF:1s9r1w8z1u2w1xmq1lkv1xmk1u2e1w8r1sbj
MD5:511f970aa30f04bde97c831f2dd316eb
    Note: the name and number of jetty-util-*.jar files can differ depending on the Information Server version.

    (2) add the following lines to shared-open-source/solr/install/server/resources/sos-realm.properties file for admin "solr" and non-admin "solruser". 
    solr: OBF:1s9r1w8z1u2w1xmq1lkv1xmk1u2e1w8r1sbj,admin
    solruser: OBF:1tvn1xff1ta81vnq1y0y1y0s1vnc1ta61xfn1tvf,user
  3. Append the following to the file shared-open-source/solr/install/server/solr-webapp/webapp/WEB-INF/web.xml, at the end but inside of the <web-app> element:
      
           <security-constraint>
          <web-resource-collection>
             <url-pattern>/admin/*</url-pattern>
           </web-resource-collection>
           <auth-constraint>
            <role-name>admin</role-name>
          </auth-constraint>
        </security-constraint>
        <login-config>
          <auth-method>BASIC</auth-method>
          <realm-name>sos-realm</realm-name>
        </login-config>
  4. Restart the InfoSrvSolrCloud service as follows:
    AIX: 
    stopsrc -s InfoSrvSolrCloud
startsrc -s InfoSrvSolrCloud

    Linux: 
    service InfoSrvSolrCloud restart

    Windows: 
    net stop InfoSrvSolrCloud
net start InfoSrvSolrCloud
  5. Log in to Solr Admin UI at http://<services-tier-hostname>:58983/solr/#/ as the solr administrator user.

CAUTION:

Please be aware that the modifications to Solr Cloud that are described in this document apply to the version of the Shared Open Source patch currently installed. Those modifications will disappear the next time you install (the next version of) the Shared Open Source patch. Before the next installation, you might want to save the modified files and merge them back manually after the installation.

[{"Type":"MASTER","Line of Business":{"code":"LOB10","label":"Data and AI"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSZJPZ","label":"IBM InfoSphere Information Server"},"ARM Category":[{"code":"a8m0z000000bqp5AAA","label":"Zookeeper - Kafka - Solr - Shared Open Source Services"}],"ARM Case Number":"","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF016","label":"Linux"},{"code":"PF033","label":"Windows"}],"Version":"11.5.0;11.7.0;11.7.1"}]

Document Information

Modified date:
18 March 2024

UID

swg21976111

Page Feedback