Upgrading your server when connected to a remote Db2 server

If you are completing a side-by-side upgrade, and the V8.1.3 Cloud APM server that you are upgrading from is connected to an existing remote Db2 server instead of the default local Db2 server, you must back up the databases separately to avoid any issues between the existing V8.1.3 server and the new V8.1.4 server that use these databases.

Before you begin

  • When you perform the Cloud APM server upgrade, the Db2 server for the Cloud APM server V8.1.4.0 must be at the same version as the Db2 server used by the Cloud APM server V8.1.3. Check the Db2 version that is in use by the Cloud APM server V8.1.3:
    • If Db2 is installed on the same system as the Cloud APM server, enter the following command to check the Db2 version:
      install_dir/db2/V10.5/bin/db2level
      where install_dir is the directory where Db2 is installed.
    • If Db2 is installed on a remote system, ask the Db2 administrator to check the Db2 server version.
    Important:

    The Db2 server for your V8.1.4 Cloud APM server must be at the same version as your V8.1.3 Cloud APM server during the Cloud APM server upgrade.

    If the Db2 version for your V8.1.3 Cloud APM server is Db2 Advanced Enterprise Server Edition V10.5 fix pack 9 (or later), complete the steps in this technote Upgrading your Cloud APM server from V8.1.3 to V8.1.4 when your Db2 server is running V10.5 fix pack 9 or a later fix pack now to continue your Cloud APM server upgrade from V8.1.3 to V8.1.4.

    If you want to use Db2 V11.1 for your V8.1.4 Cloud APM server, you can upgrade your Db2 server to a supported version of Db2 V11.1 after the Cloud APM server upgrade completes. Follow the procedure in Upgrading the Db2 server to Db2 version 11.5.x.

  • If your Cloud APM server is connected to a remote Db2 server, Db2 Advanced Workgroup Server Edition V11.1 is supported when you are upgrading your Cloud APM server from V8.1.3 to V8.1.4.
  • If you are upgrading the Cloud APM server on the same system and you have a remote Db2 server, follow the instructions in Upgrading the server on the same system.
  • If your V8.1.3 Cloud APM server is connected to a remote MongoDB, install MongoDB V3.2.12. For more information, see Installing MongoDB V3.2.12 on your remote system.
  • AIXThe backup.sh script is not supported on AIX. Use the Db2 utilities to back up your databases. For more information, see Backup overview and BACKUP DATABASE command in the Db2 V10.5.0 topic collection on IBM® Knowledge Center.
  • AIXThe restore.sh script is not supported on AIX. Use the Db2 utilities to restore your databases. For more information, see RESTORE DATABASE command.
  • If LDAP is enabled on your V8.1.3 Cloud APM server to authenticate Cloud APM console users, complete the following steps on your V8.1.3 Cloud APM server before you back up your V8.1.3 data:
    1. Retrieve the value of the realm attribute from the install_dir/wlp/usr/shared/config/ldapRegistry.xml file.
    2. Check the value of the oauthRealm attribute in the install_dir/wlp/usr/shared/config/oauthVariables-onprem.xml file. If the value of oauthRealm attribute does not match the value of the realm attribute in the ldapRegistry.xml file, update the value of the oauthRealm attribute to match the value of the realm attribute.
    3. Complete the following steps to update the install_dir/wlp/usr/servers/apmui/server-oauth2-tai.xml file to add the user from the install_dir/wlp/usr/servers/server1/cscs/conf/cscsRoleAdmin.conf file:
      1. Find the properties line <properties, and identify the systemUser parameter, if it does not exist you will need to add it in the next step. Identify the closing tag /> for the properties line.
      2. Add a new line or edit the existing line before the /> closing tag as follows:
        systemUser="testuser LDAP distinguished name"
        where testuser matches the user string from the cscsRoleAdmin.conf file, for example:
        systemUser="CN=testuser,CN=users,dc=adtest,dc=mycity,dc=mycompany,dc=com"
        Note: Do not include the user:prefix or realm name that was specified in cscsRoleAdmin.conf.
      3. Confirm that the /> closing tag was not deleted, then save and close the file.
  • If you modified the trust store password for your Cloud APM server V8.1.3, change the password back to the default password before performing the server upgrade. After the upgrade completes, you can change the password back to your custom trust store password. For details, Changing the password for the shared truststore.
  • If a custom password is configured for MongoDB on your V8.1.3 Cloud APM server, you must set the MongoDB password back to the default value before running backup.sh. After the upgrade to version 8.1.4.0 is complete, you can set the MongoDB password back to a custom password. For more information, see Default users and passwords.

About this task

  • The Cloud APM server stores data in the WAREHOUS, SCR32, and DATAMART databases on the Db2 server. Before you install V8.1.4 of the Cloud APM server on the system that you are using for the upgrade, you must create copies of these three databases with names different from the names that were used for the V8.1.3 server.
  • In this procedure, the names of the three Db2 databases are referenced as follows:
    Table 1. Db2 database names
      V8.1.3 Database Name New V8.1.4 Database Name
    Warehouse WAREHOUS W1
    Datamart DATAMART D1
    SCR SCR32 S1
    Different names can be used for the V8.1.4 databases by substituting the desired name for the name that is listed in Table 1 in the following procedure.
  • In the V8.1.4 release, if your Cloud APM server is connected to a remote Db2 server that is also running on a V8.1.4 Cloud APM server, you can create a custom name for the Db2 instance user or you can accept the default db2apm name. In previous Cloud APM releases, the db2apm instance user name only was supported. For the V8.1.4 release, because the steps in this procedure refer to an existing remote Db2 server, which is a pre-V8.1.4 remote Db2 server, you must complete these steps as the db2apm user.
  • Most steps are run as a root user. Some steps are run as the Db2 instance user db2apm and include an su to the Db2 instance user and a subsequent exit to return to the root user.

Procedure

Complete the following steps as a root user to clone the WAREHOUS, SCR32, and DATAMART databases before the upgrade and to back up these databases separately.

  1. Download the Cloud APM server installation image. Extract the server installation files for your offering. Complete steps 1 to 4 in Upgrading the server side-by-side.
  2. Copy the backup.sh, rename.sh, and restore.sh scripts from the tools directory and the setup-dbconfig-linux_64.bin script from the packages/SCR directory on the V8.1.4 Cloud APM server installation image to the remote Db2 server system. You can connect to the existing Db2 server that was configured for Cloud APM V8.1.3.
  3. Run the backup.sh script on the remote Db2 server system. If you are using the default Cloud APM administrator user IDs and passwords, run the backup.sh without parameters.
    For example:
    [root]# /root/min/tools/backup.sh
    Performing a backup...
    Processing the "install" component...
    Processing the "db2" component...Backup operation completed.
    Creating the backup file...All tasks finished successfully.
    Your backup is available in the following file:
    /tmp/backups/backup_20160824_061708.tar
    
    If you are using non-default user IDs and passwords, run the script with these parameters. For example,
    [root]# /root/min/tools/backup.sh  -u username -p password -z external_db2_instance
    where:
    username
    The non-default Cloud APM administrator user ID.
    password
    The non-default Cloud APM administrator password
    external_db2_instance
    The remote Db2 server instance user name.
    The script outputs the name of the backup file.
  4. Run the script setup_dbconfig-linux_64.bin on the remote Db2 server system to set up the V8.1.4 SCR database. Run the script as the same Db2 instance user db2apm who ran the script during the creation of the V8.1.3. SCR database. Do not run the script as user root.
    su - db2apm
    /path/setup-dbconfig-linux_64.bin -i console
    exit
    Follow the console prompts:
    1. When you are prompted to choose a locale, enter 2 to select English.
    2. Enter 1 to accept the license agreement.
    3. Enter the absolute path to the installation folder or press Enter to accept the default path. The default path is the setup-dbconfig installation path for V8.1.3. Do not change this path, unless the path is incorrect. In this procedure, the path is referred to as /opt/ibm/db2/tbsmdb.

      The Cloud APM SCR database configuration tools installed by the setup_dbconfig-platform_64.bin script should not be installed into the same directory as the Tivoli Business Service Manager database configuration tools if the databases for both products reside on the same remote Db2 server.

    4. Enter 1 to select IBM Cloud Application Performance Management(APM) as the product that uses the database.
    5. Enter 1 to select Simple as the type of installation to run.
    6. Enter 2 to instruct the installer not to create the database schema, including the tables, table spaces, and views.
    7. Enter the following database configuration details when you are prompted:
      • Database Name (maximum 8 characters) (DEFAULT: SCR32): S1
      • Database host name or IP address (DEFAULT): host_name_or_IP_address
      • Database Port (DEFAULT: 50000): port
      • Database Path (DEFAULT): default_path
      where:
      port
      The port number that was specified when the SCR database was created. If you don't know the value, see the port number that is specified for the db2c_db2apm service in the /etc/services file.
      default_path
      The file system where the database was created. The default path is the home directory of the db2apm user. If a different file system was specified when the database was created, that file system should be specified instead.
    8. Enter 2 to specify not to encrypt the database.
  5. As user root, run the restore.sh script on the remote Db2 server system to restore the backup that you created in step 3. This backup file is restored with the new V8.1.4 database names W1, S1, and D1.
    Run the restore.sh with the following options and values:
    [root]# min/tools/restore.sh -f backup_file
    -j metric_db_name -k scr_db_name -l datamart_db_name -z external_db2_instance
    
    For example:
    [root]# min/tools/restore.sh -f /tmp/backups/backup_20150824_061708.tar 
    -j W1 -k S1 -l D1 -z db2apm
    Restoring the data...Processing the "db2" component...
    Restore operation completed.
    All tasks finished successfully.
    where:
    backup_file
    The backup file name that you created in step 3.
    metric_db_name
    The optional new name for the metric database.
    scr_db_name
    The optional new name for the topology database.
    datamart_db_name
    The optional new name for the DATAMART database.
    external_db2_instance
    The remote Db2 server instance user name.
  6. Upgrade the database schema and Java™ UDF routines in the restored Db2 SCR database.
    Enter:
    
    cd install_path/tbsmdb/tools/XMLtoolkit/bin
    ./scr_restore.sh -U db2apm -P passw0rd -n
    
    where install_path is the path that you entered when you set up setup-dbconfig.
    If you do not want the password visible on the command line, you can omit the -P flag and the script prompts for the password. For example:
    ./scr_restore.sh -n
    Enter the database password:
  7. Enter the following command as the db2apm user for each database to adjust the buffer pool and transaction log after you determine the size of your environment.
    su - db2apm
    db2 connect to W1
    db2 alter bufferpool IBMDEFAULTBP IMMEDIATE size 
    W1_PAGE_COUNT
    db2 disconnect W1
    db2 connect to D1 
    db2 alter bufferpool IBMDEFAULTBP IMMEDIATE size D1_PAGE_COUNT
    db2 disconnect to D1 
    db2 connect to S1
    db2 UPDATE DATABASE CONFIGURATION FOR S1 USING LOGSECOND 25 
    db2 alter bufferpool IBMDEFAULTBP IMMEDIATE size 5000
    db2 alter bufferpool TBSMCFG16KBP IMMEDIATE size 1000
    db2 alter bufferpool TBSM4KBP IMMEDIATE size 1000
    db2 alter bufferpool TBSM32KBP IMMEDIATE size 1000
    db2 alter bufferpool TBSMSCR16KBP IMMEDIATE size 
    S1_PAGE_COUNT
    db2 disconnect to S1
    exit
    where:
    W1_PAGE_COUNT
    The buffer pool page count (8 K page sizes): 50000 for small; 100000 for medium; or 200000 for large.
    D1_PAGE_COUNT
    The buffer pool page count (8 K page sizes): 100000 for small; 200000 for medium; or 300000 for large.
    S1_PAGE_COUNT
    The buffer pool page count (16 K page sizes): 30000 for small; 50000 for medium; 100000 for large.

    Estimate the size of your environment (small, medium, or large) from the server requirements table in Cloud APM server hardware requirements.

  8. Install the IBM Data Server Client on the virtual machine or computer system that you are using for the upgrade to Cloud APM server V8.1.4 by completing step 3 in Connecting to a remote Db2 server.
  9. Install the 8.1.4 version of the Cloud APM server. Complete steps 5 to 13 in Upgrading the server side-by-side.
    Use the information in steps 10 and 11 in this procedure to guide your responses.
  10. After the installation script detects that the backup of the V8.1.3 Cloud APM server information is configured with an existing remote Db2 server, respond to the prompts to provide or accept default values for the following configuration parameters to establish a connection with the remote Db2 database:
    1. hostname/IP address to the DB host or accept the default [localhost]:
    2. port number of the DB2 instance or accept the default [50000]:
    3. password for the user "itmuser":
    4. remote DB2 instance name or accept the default [db2apm]:
    5. password for the instance user "db2apm":
  11. When you are prompted to enter the names of the new V8.1.4 databases (W1, S1, and D1) that you created when you specified the restore.sh command previously, accept the default values or enter the new values depending on your requirements.
    If you plan to use the V8.1.3 server, enter the names of the new databases (W1, S1, and D1) that you created when you ran the restore.sh command in step 5. Otherwise, if you accept the default values, you corrupt the databases that continue to be used by V8.1.3 of the Cloud APM server.
    Important: If you are entering the new database names, these names must match the names that are used with the restore.sh script in step 5 on the remote Db2 server:
    
    name of Metric database or accept the default [WAREHOUS]: W1
    name of Topology database or accept the default [SCR32]: S1
    name of Metric database or accept the default [DATAMART]: D1

    The V8.1.4 Cloud APM server installation continues.

    If the installer detects any agent configuration packages in install_dir/ccm/depot from a previous installation of the Cloud APM server, it warns you that it renamed the old packages and created new agent packages. The old packages are named install_dir/ccm/depot.old.

    If the installer detects a keyfiles directory in install_dir from a previous installation of the Cloud APM server, it warns you that it renamed the old keyfiles directory and created a new directory. The old keyfiles directory is named install_dir/keyfiles.old.

    A prerequisite scan of your environment starts and takes a few moments to complete. When the installation finishes, V8.1.4 of the Cloud APM server is configured with the existing remote Db2 server, which is the same instance as V8.1.3 but it is using different databases. It is using the databases that were created in step 5.
  12. If you are not continuing to use the V8.1.3 Cloud APM server, you can use the V8.1.3 default database names with your V8.1.4 system by completing the following steps:
    1. When the V8.1.4 installation is complete, shut down the V8.1.3 and V8.1.4 Cloud APM server systems by entering the following command from any directory on both systems:
      apm stop_all
    2. On the remote Db2 server system, run the rename.sh script. Include the database names that you created in step 5 as arguments.
      For example, the following command renames the V8.1.4 databases from W1 to WAREHOUS, S1 to SCR32, and D1 to DATAMART:
      ./rename.sh W1 S1 D1
    3. On the V8.1.4 Cloud APM server system, start the kafka and server1 components by entering the following commands:
      
      apm start kafka
      apm start server1
    4. On the V8.1.4 upgraded system, enter the following command:
      install_dir/ccm/db_common.sh restore_original_db_names

      where install_dir is the Cloud APM server installation directory.

    The V8.1.4 Cloud APM server is reconfigured to use the original databases: WAREHOUS, SCR32, and DATAMART.

  13. Shut down the V8.1.3 Cloud APM server system if it is running by entering the following command from any directory:
    shutdown -P now
  14. Reassign the V8.1.3 Cloud APM server IP address or both the IP address and host name to the V8.1.4 server. Whether you reassign the IP address only or both the IP address and host name depends on your configuration.
  15. Migrate your Hybrid Gateway configuration:
    • To migrate the Hybrid Gateway immediately after a restore, run these commands on the Cloud APM server as the root user:
      1. Copy the restored config.properties file to the ccm directory:
        cp install_dir/ccm/properties/config.properties.restored install_dir/ccm/properties/config.properties
      2. Update the date and time of the config.properties file so that any Hybrid Gateways already running will reload their configuration:
        touch config.properties
      3. Copy the updated config.properties file to the Central Configuration Services component:
        cp install_dir/ccm/properties/config.properties install_dir/wlp/usr/servers/min/dropins/CentralConfigurationServer.war/common/config.properties
    • If any configurations were done on the upgraded system after the restore, complete these steps:
      1. Copy these two lines from theinstall_dir/ccm/properties/config.properties.restored file:
        com.ibm.tivoli.ccm.encryption\:key= 
        com.ibm.tivoli.ccm.gaian.connect\:gaianReq=
      2. Replace the same lines in install_dir/ccm/properties/config.properties with the lines that you just copied.
      3. Copy config.properties to the Central Configuration Services component:
        cp install_dir/ccm/properties/config.properties install_dir/wlp/usr/servers/min/dropins/CentralConfigurationServer.war/common/config.properties
    To verify that the config.properties file in install_dir/wlp/usr/servers/min/dropins/CentralConfigurationServer.war/common/ was replaced, check that the modified date and time are current.

Results

The Cloud APM server upgrade to V8.1.4 is complete and connected to the existing remote Db2 server that you were using with V8.1.3.

What to do next

  • If you are using the default out of the box certificates for accessing the Cloud APM console, you must complete the steps in this technote V8.1.4 Application Performance Management UI certificates are expiring in upgraded environments to update the default certificates to prevent them from expiring in April 2019.
  • Before you use the Cloud APM console that you upgraded, clear your web browser cache and restart your browser. Clearing the cache avoids display issues that new capabilities in this update introduced to some of the user interfaces.
  • If you want to use the old agent configuration packages from a previous installation for agent installations, complete these steps:
    1. Go to the install_dir/ccm directory.
    2. Delete the agentconfig file.
    3. Change the name of the agentconfig.old file to agentconfig.
  • If you configured HTTPS communication between the Cloud APM server and agents in your V8.1.3 Cloud APM server, you must change clientAuthentication to true. Copy the <ssl> xml element that contains the enabledCiphers attribute from the install_dir/wlp/usr/servers/min/server.xml file to the install_dir/wlp/usr/servers/min/user-exit.xml file if it does not already exist in the user-exit.xml file. Then add this clientAuthentication="true" line after the enabledCiphers line in the user-exit.xml file. Remove the <ssl> xml element from the server.xml file. The following code example shows you where to add the clientAuthentication="true" line in the user-exit.xml.
    <ssl
    id="defaultSSLConfig"
    sslProtocol="TLSv1.2"
    enabledCiphers="TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
    clientAuthentication="true"
    serverKeyAlias="server_key"
    clientKeyAlias="IBM_Tivoli_Monitoring_Certificate"
    keyStoreRef="defaultKeyStore"/>
  • If you configured HTTPS communication between the Cloud APM server and agents in your V8.1.3 Cloud APM server and used the default certificates, change the communication protocol that the Cloud APM server agents use to HTTPS. For instructions, see Configuring the communications protocol for server agents.
  • If you configured HTTPS communication between the Cloud APM server and agents in your V8.1.3 Cloud APM server and did not use the default certificates, complete these steps:
    1. Encode the (xor) server keystore password that you used when you created certificates for the V8.1.3 Cloud APM server:
      /opt/ibm/wlp/bin/securityUtility encode
    2. Copy the <keyStore> xml element from the install_dir/ibm/wlp/usr/servers/min/server.xml to the install_dir/wlp/usr/servers/min/user-exit.xml file if it does not already exist in the user-exit.xml file. Then in the user-exit.xml file, replace the value of the password attribute with the newly encoded password from the step 1. Remove the <keyStore> xml element from the server.xml file.
    3. Go to the install_dir directory.
    4. Delete the keyfiles directory.
    5. Change the name of the keyfiles.old directory to keyfiles.
    6. Update the certificates that are used by the monitoring agents to connect to the Cloud APM server to use the new keystore. For instructions, see Configuring certificates between the server and agents for HTTPS communication.
    7. Update the communication protocol and certificates that are used by the Cloud APM server agents. For instructions, see Configuring the communications protocol for server agents.
  • If the system where you installed the Cloud APM server is using LDAP to authenticate the root user or Db2 users, and you updated the passwords for the itmuser and the Db2 instances users when following the procedure referenced in step 9.a, then complete step 9 in the Installing on a system using an external directory service topic.
  • Review the WAREHOUS database settings tat were applied by the restore process.
    update database config for warehous using DFT_DEGREE any
    update database config for warehous using LOGBUFSZ 1024
    update database config for warehous using LOCKLIST AUTOMATIC
    update database config for warehous using SORTHEAP AUTOMATIC
    update database config for warehous using SHEAPTHRES_SHR AUTOMATIC
    update database config for warehous using NUM_IOCLEANERS AUTOMATIC
    update database config for warehous using NUM_IOSERVERS AUTOMATIC
    update database config for warehous using LOGFILSIZ 4096
    update database config for warehous using LOGPRIMARY 10
    update database config for warehous using DBHEAP AUTOMATIC
    update database config for warehous using LOGSECOND 40
    update database config for warehous using AVG_APPLS AUTOMATIC
    update database config for warehous using logarchmeth1 OFF
    update database config for warehous using logarchmeth2 OFF
    update database config for warehous using DATABASE_MEMORY 250000 AUTOMATIC
    Cloud APM does not support pruning of database logs, as a result, the logarchmeth1 and logarchmeth2 settings are set to OFF. If you back up your Warehouse database and you support log pruning in your environment, modify these settings.
  • Most V8.1.3 Cloud APM agents are compatible with the V8.1.4 Cloud APM server. However, you must upgrade the following agents after you upgrade the Cloud APM server to version 8.1.4.0:
    • If you are using the Synthetic Playback agent, you must upgrade the agent by using the latest agent package on Passport Advantage. To view the Passport Advantage part numbers for the agent packages, see: Part numbers. Then, you must apply the latest Cloud APM 8.1.4.0 server interim fix that is available from Fix Central.
    • The Monitoring Agent for WebSphere Applications must be upgraded by using the latest agent package on Passport Advantage. To view the Passport Advantage part numbers for the agent packages, see: Part numbers.
    • The Monitoring Agent for MongoDB must be upgraded by using the latest agent package on Passport Advantage. To view the Passport Advantage part numbers for the agent packages, see: Part numbers.