Migrating to MessageSight V2.0

Answer

There are two different ways to migrate from MessageSight V1.2 to V2.0:

• Config Export/import Migration


• Store Migration via HA

Config Export/Import Migration Advantages/Disadvantages

This process migrates only the configuration items from the server like Endpoints, Messaging and Connection Policies, LDAP config etc; any config object statically configured with commands by an administrator.

• Advantages:
• More flexible
• Much simpler
• Doesn't require an outage
• Easier to verify
• Easy to test and practice
• Doesn't require the V2.0 machine to have the same amount of physical memory
• Disadvantages:
• Doesn't preserve the store
• Requirements:
• Must be at the latest version of V1.2 IFIX on V1.2 server
• Assumes a V2.0 server is installed and configured and empty of all custom objects
• Considerations:
• Most customers aren't relying on message store and have clients that can recreate subscriptions and topics as needed
• With recent V1.2 IFIXES, the imaserver backup, which exports the server config can be done with the server in production mode (using the undocumented "V2Migration=True" flag) avoiding an outage
• The export is completely separate from the import step, so one export can be reused for multiple import tests
• Requires less complicated network interface setup, since this process doesn't require multiple network interfaces configured on the V2.0 server
• Once you've installed and configured a V2.0 server, you cannot reduce the amount of physical memory in that server without cleaning the store.

This process migrates both the server config as well as the entire contents of the server store. The store typically contains buffered/retained messages, subscriptions and topics; basically any persistent messages and any objects dynamically created by clients.

• Advantages:
• Preserves the store
• The store typically contains:
• Buffered messages
• Retained messages
• Subscriptions
• Client states
• Disadvantages:
• V2.0 server must have the same amount of physical memory as the V1.2 server in the HA pair
• Much more complicated process
• More difficult to test
• More difficult to avoid outages
• Must disconnect existing V1.2 standby to pair with new V2.0 server
• Requirements:
• Must be at the latest version of V1.2 IFIX on V1.2 server
• Assumes a V2.0 server is installed and configured and empty of all custom objects
• V1.2 and V2.0 servers must be able to communicate to each other over TCP/IP.
• V1.2 and V2.0 servers must be configured with the same amount of physical memory
• Considerations:
• If you must retain the contents of your store, this is the only way to migrate
• Must have multiple interfaces on your V2.0 server so you can configure HA
• If the V1.2 appliance is physical, you will need to plan for at least one outage/restart

• We do not support migration from 1.1 directly to 2.0. It may be possible to migrate using (only) the export/import migration process documented below to migrate configuration from 1.1 directly to 2.0, depending on the existing config. However, we do not support it and cannot guarantee it will work. If it fails, you will need to use the supported migration path (V1.1 --> V1.2 --> V2.0).
• An entire HA environment cannot be migrated. This process migrates a single V1.2 server (presumably your authoritative master, if 1.2 is configured with HA) to a single V2.0 server in. After that is accomplished, you will need to reconfigure V2.0 HA via the normal configuration mechanisms.
• The migration does not convert explicit IP addresses specified in the V1.2 configuration. Any explicit IP addresses specified in the config (for example, to bind an endpoint to listen on a specific interface) is not migrated to the new addresses of the new server. These need to be adjusted manually after the migration is completed.
• TLS certificates may need to be manually updated if migrating to a new host. If you have TLS certificates that are only valid for a specific host (and you are migrating to a new server host), the migration process cannot migrate these to be valid for the new host. You will need to replace these certificates after the migration completes to be valid for your new server host.
• We strongly recommend updating the V2.0 target server to be to be at the latest V2.0 IFIX level. The migration is achieved mostly by the code on the V2.0 side and this guarantees that you have the latest set of migration fixes.
• See other requirements for completing the migration below, including planning or outages, backing up the config and testing. For environments with simple config, it may be easier to simply recreate the configuration objects manually in V2.0, rather than go through the migration. If this is possible, this is recommended.

• Decide which migration method you will use: export/import config migration, or the config/store migration via HighAvailability
• We recommend using the Config Export/Import method, since it's simpler and easier to test. But if you must preserve your store, then you need to use the Store Migration method
• We strongly recommend testing and practicing either migration method multiple times, but especially the Store Migration method.
• If you can't practice/test the Store Migration , we recommend using the Config Export/Import method instead.
• Evaluate and Plan for outages, depending on your scenario
• Know what to do if something goes wrong!
• If choosing the Store Migration method for your migration, please contact IBM Support so we can review and verify your migration plan

• Changing a physical appliance to do HA replication over TCP requires an imaserver restart
• If you are migrating from a V1.2 HA config with physical appliances, once you restart after running advanced-pd-options _setconfig imaserver ReplicationProtocol 1', you must complete the migration, or revert the config to continue with normal V1.2 HA replication (advanced-pd-options _setconfig imaserver ReplicationProtocol 0', followed by another imaserver restart, (This is a hidden command, so it's not documented in the command help.)
• Any changes to the HighAvailability config requires an imaserver restart
• Experienced MessageSight admins should be able to minimize the outages for imaserver restarts to several minutes.
• While the V1.2 source primary server can be available for production traffic while the migration is taking place, we recommend disabling production traffic until the migration is complete.
• We strongly recommend practicing this migration; however that requires:
• Potential outage for taking a store backup in production (see below)
• An available appliance that's not in production for restoring the backup and practicing the migration (we do not recommend using your production appliance(s) to test this migration).
• Config Migration:
• This requires no outage. The imaserver backup command can be done while the server is up in production mode. The only limitation is that no config changes can be made while the imaserver backup command is running (but this is uncommon anyway).

• Go into maintenance mode:
imaserver runmode maintenance
imaserver stop
imaserver start
• Use the imaserver backup command to backup the store:
imaserver backup "Password=backup_password" "StoreDestinationURL=scp_url" "StoreDestinationUserID=target_server_user_id_for_scp" 'StoreDestinationPassword=target_server_user_passwd'
Example:
imaserver backup "Password=backup_password" "StoreDestinationURL=scp://10.10.0.2:/data/backup/store_backup_201605091530" "StoreDestinationUserID=root" 'StoreDestinationPassword=mypassword'
• Go back into production mode:
imaserver runmode production
imaserver stop
imaserver start
• Copy the config backup file generated by the backup to the same server as used for the store backup:
file put imaBackup_file scp_url
Example:
file put imaBackup.153121-05092016 scp://root@10.10.0.2:/data/backup
• Copy the file to the new appliance that will be used for testing:
file get scp_url imaBackup_file
Example:
file get scp://root@10.10.0.2:/data/backup/imaBackup.153121-05092016 imaBackup.153121-05092016
• Go into maintenance mode on the test appliance:
imaserver runmode maintenance
imaserver stop
imaserver start
• Restore the store backup and config backup:
imaserver restore "Name=imaBackup_file" "Password=backup_password" "Appliance=False" "StoreSourceURL=scp_url_for_store_backup" "StoreSourceUserID=target_server_user" "StoreSourcePassword=target_server_user_password"
Example:
imaserver restore "Name=imaBackup.153121-05092016" "Password=backup_password" "Appliance=False" "StoreSourceURL=scp://10.10.0.2:/data/backup/store_backup_20160509" "StoreSourceUserID=root" "StoreSourcePassword=mypassword"
• You are ready to try out the store migration process using the test box as the source V1.2 server

Overview:



• Export the config from the V1.2 source server


• Copy exported config file to commonly accessible server


• Upload the config file to the V2.0 server with a REST API call


• Import the config backup with a REST API call


• Verify and adjust the imported config


• Point application traffic at the new server



Detailed Steps:

• Export the V1.2 server config:
• Export the MessageSight config from the V1.2 source server with:

imaserver backup "Password=password" "V2Migration=True"
• Example:

imaserver backup "Password=mybackuppw" "V2Migration=True"
IBM MessageSight configuration backup imaBackup.215448-05262016 has been created.
Using this flag, the V1.2 server can backup its config without going into maintenance mode.

• Copy exported config file to commonly accessible server:
• Upload the backup file to an ssh server for storage with:

file put file scp://user@hostname_or_ip:/path/to/directory

• Upload the config file to the V2.0 server:
curl -X PUT -T /path/to/directory/imaBackup.215448-05262016
http://server:port/ima/v1/file/imaBackup.215448-05262016
• Import the config backup with a REST call:
curl -X POST http://server:port/ima/v1/service/import -d '{ "FileName":"imaBackup.215448-05262016","Password":"mybackuppw","DisableObjects":true|false,"ApplyConfig":true|false}'
The important flags one might change are:
DisableObjects parameter:
• true: All config objects (for example, endpoints) are disabled when first restarting after performing the migration. This allows the admin to verify all the config objects. If the endpoint was configured to listen on a specific IP ADDRESS, this would need to be fixed after the migration (the migration doesn't not update these addresses during the migration.
• false: All objects are enabled after the first restart after migrating. This is risky since there may be config items that need to be updated after the migration.
ApplyConfig parameter:
• false: don't apply config to V2.0 server. Convert input data to V2.0 format and print to standard output (usually so it can be redirected to a file and verified or saved).
• true: apply config to the V2.0 server so the config is actually loaded
The server restarts and goes into maintenance mode automatically
• Check the imported config:

Check/adjust the config to make sure it is correct and restart your server. The migration does not change any endpoints that are explicitly configured to listen on an IP address. As a result, you must check that all your endpoints that are defined to listen on specific interfaces are listening on existing network interfaces on the V2.0 server. If you set DisableObjects to true above, then you will need to enable these objects and restart your server. There is no need to set the server in production mode; it should come up in production mode on the next restart if there are no config issues.

• Point your application traffic at your new server:

Your new server is ready for traffic. Whenever you are ready, you can switch traffic from your V1.2 server to your V2.0 server.

Overview:



• Preconfigure your V1.2 server if necessary


• Configure your V1.2 server for HA if necessary


• Restart your V1.2 server if necessary


• Configure your V2.0 server for HA


• Restart your V2.0 server with clean store


• Disable HA on your V2.0 server


• Check the new config on your V2.0 server


• Restart your V2.0 server in production mode



Detailed Steps:
• Preconfiguration steps on the source V1.2 server:
• Configure the source server to replicate via TCP:
• Physical appliance: for a physical appliance run:

advanced-pd-options _setconfig imaserver
ReplicationProtocol 1'
(This is a hidden command, so it's not documented in the command help.)
This requires a imaserver restart to take affect:
imaserver stop
imaserver start
• Virtual appliance: already configured with this setting
• Configure/Verify HA on the source V1.2 server:
• If HA is already enabled on the V1.2 server(s):
• The primary node will be used for migration
• If the V2.0 server will be configured to use the same IP address as the original standby, no extra configuration required on the primary
• If the V2.0 server will not be configured to use the same IP address as the original standby: as long as the V2.0 server is configured properly (with RemoteDiscoveryNIC pointing to the LocalDiscoveryNIC of the V1.2 server) there is no need to modify the V1.2 HA configuration
• As long as you don't need to modify the HA config, you will not need to restart
• If the new V2.0 server has a different ip address than the existing V1.2 standby, the existing standby must be removed from the HA config (and this requires a restart).
• If HA is not already enabled on the V1.2 server(s):
• Plan for an outage, since the primary server must be restarted after changing the HA config
• Configure the following HighAvailability configuration items on the V1.2 server using instructions from Configuring your system for high availability
• Example:

imaserver create HighAvailability "EnableHA"= "True" "StartupMode"="StandAlone" "Group"="TestGroup1" "RemoteDiscoveryNIC"=10.78.54.140" "LocalReplicationNIC"="10.77.207.5" "LocalDiscoveryNIC"="10.78.54.137"
• Note: the key is to have the RemoteDiscoveryNIC with the same value on the V2.0 server as LocalDiscoveryNIC on the V1.2 server.
• Check the status with: 'imaserver show HighAvailability'


• Restart your source server if HA was configured or updated
• Restart the source server:

imaserver stop
imaserver start
• After restart check the status of the server: 'imaserver harole'
• Because we set StartupMode=StandAlone, the primary will come up even though its standby is not available


• Configure your V2.0 server for HA:
• Configure the HighAvailability object using the instructions for Configuring your system for high availability for V2.0.
• Note: if HighAvailability was already configured on the source server and the new server is reusing the IP addresses of the old standby, the "Group" name should be the same as it was for the original V1.2 HA setup
• Example:

curl -X POST http://10.78.54.140:9089/ima/v1/configuration  -d '{ "HighAvailability": { "EnableHA": true, "StartupMode": "AutoDetect", "LocalDiscoveryNIC":"10.78.54.140", "RemoteDiscoveryNIC":"10.78.54.137", "LocalReplicationNIC":"10.77.207.15", "Group":"TestGroup1" }}'
• To check the status of the HighAvailability config after configuring it, here's an example REST call:

curl -X GET http:// 10.78.54.140:9089/ima/v1/configuration/HighAvailability[

• Restart the V2.0 server with CleanStore flag set:
curl -X POST http://184.173.54.140:9089/ima/v1/service/restart  -d '{"Service":"Server","CleanStore":true}' [
• The server should come back up as the standby:
• Check status with:
curl -X GET http:// 10.78.54.140:9089/ima/v1/configuration/HighAvailability[
curl -X GET http:// 10.78.54.140:9089/ima/v1/service/status[
• Disable HA on the V2.0 server:
• REST call:

curl -X POST http://10.78.54.140:9089/ima/v1/configuration  -d '{ "HighAvailability": { "EnableHA": false, "StartupMode": "AutoDetect", "LocalDiscoveryNIC":"10.78.54.140", "RemoteDiscoveryNIC":"10.78.54.137", "LocalReplicationNIC":"10.77.207.15", "Group":"TestGroup1" }}'[
• The server starts in maintenance mode intentionally so that the admin can review the config (since the migration cannot migrate some config items properly)


• Important: review the new migrated config, particularly:
• Endpoint config, particularly if your endpoints were listening on specific interfaces. The migration is not able to update these explicit interfaces to their new values, if they are different.
• Existing LDAP config, if it needs to be changed
• Any other config that may need to be modified for the new environment


• Restart the V2.0 server to start in production mode:
• REST call:

curl -X POST http://10.78.54.140:9089/ima/v1/service/restart -d '{ "Service":"Server" }'[
• Check the status of the server to verify the migration:
curl -X GET http://10.78.54.140:9089/ima/v1/service/status[
• Your store and config should be migrated now
• You can point traffic at your new V2.0 server whenever you are ready.

• The HighAvailability sync from the V1.2 server to the V2.0 server is a push sync, and doesn't change anything on the V1.2 server
• The Export/Import config backup process is just a server configuration backup, and so doubles as migration step and an appliance backup.
• However, if something does go wrong, here are some resources for getting the V1.2 environment back up and running:
• Store Migration method fails:
• HA already existed on V1.2 before migration:
• Physical appliance:
• Reconfigure HA on the V1.2 server to point at the original V1.2 standby using instructions from Configuring your system for high availability
• Disable HA replication over TCP:

advanced-pd-options _setconfig imaserver
ReplicationProtocol 0'
(This is a hidden command, so it's not documented in the command help.)
• Restart the source server:

imaserver stop
imaserver start
• Verify the server and HA status:

status imaserver
status imaserver
• Virtual appliance:
• Reconfigure HA on the V1.2 server to point at the original V1.2 standby using instructions from Configuring your system for high availability
• Restart the source server:

imaserver stop
imaserver start
• Verify the server and HA status:

status imaserver
status imaserver
• HA did not exist on V1.2 before migration
• Physical appliance:
• Disable HA on the V1.2 server using instructions from Configuring your system for high availability
• You shouldn't need to disable replication over TCP (unless you plan to configure V1.2 --> V1.2 HA at a later point)
• Restart the source server:

imaserver stop
imaserver start
• Verify the server and HA status:

status imaserver
You should see output like this:
status imaserver
Status = Running (production)
ServerUpTime = 0 days 0 hours 0 minutes 48 seconds
status imaserver