Checking status of OpenStack services

After an IBM Cloud Manager with OpenStack topology is deployed, you can check the status of the IBM Cloud Manager with OpenStack services. You can use the OpenStack Dashboard or issue an IBM Cloud Manager with OpenStack command to check more detailed status.

About this task

To check the services status with OpenStack Dashboard, log in and select Admin > System Panel. For an HA topology, IBM Cloud Manager with OpenStack controller services with an active/active configuration should be up on all HA controller nodes. Controller services with an active/passive configuration should be up only on one HA controller node. If a node is in standby mode or is offline, the services should be down on the node.
If you want a more detailed status check of the IBM Cloud Manager with OpenStack services, issue the following command (from the deployment server) for each node in your topology deployment.
$ knife os manage services status --node your-node-FQDN

The status of all IBM Cloud Manager with OpenStack services on the node is displayed.

Alternatively, issue the following command (from the deployment server) to get detailed status for each node in a known topology by using a topology JSON file:
$ knife os manage services status --topology-file your-topology-name.json
The status of all IBM Cloud Manager with OpenStack services on all of the nodes is displayed.

In an HA topology, the knife os manage services status command provides the status of the Pacemaker cluster and the resources that are managed by Pacemaker on each node. In addition, it includes status information about the RabbitMQ cluster and DB2 HADR.

For RabbitMQ, the output from the rabbitmqctl cluster_status command is shown. It includes the nodes in the cluster, the running nodes as seen by RabbitMQ clustering, and network partitions. This information can be helpful when OpenStack logs indicate that there are messaging problems. For more information, see Loss of HA controller might cause RabbitMQ partitioning and messaging failures.

The following example shows RabbitMQ status output where all nodes are running and there are no partitions:
RabbitMQ Cluster Status:

Cluster status of node 'rabbit@10.5.48.181' ...
[{nodes,[{disc,['rabbit@10.5.48.181','rabbit@10.5.48.182',
                'rabbit@10.5.48.184']}]},
 {running_nodes,['rabbit@10.5.48.184','rabbit@10.5.48.181',
                 'rabbit@10.5.48.182']},
 {cluster_name,<<"RegionOne">>},
 {partitions,[]}]
For DB2 HADR, selected monitor output from the db2pd -hadr command is shown. It shows the HADR role and HADR state for each standby server. On the HADR primary, the output includes the status of all the HADR standby servers. On an HADR standby server, it includes only the status of that server. The output also includes a summary that indicates whether HADR appears to be healthy. Under normal conditions, the following conditions should be true:
  • HADR is active for all databases on the server.
  • All databases have the same role (all are PRIMARY or all are STANDBY).
  • The HADR state for each standby is PEER (for the first standby - also known as the principal standby) or REMOTE_CATCHUP. A state of DISCONNECTED indicates that the remote HADR service is not running (either the node or DB2 is down, or HADR detected an error that indicates HADR split brain or dual primary occurred).
For a healthy system with three HA controllers, the output on a HADR primary looks similar to the following output for each database:
*** Database: ceilodb2 ***

Database Member 0 -- Database CEILODB2 -- Active -- Up 0 days 15:42:53 -- Date 2015-07-08-08.26.53.560063
                            HADR_ROLE = PRIMARY
                           HADR_STATE = PEER
                  STANDBY_MEMBER_HOST = 10.5.48.182
                            HADR_ROLE = PRIMARY
                           HADR_STATE = REMOTE_CATCHUP
                  STANDBY_MEMBER_HOST = 10.5.48.184

[additional output not shown]

*** Summary: ***

All databases are PRIMARY
DB2 HADR looks OKAY.
On an HADR standby, the output would look similar to the following output:
*** Database: ceilodb2 ***

Database Member 0 -- Database CEILODB2 -- Standby -- Up 0 days 00:03:27 -- Date 2015-07-08-10.57.53.416066
                            HADR_ROLE = STANDBY
                           HADR_STATE = PEER
                  STANDBY_MEMBER_HOST = 10.5.48.182

[additional output not shown]

*** Summary: ***

All databases are PRINCIPAL STANDBY
DB2 HADR looks OKAY.