Instance Stuck Deleting in SmartCloud Orchestrator

Technote (troubleshooting)


Problem(Abstract)

Virtual Instances stuck in the Deleting state from the SCO UI.

Symptom

In the "Virtual Systems Instances" section of the UI, some instances are stuck in deleting state. These were deployed to VMware, and they are gone from VMware, also gone from nova list, but still showing in SCO UI. I have restarted Workload Deployer and they did not disappear.


Cause

An active task in the task list associated with this virtual instance may be blocking the delete task.

Diagnosing the problem

First, with the SCO UI, go to Administration -> Task Queue. Look for tasks that are associated with the deletion of the specific VM. The task is probably still in the "Active" state, but is hung.

Resolving the problem

If you find any tasks listed with the virtual system name in them, delete them if possible.

After that, you can try the delete button again from the Virtual Instances panel and the removed tasks will now allow the next deletion to complete instead of just queuing up. There is often a task that is "blocking" your delete and causing it to queue up.

If that does not help, then you can use REST API to clear the task.

Note: You can install REST clients as add-ons into your browser (RESTClient is an add-on that can be used for Firefox).

Option 1:

After the RESTClient is installed in Firefox and you have restarted Firefox, log into SCO, and go to Administration -> Task Queue. Look for tasks associated with the deletion of the VM. Those tasks are probably in the Active state.

Next in the RESTClient, using the "GET" method, use the following URL:

https://<SCO IP address>/resources/tasks

The response header should be "200 OK". Next look in the Response body. It will contain a list of tasks. You need to find the same task associated with the deletion, and find its "id" attribute, for example:


     {
    "updateduserid": 2,
    "actiontime": 1380814996001,
    "priority": 0,
    "active": "N",
    "created": 1378222970818,
    "dependency": -1,
    "name": "RM51000(\"Before+virtual+system+instance+deletion\"
    ,\"Oracle-3853a5c5-24ea-4d1f-a0f2-f607e1cbce07\")",
    "currentstatus": "RM01036",
    "createduserid": 2,
    "currentmessage": "RM01036",
    "taskcountry": "US",
    "immortal": 0,
    "tasklang": "en",
    "recoveredevent": "",
    "event": "instance-process-stage-plugpoint",
    "updated": 1378224410901,
    "id": 3593,
    "counter": 0,
    "failedevent": "",
    "disk": 0,
    "processingflag": 0
    },


The id field is "3593".

Then using the RESTClient, you need to delete the task. In the method field, select DELETE, and the URL field contains:

https://<SCO IP address>/resources/tasks/<task id>

This should delete the task.

Option 2:
If you don't find any tasks listed with the virtual system name in them, force the VM instance status into 'stopped', and remove the VM instance afterwards.

To do this first select the VM instance in the "Virtual System Instances" panel by clicking on it. Make a note of the instance id (<instance_id>) at the end of the URL in the address bar.

https://<SCO.IP.address>/dashboard/runtime/virtual/?domainName=Default&projectName=<project_name>#<instance_id>

Next, in the RESTClient, using the "PUT" method, use the following URL and request body:

REST PUT https://<SCO IP address>/resources/virtualSystems/<instance_id>
Body { "desiredstatus" : "RM01011" }

Send the PUT request, expected status return code is 200.

Status Code: 200 OK

{
"desiredstatus_text": "Stopped",
"currentstatus_text": "Stopping",
"name": "<virtual_system_instance>",
"desiredstatus": "RM01011",
"pattern": "/resources/patterns/<pattern_id>",
"id": <instance_id>,
"currentmessage_text": "Stopping virtual machines",
"created": 1392024033796,
"currentstatus": "RM01010",
"currentmessage": "RM07028",
"owner": "/resources/users/<user_id>",
"updated": 1396620773989
}

Check the status of the VM afterwards in the UI by clicking the "Refresh" button. The status of the VM instance should show "Stopped". It should now be possible to remove the VM instance from the UI by clicking on the "Delete" button in the task bar of the "Virtual System Instances" panel.

Rate this page:

(0 users)Average rating

Document information


More support for:

IBM Cloud Orchestrator

Software version:

2.2, 2.2.0.1, 2.3

Operating system(s):

Linux Red Hat - xSeries

Reference #:

1648846

Modified date:

2014-04-04

Translate my page

Machine Translation

Content navigation