IBM Support

VMware snapshot consolidation failure due to virtual machine file lock

Troubleshooting


Problem

After a successful backup or restore operation the consolidation of virtual machine snapshots by VMware may fail due to a virtual machine file lock leading to various symptoms.

Symptom

When a backup or restore operation using the Tivoli Storage Manager Backup-Archive Client and the Tivoli Storage Manager for Virtual Environments - Data Protection for VMware completed successful, snapshot consolidation failure may be seen in the VMware vCenter. The following warning may be seen in the Tivoli Storage Manager client error log and output :
 ANS2718W The virtual machine 'virtualmachinename' requires snapshot consolidation.                                

Other symptoms could be:
-- You are unable to commit the snapshots even if the virtual machine is off.
-- The CID chain is not broken and there are no files missing .
-- If you power on the virtual machine, you receive the message indicating that the .vmdk file is busy.
-- The virtual machine does not power on.
-- Cloning the virtual disks of the virtual machine fails.
See also the link below to the VMware KB article 1010310 "Unable to commit virtual machine snapshots because another task is in progress".

Cause

The proxy host on which the Tivoli Storage Manager client is running will hold locks on vmdk files while the backup or restore is running then release it when closing the virtual machine file.
If for any reason, the client is abnormally interrupted while holding a lock, then the lock will not be automatically removed and will prevent further operations with the virtual machine file.

Environment

VMware vSphere/ESX/ESXi 4.0, 4.1, 5.0, 5.1, 5.5, 6.0

Diagnosing The Problem

In the hostd logs you may find messages like the following :

2015-06-29T13:12:37.741Z [2431CB90 info 'Libs' opID=3e4eb083-61] SNAPSHOT: SnapshotCombineDisks: Failed to open disk
'/vmfs/volumes/544249e5-1ba21a26-e60d-5cf3fc947080/423a901d-906b-61b2-839c-cd817c440971/423a901d-906b-61b2-839c-cd817c440971_1001_0.vmdk': Failed to lock the file (16392).
2015-06-29T13:12:37.741Z [2431CB90 info 'Libs' opID=3e4eb083-61] SNAPSHOT: SnapshotCombineDisks failed: One or more disks are busy (20)
2015-06-29T13:12:37.741Z [2431CB90 info 'Libs' opID=3e4eb083-61] SNAPSHOT: SnapshotConsolidate failed: One or more disks are busy (20)
2015-06-29T13:12:37.741Z [2431CB90 info 'Libs' opID=3e4eb083-61] SNAPSHOT: Snapshot_Consolidate failed: One or more disks are busy (20)

See also the link below to the VMware technote 10051 "Investigating virtual machine file locks on ESXi/ESX".

Resolving The Problem

Manually remove the lock, for example by rebooting the proxy host.

[{"Product":{"code":"SSERB6","label":"IBM Spectrum Protect for Virtual Environments"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Component":"Data Protection for VMware","Platform":[{"code":"PF016","label":"Linux"},{"code":"PF033","label":"Windows"}],"Version":"6.3;6.4;7.1;7.1.1;7.1.2;7.1.3","Edition":"","Line of Business":{"code":"LOB26","label":"Storage"}}]

Document Information

Modified date:
17 June 2018

UID

swg21967938