IBM Support

Common error message returned by VMware vStorage API for Data Protection (VDAP) and reported in the Tivoli Storage Manager for Virtual Environments: Data Protection for VMware data mover

Technote (troubleshooting)


This document applies only to the following language version(s):

English

Problem(Abstract)

The following are common error messages returned by VMware vStorage API for Data Protection (VDAP). The Tivoli Storage Manager for Virtual Environments: Data Protection for VMware data mover uses these APIs to backup and restore virtual machines in a vCenter Server environment. The VDAP is made up of two major components. The vSphere Management SDK and the Virtual Disk Development Kit (VDDK).

For more information about VDAP, see the VMware FAQ at http://kb.vmware.com/kb/1021175

Symptom

When you experience a virtual machine backup or restore problem with Data Protection for VMware data mover, a detailed error message from the vStorage API will be reported by the data mover. These vStorage API error messages can be found on the data mover, which is the machine running the Tivoli Storage Manager Backup/Archive Client and the messages are in the dsmerror.log file. Often the error message will be reported to the console if the client is running in the command-line mode.

In these messages, note the name of the Tivoli Storage Manager Backup/Archive Client function and the error message text. You will need these values when you read the "Resolving the problem" sections of this document.

Example:

01/22/2014 13:10:20 ANS9365E VMware vStorage API error for virtual machine 'win2012 - maximovm12'.
TSM function name : VixDiskLib_Open
TSM file : vmvddksdk.cpp (1735)
API return code : 13
API error message : You do not have access rights to this file


Cause

There are many causes and it depends on the error message and it can also depend on the transport used to backup or restore the virtual machine.


Environment

Tivoli Storage Manager data mover for virtual environments

Diagnosing the problem

The error message will be output to the Tivoli Storage Manager data mover error log and the console.

Resolving the problem

VMware also has a “Best Practices” document that describes the use of the advanced transport mechanisms (SAN, HotAdd and NBD/NBDSSL). A link to that best practices document is supplied in each message explanation where it might be helpful to review the VMware best practices.


Message: VixDiskLib_Open => "You do not have access rights to this file"
Tivoli Storage Manager operations: Backup VM, Restore VM

Transport: ALL
Explanation and actions:
The data mover cannot resolve the fully qualified domain name (FQDN) of the ESXi host. Ensure that the data mover is configured to use the DNS or other services. Ping the ESXi, by name, from the data move and confirm it resolves the IP address.
1) Verify that the vCenter is licensed to use the "Storage APIs" feature.
2) To backup a target virtual machine, you must specify which VMware userid to use with the Tivoli Storage Manager data mover. This is done with the 'dsmc set password -type=VM' command. For more information, see https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.0/client/r_cmd_setpassword.html.

For advanced transport best practices, see https://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.vddk.pg.doc%2FvddkBkupVadp.9.5.html.

Transport: SAN
Explanation and actions:
1) The data mover must be installed on a physical system.
2) The Fibre Channel SAN is not accessible by the data mover node. Ensure that the LUN is mapped to the data mover.
3) The iSCSI SAN target is not connected to the data mover node. Ensure that the iSCSI target is connected to the data mover.
4) The SAN Policy must be set to OnLineAll using the Windows program called diskpart. For more information, see http://kb.vmware.com/kb/2000767.

For advanced transport best practices, see https://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.vddk.pg.doc%2FvddkBkupVadp.9.5.html.

Transport: SAN
Explanation and actions:
The guest to backup has an UUID that is not unique in the VMware environment.
The error will be seen only when using SAN transport and with client 7.1.2.0 to 7.1.3.1 because the guest UUID uniqueness check is enforced by VMware starting with VDDK 6.0.0 :

SAN mode VDDK 6.0 searches for virtual machines by BIOS UUID.
When using SAN transport, VDDK 6.0.0 tries to find requested virtual machines by looking up their BIOS UUID, instead of by the MoRef provided, as in previous VDDK releases. Because a BIOS UUID is not guaranteed unique, unlike a MoRef, this new behavior may cause problems after cloning, out of place VM restore, and so forth. The problem happens with both SAN backup and restore when two VMs managed by the same vCenter Server have identical BIOS UUID, whether the VMs are powered on or off. The error occurs whenVixDiskLib_Open(Ex) fetches the virtual disk's block map. A fix has been identified and will be available in a future release.

This is corrected with VDDK 6.0.1 which is included in Tivoli Storage Manager Client 7.1.3.2 and higher.

Transport: HOTADD
Explanation and actions:
The data mover must be installed on a virtual system in the same vCenter as the target virtual machine.

  • The data mover does not have access to the target virtual machines datastore. Endure the data mover has access to the target virtual machines datastore.
  • The SAN Policy must be set to OnLineAll using the Windows program called diskpart. For more information, see http://kb.vmware.com/kb/2000767.

For advanced transport best practices, see https://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.vddk.pg.doc%2FvddkBkupVadp.9.5.html.


Transport: NBD, NDBSSL
Explanation/Actions:
A network path is not available from the data mover to the ESXi system that is hosting the target virtual machine.
  • ESXi host is not responding on port TCP 902, which is the port used by the NBD transport. Use Telnet to connect to the ESXi server to see if it accepts the connection (telnet <esxihost> 902). For more information, see http://kb.vmware.com/kb/1003487.

For advanced transport best practices, see https://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.vddk.pg.doc%2FvddkBkupVadp.9.5.html.


Message: VixDiskLib_Open => "Cannot use hotadd transport to mount virtual machines with non-SCSI disks"
Tivoli Storage Manager operations: Backup VM, Restore VM

Transport: SAN, NBD, NDBSSL
Explanation/Actions:
  • It is unknown if this message occurs when you use SAN, NBD, or NBDSSL.

Transport: HOTADD
Explanation/Actions:
Message: VixDiskLib_ConnectEx => "One of the parameters supplied is invalid"
Tivoli Storage Manager operations: Backup VM

Transport: SAN, NBD, NDBSSL
Explanation/Actions:
  • It is unknown if this message occurs when you use SAN, NBD, or NBDSSL.

Transport: HOTADD
Explanation/Actions:
  • HotAdd is the selected transport but the data mover is a physical system. HotAdd can only be used on a virtual machine.

Message: VixDiskLib_ConnectEx => "No transport modes available to access disks"
Tivoli Storage Manager operations: Backup VM, Restore VM also depends on Virtual Disk Development Kit (VDDK) version

Transport: SAN, NBD, NDBSSL
Explanation/Actions:
  • It is unknown if this message occurs when you use SAN, NBD, or NBDSSL.

Transport: HOTADD
Explanation/Actions:
  • HOTADD is the selected transport but the data mover is a physical system. HotAdd can only be used on a virtual machine.

Message: VixDiskLib_Write => "One of the parameters supplied is invalid"
Tivoli Storage Manager operations: Restore VM

Transport: SAN

Explanation/Actions:
  • LUN or iSCSI target is marked "Read-only: Yes". Clear the attribute using "diskpart" (attribute disk clear readonly). For more information, see http://kb.vmware.com/kb/2000767.

Transport: HOTADD, NBD, NDBSSL
Explanation/Actions:
  • It is unknown if this message occurs when you use HOTADD, NBD, or NBDSSL.

Message: VixDiskLib_Open => "The host is not licensed for this feature"
Tivoli Storage Manager operations: Backup VM, Restore VM

Transport: SAN, HOTADD, NBD, NDBSSL
Explanation/Actions:
Message: VixDiskLib_Open => "Cannot hot-add the proxy's own disks to itself "
Tivoli Storage Manager operations: Backup VM

Transport: SAN, NBD, NDBSSL
Explanation/Actions:
  • It is unknown if this message occurs when you use SAN, NBD, or NBDSSL.

Transport: HOTADD
Explanation/Actions:
Message: Snapshot Clean-up => "Device or resource busy" or "The file is already in use"
Tivoli Storage Manager operations: Backup VM

Transport: SAN, NBD, NDBSSL
Explanation/Actions: See VMware's Virtual Disk Development Kit (VDDK) 5.1.1 Release Notes https://www.vmware.com/support/developer/vddk/vddk-511-releasenotes.html.
Transport: HOTADD
Explanation/Actions:
  • It is unknown if this message occurs when you use HOTADD.


Message: VixDiskLib_WriteMetadata => "The operation is not supported for the specified parameters"
Tivoli Storage Manager operations: Restore VM

Transport: SAN, HOTADD
Explanation/Actions:
Transport: NBD/NDBSSL
Explanation/Actions:
  • It is unknown if this message occurs when you use NBD or NBDSSL.

Message: VixMntApi_GetVolumeInfo => The problem here is that no data is returned and we get no error message from the VDDK SDK.
Tivoli Storage Manager operations: Backup VM, Restore VM

Transport: SAN, HOTADD, NBD, NDBSSL
Explanation/Actions:
Message: "Volume/NTFS filesystem is corrupt (87)"
Tivoli Storage Manager operations: Backup VM

Transport: SAN, NBD, NDBSSL
Explanation/Actions:
  • It is unknown if this message occurs when you use SAN, NBD, or NBDSSL.

Transport: HOTADD
Explanation/Actions:
Message: VixDiskLib_Write => "Unknown error" and the return code is -1 also a trace will show "The media is write protected"
Tivoli Storage Manager operations: Restore VM

Transport: SAN, HOTADD
Explanation/Actions
  • The SAN Policy must be set to OnLineAll using the Windows program "diskpart". For more information, see VMware KB http://kb.vmware.com/kb/2000767.
  • The LUN could be marked as offline or read only use diskpart tool as recommended by the VMware advance transport best practices document.

For advanced transport best practices, see https://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.vddk.pg.doc%2FvddkBkupVadp.9.5.html.


Transport: NBD, NDBSSL
Explanation/Actions:
  • It is unknown if this message occurs when you use NBD or NBDSSL.


Message: VixMntApi VixDiskLib_Read => "Unknown error" and the return code is 1 also a trace will show "The media is write protected"
Tivoli Storage Manager operations: Backup VM

Transport: SAN, HOTADD
Explanation/Actions:
For advanced transport best practices, see https://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.vddk.pg.doc%2FvddkBkupVadp.9.5.html.

Transport: NBD, NDBSSL
Explanation/Actions:
  • It is unknown if this message occurs when you use NBD or NBDSSL.

Message: visdkCreateVmSnapshotMoRef => “Cannot create a quiesced snapshot because the create snapshot operation exceeded the time limit for holding off I/O in the frozen virtual machine."
Tivoli Storage Manager operations: Backup VM

It is important to understand that the Tivoli Storage Manager data mover only makes a request for the vCenter to take a snapshot via the VMware Infrastructure (VI). VMware provides the following KB that discusses the snapshot process. See http://kb.vmware.com/kb/1015180.

Transport: ALL
Explanation/Actions:
Transport: SAN
Explanation/Actions:
Message: visdkCreateVmSnapshotMoRef => “An error occurred while quiescing the virtual machine. See the virtual machine's event log for details."
Tivoli Storage Manager operations: Backup VM

Transport: SAN, HOTADD, NBD, NDBSSL
Explanation/Actions:
  • The Volume Shadow Copy Service (VSS) was unable to flush application data to disk in the virtual machine because the Input/Output was too high. The I/O must be reduced. You might need to remove VMware Tools from the virtual machine and then reinstall them; or, quiescing must be disabled. This is a complex problem; for a detailed explanation, see http://kb.vmware.com/kb/1018194.

Message: visdkCreateVmSnapshotMoRef => “File is larger than the maximum size supported by datastore."
Tivoli Storage Manager operations: Backup VM

Transport: SAN, HOTADD, NBD, NDBSSL
Explanation/Actions:
Message: VixDiskLib_Read => "One of the parameters supplied is invalid"
Tivoli Storage Manager operations: Backup VM

Transport: SAN
Explanation/Actions:
  • This error can occurs in the Fibre Channel SAN transport when there is no available paths to the LUN, this could be a temporary SAN I/O failure.
  • The disk lease time might have expired. To increase it, see http://kb.vmware.com/kb/1006711.
  • The vStorage for Array Integration or VAAI is not enabled or the storage hardware does not fully support VAAI. For information, see http://kb.vmware.com/kb/1021976.
  • Use NBD/NBDSSL until the Fibre Channel SAN issue can be corrected.
Transport: HOTADD, NBD, NDBSSL
Explanation/Actions:
  • It is unknown if this message occurs when you use HOTADD, NBD, or NBDSSL.

Message: visdkWaitForTask => “A specified parameter was not correct.”
Tivoli Storage Manager operations: Restore VM

Transport: SAN, HOTADD, NBD, NDBSSL
Explanation/Actions:
  • In some cases restoring a virtual machine directly to an ESXi host can cause problems if the virtual machine has attributes that are vCenter-specific. The restore process attempts to remove these attributes, but it is a complex process that is not always successful. A VMware KB article might help to determine which attribute is causing the problem (see http://kb.vmware.com/kb/2012122). The vmbackup_save_local and vmrestore_ovf_override test flags can be used to work-around this issue. For information, see http://www.ibm.com/support/docview.wss?uid=swg21636933.

Message: visdkWaitForTask => “A general system error occurred: Protocol error from VMX.”
Tivoli Storage Manager operations: Backup VM

Transport: SAN, HOTADD, NBD, NDBSSL
Explanation/Actions:
Message: visdkWaitForTask => “Virtual machine is configured to use a device that prevents the snapshot operation: Device '' is a SCSI controller engaged in bus-sharing”
Tivoli Storage Manager operations: Backup VM

Transport: SAN, HOTADD, NBD, NDBSSL
Explanation/Actions:
Message: visdkCreateVM => “The operation is not supported on the object.”
Tivoli Storage Manager operations: Restore VM

Transport: SAN, HOTADD, NBD, NDBSSL
Explanation/Actions:
Message: VixDiskLib_Read => “The server refused connection”
Tivoli Storage Manager operations: Backup VM

Transport: NBD, NDBSSL
Explanation/Actions:
  • This message occurs intermittently when you are using older (before Tivoli Storage Manager 7.1) Tivoli Storage Manager clients when the virtual machine has large vmdk files and there is a firewall timeout setting for the communications between the ESXi hosts and the data mover host. The problem occurs because at the start of the backup session, connections are opened for all VMDKs. If one of the VMDK files is large or simply takes too long to be processed, the firewall can cut the idle connections of the remaining VMDKs in the sequential processing list. When the next VMDK is to be processed, the connection is refused and the backup fails. Upgrading to Tivoli Storage Manager 7.1, or newer, or using the SAN or HOTADD transport should fix this problem.

Message: VixDiskLib_Read => “SOAP 1.1 fault: SOAP-ENV:Client[no subcode] "No Data" Detail: get host by name failed in tcp_connect()”
Tivoli Storage Manager operations: Backup VM, Restore VM

Transport: SAN, HOTADD, NBD, NDBSSL
Explanation/Actions:
  • This happens intermittently when the communication between the data mover and the vCenter fails, either because the vCenter is too busy to respond in time, or there is a firewall or “intelligent” network appliance that blocks the traffic.

Historical Number

v1.0

Document information

More support for: IBM Spectrum Protect for Virtual Environments
Data Protection for VMware

Software version: All Versions

Operating system(s): Linux, Windows

Software edition: All Editions

Reference #: 1662673

Modified date: 05 November 2015