Teaching Forklift to offload VM disk copies to the array

Why bother

Migrating a VM from vSphere to OpenShift Virtualization normally means reading every block out of the source datastore, pushing it across the network, and writing it into a PVC. For a handful of VMs that is fine. For thousands it is the project schedule.

When source and target volumes live on the same Fibre Channel array, none of that traffic needs to exist. VAAI XCOPY tells the array to clone the extents internally. Forklift ships a populator backend that knows how to do this for Primera/3PAR arrays. In our environment it did not work.

The four fixes

1. A JSON struct tag that never matched. The backend decoded the array’s volume response into a struct whose WWN field carried the wrong tag, so the WWN came back empty and every lookup downstream silently failed. One line, and the least interesting bug of the four — but the one that made everything above it look broken.

2. Server-side path filtering the array does not implement. The backend asked the array’s REST API for FC paths using a bracketed sub-filter. The API accepts the query and ignores the filter. The fix was to stop trusting it: fetch the paths and match client-side. Worth generalising — a REST API that returns 200 for a filter it does not support is worse than one that returns 400.

3. Host creation assumed iSCSI-shaped input. Host registration was built around an identity the array does not have in a pure FC environment. Creating hosts from WWPNs alone made registration behave.

4. Initiator group names longer than the array accepts. Generated igroup names exceeded the array’s name limit and were rejected at create time. Clamping to 27 characters resolved it. Names generated from Kubernetes object names hit this class of limit constantly — clamp early.

The prerequisite nobody documents

The ESXi-side helper VIB has to be installed on the hosts beforehand. It is not deployed automatically as part of the migration tooling, and without it the offload path fails in a way that looks like a storage problem rather than a missing dependency.

One environmental note in the same category: the hook pod’s SSH path to the hypervisors was blocked by the firewall between those zones. Pre-staging from a host inside the permitted zone was the working path. If your migration tooling assumes flat connectivity to hypervisors, check that assumption before you start debugging the tooling.

Result

First successful canary migration with offload confirmed active: clone in about 11 seconds, end-to-end migration about 55 seconds. The same VM copied conventionally is bounded by network and datastore read throughput and is not in the same order of magnitude.

The broader lesson keeps being true: when a vendor integration “supports” your array, that means someone tested it against one configuration. Read the backend before you plan a schedule around it.