Repository navigation
Snapshots copy from Linstor to NFS fail #14011
Description
ISSUE TYPE
- Bug Report
COMPONENT NAME
Storage - Linstor snapshot
CLOUDSTACK VERSION
4.22.1.0
CONFIGURATION
N/A
OS / ENVIRONMENT
- Cloudstack 4.22.1.0
- Hypervisor KVM
- AlmaLinux 9.8
- Primary storage: Linstor (LVM-thin)
- Secondary storage: NFS
SUMMARY
createSnapshot on any volume backed by a Linstor primary storage pool fails 100% of the time with: Failed to create Snapshot due to an internal error creating Snapshot for volume <volume-uuid>.
The failure is deterministic and reproducible on every attempt, regardless of volume, VM, or snapshot name.
First the snapshot is created succesfully on the Linstor backend, but then the copy to NFS (secundairy storage) fails as CloudStack's KVM agent invokes qemu-img convert against a /dev/mapper/... device path that does not match the path the kernel actually created.
Actual device location: /dev/mapper/linstor_pool-lv--thin-cs--12fc4055--3985--4025--8eb5--d6fd53effe37_00000
Path CloudStack looks for: /dev/mapper/linstor_pool-lvm-thin-cs--12fc4055--3985--4025--8eb5--d6fd53effe37_00000_cs--d7aea646--5f40--46a2--b9dc--77e41ea29336
Notice the missing dashes in the linstor_pool-lv--thin-cs-name.
STEPS TO REPRODUCE
Create disk snapshot on a Linstor volume with NFS as secundairy storage and observe the error message.
EXPECTED RESULTS
Snapshot is created on the Linstor backend and successfully copied to secondary (NFS) storage.
ACTUAL RESULTS
The error logged in management-server.log:
ERROR [o.a.c.a.c.u.s.CreateSnapshotCmd] Failed to create Snapshot due to an internal error creating Snapshot for volume 12fc4055-3985-4025-8eb5-d6fd53effe37 java.lang.RuntimeException: Unexpected exception
at com.cloud.storage.VolumeApiServiceImpl.takeSnapshotInternal(VolumeApiServiceImpl.java:3999)
at com.cloud.storage.VolumeApiServiceImpl.takeSnapshot(VolumeApiServiceImpl.java:3910)
...
Caused by: com.cloud.utils.exception.CloudRuntimeException: Exception: org.apache.cloudstack.utils.qemu.QemuImgException
Message: qemu-img: Could not open '/dev/mapper/linstor_pool-lvm-thin-cs--12fc4055--3985--4025--8eb5--d6fd53effe37_00000_cs--d7aea646--5f40--46a2--b9dc--77e41ea29336': Could not open '/dev/mapper/linstor_pool-lvm-thin-cs--12fc4055--3985--4025--8eb5--d6fd53effe37_00000_cs--d7aea646--5f40--46a2--b9dc--77e41ea29336': No such file or directory
Stack: org.apache.cloudstack.utils.qemu.QemuImgException: ...
at org.apache.cloudstack.utils.qemu.QemuImg.convert(QemuImg.java:495)
...
at com.cloud.hypervisor.kvm.resource.wrapper.LinstorBackupSnapshotCommandWrapper.convertImageToQCow2(LinstorBackupSnapshotCommandWrapper.java:99)
at com.cloud.hypervisor.kvm.resource.wrapper.LinstorBackupSnapshotCommandWrapper.execute(LinstorBackupSnapshotCommandWrapper.java:156)
...
at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:475)
at org.apache.cloudstack.storage.snapshot.DefaultSnapshotStrategy.backupSnapshot(DefaultSnapshotStrategy.java:204)
at com.cloud.storage.snapshot.SnapshotManagerImpl.backupSnapshotToSecondary(SnapshotManagerImpl.java:1787)
at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1660)
...
Suggested fix: apply LVM's standard dash-doubling escape to the volume-group name the same way it's already correctly applied to the cs--<uuid> logical-volume/snapshot name portion.
Activity
boring-cyborg commentedon Aug 31, 2026
Thanks for opening your first issue here! Be sure to follow the issue template!
[-][BUG][4.22.1.0] Snapshots copy from Linstor to NFS fail[/-][+]Snapshots copy from Linstor to NFS fail[/+]DaanHoogland commentedon Sep 2, 2026
@rp- could you have a look?
github-actions commentedon Sep 2, 2026
🎯 Triage report
Deterministic failure when backing up a snapshot from Linstor primary storage to NFS secondary storage on KVM. The KVM agent constructs a
/dev/mapper/...device-mapper path for the LVM-thin volume that does not match the actual path the kernel creates, because the volume-group name's dashes are not escaped using LVM's standard dash-doubling convention (only thecs--<uuid>portion is escaped). The reporter included full stack traces and a concrete suggested fix.📊 Assessment
LinstorBackupSnapshotCommandWrapper(KVM agent) duringqemu-img convert🔗 Similar issues
No similar open issues found; this appears to be a distinct, Linstor-specific path-escaping bug.
💡 Notes and suggestions
vg-name→vg--name). The current code appears to correctly escape thecs--<uuid>LV/snapshot suffix but not the VG name segment (linstor_pool-lvm-thin) which should becomelinstor_pool-lvm--thin.LinstorBackupSnapshotCommandWrapper.convertImageToQCow2(and any shared path-building helper used to construct the/dev/mapper/...path for Linstor-backed volumes) — apply the same dash-doubling escape uniformly to the whole device-mapper name, not just the LV suffix./dev/mapper/...path matchesdmsetup/kernel-created naming.Add this agentic workflows to your repo
To install this agentic workflow, run
rp- commentedon Sep 7, 2026
Yes, valid report and I opened a PR to fix this.
A workaround for now is to use VG names that don't use any
-linstor: escape dashes in the LVM volume group name of snapshot paths (…