<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Live-Migration on rHomelab</title><link>https://rhomelab.com/tags/live-migration/</link><description>Recent content in Live-Migration on rHomelab</description><generator>Hugo</generator><language>en-US</language><lastBuildDate>Tue, 13 Oct 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://rhomelab.com/tags/live-migration/index.xml" rel="self" type="application/rss+xml"/><item><title>Proxmox Live Migration: How It Actually Works, and Why It Sometimes Won't</title><link>https://rhomelab.com/proxmox/proxmox-live-migration-mechanics/</link><pubDate>Tue, 13 Oct 2026 00:00:00 +0000</pubDate><guid>https://rhomelab.com/proxmox/proxmox-live-migration-mechanics/</guid><description>What actually happens during a Proxmox live migration, why LXC containers migrate differently than VMs, the bandwidth and storage prerequisites people skip, and the specific failure modes that make a migration refuse to go live.</description><content:encoded><![CDATA[<p>Clustering gets you live migration, moving a running guest from one node to another with no downtime, but &ldquo;it just works&rdquo; is doing a lot of hiding in that sentence. Live migration has real mechanics underneath it, real prerequisites that aren&rsquo;t optional, and a short list of device types and configurations that simply cannot migrate live no matter how the storage is set up. Knowing which category a given VM or container falls into before you click migrate saves you from finding out mid-migration, which is the worse time to learn it.</p>
<h2 id="what-a-live-migration-is-actually-moving">What a live migration is actually moving</h2>
<p>For a VM, Proxmox uses QEMU&rsquo;s built-in live migration, and the mechanics are the same ones VMware or any other KVM-based hypervisor uses: the source node starts streaming the VM&rsquo;s RAM contents to the target node while the VM keeps running and keeps writing to that same memory. Pages that get modified after they&rsquo;ve already been copied, &ldquo;dirty&rdquo; pages, get marked and re-sent in another pass. This is pre-copy migration: copy, note what changed, copy the changes, repeat, and each pass is smaller than the last as the set of actively-dirty pages narrows. Once the remaining dirty set is small enough to transfer in a few hundred milliseconds, Proxmox briefly pauses the VM, sends that last sliver of state plus the CPU register and device state, and resumes it on the target node. The pause is real but usually sub-second, which is why it reads as &ldquo;no downtime&rdquo; even though a strict reading says there was a very short one.</p>
<p>This has a direct consequence worth planning around: a VM that&rsquo;s actively dirtying a lot of memory fast, a busy database, a VM mid-compile, anything hammering RAM, takes longer to migrate and holds a bigger final pause than an idle VM, because the dirty set never shrinks as quickly. Migrating a loaded VM isn&rsquo;t dangerous, but it is slower and holds the final freeze longer than migrating the same VM at idle, and if you&rsquo;re doing planned maintenance, timing the migration for a quiet moment is worth the thirty seconds of patience.</p>
<h2 id="lxc-containers-migrate-differently-and-it-matters-which-way-you-ask-for-it">LXC containers migrate differently, and it matters which way you ask for it</h2>
<p>Containers don&rsquo;t have QEMU&rsquo;s live-migration machinery underneath them, because there&rsquo;s no guest kernel memory being tracked the same way, the container shares the host kernel. Proxmox offers two paths for a running container: a restart-based migration, which checkpoints the container, stops it, moves it, and starts it again on the target (a real, if usually brief, outage), and a true live migration path using CRIU (Checkpoint/Restore In Userspace) that can move a running container&rsquo;s process state without a full stop/start cycle, when the container&rsquo;s workload is compatible with CRIU&rsquo;s checkpoint model. CRIU-based container migration is newer, pickier about what&rsquo;s inside the container, and worth testing against your actual workload before relying on it for anything time-sensitive. If a container holds open resources CRIU doesn&rsquo;t know how to checkpoint cleanly, a database with an exotic locking scheme, something bound tightly to a specific kernel module state, the safer bet is to treat container migration as &ldquo;brief restart&rdquo; rather than assume VM-grade zero-downtime behavior by default.</p>
<h2 id="the-prerequisite-nobody-skips-on-purpose-until-they-do">The prerequisite nobody skips on purpose, until they do</h2>
<p>Live migration for a VM needs its disk reachable from both the source and target node at the same time, true shared storage like Ceph or an NFS/iSCSI-backed store, or synchronous enough replication that both sides agree on the data. If a VM&rsquo;s disk lives only on the source node&rsquo;s local storage, local-lvm, a local ZFS pool with no replication configured, Proxmox can&rsquo;t live migrate it, because there&rsquo;s nothing on the target node to resume into. The web UI will often offer &ldquo;migrate&rdquo; anyway, but what it actually does in that case is an offline migration: shut the VM down, copy the full disk image across the network, then start it on the target. That&rsquo;s a real outage for however long the disk copy takes, which scales directly with disk size and network speed, not the quick pre-copy dance described above. People who set up a cluster expecting every migration to be the zero-downtime kind are usually caught off guard here, because the UI doesn&rsquo;t make a hard distinction between &ldquo;live&rdquo; and &ldquo;offline&rdquo; migration up front, it just tries to do the best available option and tells you what it ended up doing after the fact.</p>
<p>ZFS storage replication (covered in more depth in the clustering article) gets you most of the way there without running full shared storage: the disk is replicated to the target node on a schedule, as often as every minute, and a live migration triggers one final incremental sync of whatever changed since the last scheduled replication before cutting over. That last sync is usually small and fast if the replication interval is tight, but it&rsquo;s still a real step in the process, not instantaneous, and a VM generating a lot of write traffic between replication runs makes that final sync bigger.</p>
<h2 id="migration-network-dont-let-it-share-the-vms-own-traffic-by-accident">Migration network: don&rsquo;t let it share the VM&rsquo;s own traffic by accident</h2>
<p>By default, Proxmox routes migration traffic over whichever network interface is configured for cluster communication, which on a lot of home setups is the same single NIC everything else uses. A large VM&rsquo;s memory can be tens of gigabytes, and pushing that across a 1GbE link that&rsquo;s also carrying the VM&rsquo;s own guest traffic and the node&rsquo;s management traffic competes with itself in a way that slows the migration and can momentarily pinch bandwidth for everything else sharing that link. Proxmox lets you set a dedicated migration network in <code>/etc/pve/datacenter.cfg</code> (the <code>migration</code> setting, pointing at a specific CIDR), so migration traffic rides a separate NIC or VLAN instead of fighting production traffic for the same pipe. For anyone who&rsquo;s already split out a dedicated cluster/corosync network, as recommended elsewhere, putting migration traffic on that same isolated link, or a third dedicated one if the hardware supports it, is worth doing before the first time a large migration visibly stalls everything else on the switch.</p>
<p>Migration traffic is encrypted over SSH by default, which costs real CPU on both ends for large transfers. Proxmox exposes an option to run migration traffic unencrypted (<code>migration: type=insecure</code> in the datacenter config, or per-migration via the API), trading that CPU cost for speed on a network you trust completely. This is a reasonable trade on an isolated Infra VLAN that never sees anything outside the homelab, but it&rsquo;s a deliberate security tradeoff, not a free performance win, and it shouldn&rsquo;t be the default on anything that could plausibly see untrusted traffic.</p>
<h2 id="the-hard-nos-things-that-cant-live-migrate-full-stop">The hard no&rsquo;s: things that can&rsquo;t live migrate, full stop</h2>
<p>A handful of configurations block live migration outright, not because of a missing prerequisite you can fix, but because the feature fundamentally can&rsquo;t be moved mid-flight to different hardware:</p>
<ul>
<li><strong>PCIe passthrough devices.</strong> A GPU or other PCIe device passed through directly to a VM is bound to that specific physical hardware on that specific node. The target node would need the identical device in the identical IOMMU configuration, and even then, QEMU has no mechanism to hand off a live PCIe device mid-migration. A VM with passthrough hardware has to be shut down, migrated offline, and started on a target node that has equivalent passthrough hardware configured, or it simply can&rsquo;t move while running.</li>
<li><strong>CPU type mismatches across nodes.</strong> If a VM is configured with <code>cpu: host</code> (passing through the exact host CPU&rsquo;s feature set for maximum performance) and the target node has a different CPU model, the migration will refuse, or worse, succeed and then crash the guest on a feature the new CPU doesn&rsquo;t support. Clusters with mixed CPU generations should set guests to a generic CPU type (<code>kvm64</code>, <code>x86-64-v2-AES</code>, or similar) instead of <code>host</code> if they need to migrate freely across every node, accepting a small performance cost for portability.</li>
<li><strong>Local-only disks with no replication, revisited.</strong> Worth repeating because it&rsquo;s the most common one: no shared storage and no replication configured means no live migration is possible, period, regardless of how fast the network between nodes is.</li>
</ul>
<h2 id="reading-a-migration-thats-about-to-go-wrong-before-it-does">Reading a migration that&rsquo;s about to go wrong before it does</h2>
<p>The Proxmox web UI&rsquo;s migration dialog shows which mode it&rsquo;s about to attempt before you confirm, and it&rsquo;s worth actually reading that line rather than clicking through on muscle memory. If it says it&rsquo;s about to do an offline migration on something you expected to move live, that&rsquo;s the signal to stop and check storage configuration rather than find out from an unexpected outage. The task log during an in-progress migration also reports the live sync progress directly, so if a migration is sitting at the same percentage for an unusually long time, that&rsquo;s a real signal, usually a saturated migration link or a VM dirtying memory faster than the link can keep up, not a frozen UI.</p>
<p>Live migration is one of the genuinely useful payoffs of running a cluster, but it&rsquo;s not magic, and treating every migration as interchangeable is how people end up with an unplanned outage on the one VM that happened to have a passthrough GPU or no replicated storage. Know which guests on your cluster actually qualify for the zero-downtime path before you need to move one in a hurry, not while you&rsquo;re already trying to do it.</p>
]]></content:encoded></item></channel></rss>