<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Performance on rHomelab</title><link>https://rhomelab.com/tags/performance/</link><description>Recent content in Performance on rHomelab</description><generator>Hugo</generator><language>en-US</language><lastBuildDate>Thu, 12 Nov 2026 00:00:00 +0000</lastBuildDate><atom:link href="https://rhomelab.com/tags/performance/index.xml" rel="self" type="application/rss+xml"/><item><title>ZFS Special Vdev and L2ARC: Where Flash Actually Helps a Spinning-Disk Data Hoard</title><link>https://rhomelab.com/data-hoarding/zfs-special-vdev-l2arc-for-data-hoarders/</link><pubDate>Thu, 12 Nov 2026 00:00:00 +0000</pubDate><guid>https://rhomelab.com/data-hoarding/zfs-special-vdev-l2arc-for-data-hoarders/</guid><description>ZFS has three different flash-caching layers, special vdev, L2ARC, and SLOG, and only one or two of them actually do anything for a typical media hoard. Here&amp;#39;s which.</description><content:encoded><![CDATA[<p>Every data hoarder eventually hears that you can &ldquo;throw an SSD at ZFS&rdquo; to speed things up, and then runs into the problem that ZFS actually has three unrelated ways to do that, with very different costs and very different payoffs. A special vdev, L2ARC, and a SLOG solve three different problems, and for a typical bulk-storage pool full of media, backups, and archives, one of them is genuinely worth doing, one is situational, and one does almost nothing for this workload no matter how fast the drive is. Mixing them up is the single easiest way to spend money on flash and get disappointed.</p>
<p>This article assumes a pool of spinning disks doing what spinning disks in a data hoard normally do: storing a large, mostly-static collection of big files, written once and read occasionally, with a long tail of small files (thumbnails, <code>.nfo</code> sidecars, config, metadata) mixed in. If your workload looks different, a database, a hypervisor&rsquo;s VM storage, heavy synchronous NFS writes, the answer changes, and those cases are called out separately below.</p>
<h2 id="the-three-layers-briefly">The three layers, briefly</h2>
<p><strong>L2ARC</strong> (Level 2 Adaptive Replacement Cache) is a read cache. ZFS already caches hot data in RAM (the ARC), and L2ARC extends that cache onto a fast device when RAM runs out, holding evicted ARC entries so a repeat read can come from flash instead of spinning disk. It is purely a cache: losing the L2ARC device loses nothing, ZFS just falls back to reading from the pool like the cache was never there.</p>
<p><strong>SLOG</strong> (Separate intent LOG) accelerates the ZFS Intent Log, which exists to make <em>synchronous</em> writes safe without waiting for the full transaction group to commit to the pool. A SLOG only helps <code>sync=always</code> writes or explicit <code>fsync()</code> calls, things like NFS with default sync behavior, databases, or VM storage. It does nothing at all for ordinary asynchronous writes, which is most of what a media server, backup target, or torrent client actually does.</p>
<p><strong>Special vdev</strong> is not a cache, it&rsquo;s a real part of the pool. You tell ZFS to store metadata (and optionally small file data below a size threshold you set) on a dedicated set of fast devices instead of on the spinning disks. Unlike L2ARC, a special vdev holds data ZFS cannot regenerate from the main pool. Lose it without redundancy and you lose the pool, full stop. It needs mirrored redundancy matching the rest of the pool&rsquo;s risk tolerance, not a single bare SSD.</p>
<h2 id="why-special-vdev-is-usually-the-one-worth-doing">Why special vdev is usually the one worth doing</h2>
<p>A data hoard&rsquo;s directory-listing and metadata operations, <code>ls</code>, scrubs, <code>rsync</code> dry-runs, Plex/Jellyfin library scans, Sonarr/Radarr checking what already exists, are murder on spinning disks specifically because metadata is small and scattered, which turns a sequential-read-friendly HDD array into a seek-bound mess. None of that workload is sequential. A special vdev moves exactly that traffic onto flash permanently, not as a cache that can miss, and the difference shows up immediately in anything that touches &ldquo;does this file exist and what are its attributes&rdquo; across a large tree: full scrubs, media server library scans, and backup software doing its initial file-list pass before it even starts moving bytes.</p>
<p>The optional small-file threshold (<code>special_small_blocks</code>) extends this further: files below a size you set get written to the special vdev entirely instead of the spinning disks. For a hoard with a long tail of tiny files, subtitle files, <code>.nfo</code> metadata, thumbnail caches, this can meaningfully cut the number of HDD seeks per directory, since those tiny reads never touch spinning disk at all.</p>
<p>The catch that matters most: <strong><code>special_small_blocks</code> only applies to new writes going forward.</strong> Setting the property on an existing dataset does not retroactively move already-written small files onto the special vdev; they stay on spinning disk until rewritten. If you want the effect across your existing library, you need to actually rewrite the data (a <code>zfs send</code>/<code>receive</code> into a fresh dataset with the property already set, or a straight file copy into a new dataset, is the practical way to do it), not just flip the setting and expect it to apply itself.</p>
<p>The other catch: <strong>a special vdev is a single point of total pool failure if it isn&rsquo;t redundant.</strong> This isn&rsquo;t optional risk-tolerance tuning the way it might be for L2ARC. If your main pool is, say, a RAIDZ2 that survives two drive failures, a special vdev on a single unmirrored SSD undoes that entire guarantee, because losing that one drive takes the whole pool&rsquo;s metadata with it, and the pool is gone. Mirror it (two or three small, reliable SSDs) at a redundancy level you&rsquo;re comfortable losing the whole pool over, and size it generously, metadata plus the small-file tier can genuinely consume more space than people expect once you&rsquo;ve been running it a while. Check actual usage with <code>zpool list -v</code> after it&rsquo;s been live for a few weeks rather than guessing up front.</p>
<h2 id="why-l2arc-is-situational-not-automatic">Why L2ARC is situational, not automatic</h2>
<p>L2ARC sounds like the obvious answer, cheap flash extending your read cache, but it has a cost that catches people off guard: <strong>L2ARC headers live in RAM</strong>, not just the device itself. Every entry cached on the L2ARC device needs a small amount of ARC memory to track it, which means a large L2ARC on a RAM-constrained box can actually shrink the effective size of the real (and much faster) RAM-based ARC, making things worse, not better. This is a smaller problem on modern OpenZFS than it was years ago (compressed ARC headers help), but it&rsquo;s still a real tradeoff, not a free lunch, and it&rsquo;s the reason the old rule of thumb &ldquo;don&rsquo;t bother with L2ARC under ~32GB of RAM&rdquo; still has some truth to it even if the exact number has loosened.</p>
<p>For a data hoard specifically, L2ARC earns its keep when there&rsquo;s a genuinely hot, bounded working set that doesn&rsquo;t fit in RAM, a frequently re-watched media library, a *arr stack&rsquo;s active-download working directory, a photo library&rsquo;s thumbnail set, something with real read locality rather than a one-time archival dump nobody touches again. It does nothing for data you write once and read once, years later, if ever, because nothing about that pattern benefits from caching: there&rsquo;s no repeat read to accelerate. If your pool is mostly cold archival storage, skip it; the special vdev is doing the real work and L2ARC would just be flash sitting there mostly idle.</p>
<p>If you do add it, size expectations honestly: L2ARC is a cache, not a tier, so it only helps the fraction of your data that&rsquo;s actually re-read, and monitoring whether it&rsquo;s earning its slot (<code>arcstat</code> or <code>zpool iostat -v</code> showing real hit rates on the cache device) matters more here than for special vdev, where the benefit is structural and doesn&rsquo;t depend on access patterns.</p>
<h2 id="why-slog-is-almost-always-the-wrong-answer-here">Why SLOG is almost always the wrong answer here</h2>
<p>This is the one people reach for first because &ldquo;write cache&rdquo; sounds like exactly what a storage pool needs, and it&rsquo;s almost always the wrong tool for a data hoarder&rsquo;s workload specifically. SLOG only accelerates <em>synchronous</em> writes, and the overwhelming majority of what fills a media/archival pool, bulk file copies, torrent downloads, <code>rclone</code>/<code>rsync</code> transfers, backup software writing full files, is asynchronous by default. ZFS already batches async writes efficiently through its normal transaction group mechanism; a SLOG device sits there unused for that traffic because there&rsquo;s nothing synchronous happening to accelerate.</p>
<p>SLOG earns its cost on a different workload: NFS exports where clients force sync writes (some NFS clients default to sync, VMware&rsquo;s NFS datastore writes being the canonical example), database files living on the pool, or VM disk images where the guest OS expects write durability guarantees on every commit. If none of that describes your pool, a SLOG device is close to dead weight, and the money is better spent on the special vdev or just more spinning disk.</p>
<h2 id="putting-it-together">Putting it together</h2>
<p>For a typical data hoarder&rsquo;s pool, mixed media, backups, and archives, the practical order of priority is usually: add a mirrored special vdev first, since it fixes the directory/metadata slowness that affects every single operation on the pool, every scrub, every scan, every <code>ls</code>; add L2ARC only if there&rsquo;s a real, bounded hot set you can name (a specific frequently-accessed library, not &ldquo;the whole pool&rdquo;); and skip SLOG entirely unless you can point to a specific synchronous-write workload, an NFS datastore, a database, actually living on that pool. None of these decisions are reversible for free, a special vdev especially can&rsquo;t be removed from some older pool layouts without rebuilding, so it&rsquo;s worth being honest about which problem you actually have before buying flash to solve it.</p>
]]></content:encoded></item><item><title>CPU Type and NUMA in Proxmox: Getting VM Performance Right</title><link>https://rhomelab.com/proxmox/proxmox-cpu-type-numa-vm-performance/</link><pubDate>Tue, 10 Nov 2026 00:00:00 +0000</pubDate><guid>https://rhomelab.com/proxmox/proxmox-cpu-type-numa-vm-performance/</guid><description>How Proxmox&amp;#39;s CPU type setting and NUMA topology actually affect VM performance, when host passthrough is worth the migration tradeoff, and how to tell if a VM is quietly paying a cross-NUMA tax.</description><content:encoded><![CDATA[<p>Every new VM in Proxmox gets a CPU type setting that most people never touch after the create wizard, and a NUMA checkbox that gets ignored even more often. Both quietly shape how fast a VM actually runs, and neither shows up as an obvious bottleneck when it&rsquo;s wrong. The VM just runs a bit slower than it should, with nothing in the logs pointing at why. Knowing what these two settings are actually doing is the difference between a VM that uses the hardware it&rsquo;s been given and one that&rsquo;s leaving real performance on the table for no reason.</p>
<h2 id="what-the-cpu-type-setting-actually-controls">What the CPU type setting actually controls</h2>
<p>Proxmox&rsquo;s <code>cpu</code> setting tells QEMU which virtual CPU model to present to the guest, and that model determines exactly which instruction set extensions the guest believes it has access to. This isn&rsquo;t cosmetic. A guest that thinks it has AES-NI, AVX2, or a specific set of newer instructions will have software inside it (compression, encryption, some database engines, video encoding) actually use those instructions if it detects them, and skip slower fallback code paths if it doesn&rsquo;t.</p>
<p>The default and most common choices:</p>
<ul>
<li><strong><code>kvm64</code></strong> - a deliberately generic, conservative baseline that predates most modern instruction set extensions. It works on essentially any x86-64 host, which is exactly the point, but it leaves a lot of real CPU capability invisible to the guest.</li>
<li><strong><code>x86-64-v2-AES</code>, <code>x86-64-v3</code>, and similar</strong> - modern generic levels that assume a reasonably recent CPU baseline (SSE4, AES-NI, AVX depending on the level) without pinning to one specific vendor&rsquo;s exact feature set. A solid middle ground for homelabs with mixed-but-not-ancient hardware.</li>
<li><strong><code>host</code></strong> - passes through the physical CPU&rsquo;s exact feature set and model string to the guest, unfiltered. The guest sees essentially the same CPU the hypervisor sees, and anything the host CPU can do, the guest can ask for.</li>
</ul>
<p><code>host</code> is almost always the fastest option on a single-node setup, because it&rsquo;s the only mode where the guest isn&rsquo;t leaving instruction set capability on the table. For a homelab with one Proxmox box, there&rsquo;s rarely a good reason not to use it. The cost only shows up once more than one node enters the picture.</p>
<h2 id="the-tradeoff-host-breaks-portability-across-mixed-hardware">The tradeoff: <code>host</code> breaks portability across mixed hardware</h2>
<p>A VM set to <code>cpu: host</code> is implicitly tied to the exact CPU model it was started on. If that VM ever needs to move, live migration to a node with a different CPU model, or even just restoring a backup onto different hardware, the guest can either refuse to boot, or boot and then crash the moment something tries to use an instruction the new CPU doesn&rsquo;t actually have. QEMU doesn&rsquo;t retroactively hide capabilities the guest has already detected; it only controls what gets presented at boot.</p>
<p>This matters immediately in a cluster with mixed CPU generations, and it matters eventually even on a single node, the day that node gets replaced or a second one gets added and workloads start moving between them. The practical rule: if a VM only ever runs on one specific, unchanging piece of hardware, <code>host</code> is free performance with no real downside. If there&rsquo;s any chance it needs to move to different silicon, whether that&rsquo;s planned clustering or just &ldquo;I might rebuild this box in two years with a different CPU,&rdquo; use a generic type (<code>x86-64-v2-AES</code> is a reasonable modern default) and accept the small performance cost for portability. This is the same tradeoff covered from the migration-mechanics side elsewhere on this site; here it&rsquo;s worth deciding deliberately up front rather than discovering it mid-migration.</p>
<h2 id="numa-irrelevant-on-most-homelab-hardware-critical-on-some-of-it">NUMA: irrelevant on most homelab hardware, critical on some of it</h2>
<p>NUMA (Non-Uniform Memory Access) only matters on systems with more than one physical CPU socket, or on single-socket CPUs large enough that the manufacturer splits them into multiple NUMA nodes internally (some high-core-count EPYC and Threadripper parts do this even as a single physical chip). On that hardware, memory attached to &ldquo;your&rdquo; CPU&rsquo;s local controller is fast to reach, and memory attached to a different socket&rsquo;s controller is slower to reach, because the request has to cross an interconnect (AMD&rsquo;s Infinity Fabric, Intel&rsquo;s UPI) to get there.</p>
<p>Most homelab Proxmox boxes are a single consumer CPU on a single socket with one NUMA node total, in which case none of this applies and the NUMA checkbox is a no-op either way. Check which situation applies with:</p>
<pre tabindex="0"><code>numactl --hardware
</code></pre><p>on the Proxmox host itself. If it reports a single node, skip this whole section with a clear conscience, nothing below changes anything on that hardware.</p>
<p>On actual multi-node hardware (dual-socket servers, or the higher-core-count EPYC/Threadripper parts), a VM with vCPUs and memory that end up straddling two NUMA nodes pays a real latency cost on every memory access that crosses the interconnect, and that cost shows up as a VM that&rsquo;s measurably slower than its vCPU count and clock speed would suggest, with no obvious culprit in <code>top</code> or the guest&rsquo;s own monitoring. The fix on the Proxmox side is enabling the <strong>NUMA</strong> checkbox in the VM&rsquo;s CPU options, which tells QEMU to expose the host&rsquo;s real NUMA topology to the guest instead of presenting a single flat memory space. The guest OS and its scheduler can then make NUMA-aware placement decisions internally (keeping a process&rsquo;s threads and the memory they touch on the same node) the same way it would on bare metal. This matters most for VMs sized large enough to actually span nodes, a VM with more vCPUs or RAM than one socket&rsquo;s local share, and matters far less for small VMs that comfortably fit inside a single node&rsquo;s local resources regardless of where Proxmox happens to schedule them.</p>
<p>For the small subset of latency-sensitive workloads on multi-socket hardware (a VM doing heavy in-memory database work, or anything explicitly benchmarked and found wanting), the further step is pinning that VM&rsquo;s vCPUs to a specific physical NUMA node with the <code>numa0</code> / <code>affinity</code> style settings in the VM&rsquo;s config, so the hypervisor&rsquo;s own scheduler can&rsquo;t migrate it onto a different node&rsquo;s cores mid-run and strand it away from its own memory. This is a deliberate, workload-specific tuning step, not something to apply blanket across every VM on the box, and it trades flexibility (that VM can no longer freely use whatever cores are idle) for consistency.</p>
<h2 id="a-sane-default-for-most-homelabs">A sane default for most homelabs</h2>
<p>For the common case, one Proxmox node, a modern consumer CPU, single NUMA node:</p>
<ul>
<li>Set <code>cpu: host</code> on VMs that are pinned to that one box for the foreseeable future. There&rsquo;s no portability to protect and no NUMA complexity to worry about, so there&rsquo;s no reason to leave performance on the table.</li>
<li>Skip the NUMA checkbox entirely; it has nothing to do on single-node hardware.</li>
<li>Revisit CPU type only when clustering enters the picture, at which point decide per-VM whether that specific guest needs to migrate freely (generic type) or is staying put on known hardware regardless (keep <code>host</code>).</li>
</ul>
<p>For a dual-socket or high-core-count multi-NUMA box, the calculus flips for anything resource-heavy: check topology with <code>numactl --hardware</code> first, enable NUMA on VMs sized to actually span nodes, and only reach for explicit pinning once a specific workload has actually been measured and found to need it. Tuning a setting that isn&rsquo;t the bottleneck doesn&rsquo;t make anything faster, it just adds a configuration that someone has to remember and explain later.</p>
]]></content:encoded></item></channel></rss>