Veeam Community Edition & Kopia: The DIY “S3 Cloud Tier” for Budget-Minded Admins

by Posted on

TL;DR: Scroll down for the technical how-to.

The Veeam Community Edition is a powerhouse – until you want to move your backups off-site to the cloud without breaking the bank. Native S3 offload (Cloud Tier) is reserved for paid licenses. If you still want off-site protection without paying for terabytes of redundant data, you need a smarter approach.

I’ve found a way to carry over the efficiency of Veeam’s XFS Block-Cloning (Reflinks) into the cloud for free, powered by Kopia.

The Problem: When Synthetic Fulls Bloat Your Cloud Storage

On a Linux repository, Veeam uses Reflinks to create synthetic full backups (VBKs). These occupy almost no additional space on your local disk. However, if you simply sync these files via rsync or rclone, every VBK is transferred and billed at its full size. 100 GB of real data can easily turn into 1 TB of cloud costs.

The Solution: Kopia as a Deduplication Bridge

Kopia uses Content-Addressable Storage. It doesn’t care about filenames; it looks at the actual data. For Kopia to recognize that “Block A” in VBK_1 is identical to “Block A” in VBK_2, the data splitting (chunking) must be perfectly aligned.

I wanted to find out for sure: Which algorithm squeezes the maximum out of Veeam VBK files? I put all 23 available Kopia splitter variants to the test, pitting them against each other using a test dataset of 8,695 MB (consisting of 3 synthetic full backups).

Here are the most relevant results:

Splitter-TypePhysical (MB)SavingsDurationBenchmark (Speed)
DYNAMIC-128K-BUZHASH3.607 MB~58,5 %290s~651 MB/s
DYNAMIC-256K-BUZHASH3.744 MB~56,9 %221s~654 MB/s
DYNAMIC-512K-BUZHASH3.866 MB~55,5 %231s~663 MB/s
DYNAMIC-1M-BUZHASH4.177 MB~52,0 %229s~677 MB/s
DYNAMIC-4M-BUZHASH4.854 MB~44,2 %238s~674 MB/s
FIXED-1M8.502 MB~2,2 %316s~59,4 TB/s*

* FIXED is incredibly fast because it doesn’t calculate boundaries, but a 2% deduplication rate is a total failure for Veeam data.

Performance vs. Precision

As you can see, 128K-BUZHASH is the clear winner. Despite the small block size, performance remains high at ~650 MB/s. The bottleneck is almost always your network or source storage, not the CPU.

What about Index Overhead?

Small blocks mean more hashes in the index. In my tests:

  • 128K-Buzhash Index: ~2,89 MB
  • 1M-Buzhash Index: ~0,75 MB
  • 4M-Buzhash Index: ~0,26 MB

Extrapolated to a 100 TB repository, a 128K index would require about 32 GB of RAM. For most SMB setups, the 128K “Surgical Splitter” is the best choice to minimize storage costs.

If you’re adressing petabytes of storage or need to save on memory (not so unlikely these days), you can switch to 256K or 512K BUZHASH.

From Lab to Production: Real-World Results

After the benchmarks, I moved my production system to Kopia. My setup consists of Proxmox VM backups and various Windows and Linux agents. Even with a 14-day retention (meaning up to 3 synthetic fulls per job), the results are stunning:

  • Source Data (Logical via Veeam): ~610.0 GB
  • Storage Box Usage (Physical via SFTP): 263.0 GB
  • Total Savings: ~57 %

I’m saving over 340 GB of storage costs every single month.

My recommendation: Use DYNAMIC-128K-BUZHASH. It’s the secret weapon to keep Veeam’s block-cloning magic alive when moving to off-site storage. Kopia supports S3, Azure, Backblaze B2, Google Cloud, WebDav, SFTP, and more. I personally use a mix of S3-compatible storage and SFTP for my Hetzner Storage Box.

Conclusion

Fazit: Profi-Cloud-Backup zum Nulltarif

With this setup, Veeam Community Edition is finally “cloud-ready.” Spending 15 minutes to configure Kopia with the 128K-BUZHASH splitter saves real money every month while ensuring your backups are encrypted and deduplicated off-site.

By the way, Kopia stores all data in the repository with AES-256 encryption by default—so your data remains secure in the cloud, regardless of whether it was already encrypted by Veeam or not.

TL;DR: Console Quick-Start

# 1. Initialize repository on a local path or NAS
# (The splitter is permanently set here for the repository format)
# You will be prompted for an encryption password during repository creation
kopia repository create filesystem --path /mnt/kopia_repo --object-splitter=DYNAMIC-128K-BUZHASH

# 2. Set global policy (Retention & Splitter-Override)
# Here we define the 128K "surgeon" and retention periods for all jobs. 
# If the splitter was already configured during repo creation, it doesn't need to be set here again, and vice versa.
kopia policy set --global --keep-latest 7 --keep-daily 14 --keep-weekly 0 --keep-monthly 0 --splitter=DYNAMIC-128K-BUZHASH

# 3. Create the initial snapshot of the Veeam backups
kopia snapshot create /repo/backups/meine_veeam_jobs

# 4. Automate snapshots, e.g., via CRON 
0 5 * * * /usr/bin/kopia snapshot create --all

Published by Stefan Zimmermann