Homelab Revamp 2022 – Part 3 – The BOM

Sorry, took a while for this next edition of my Homelab Revamp series. Check out the previous articles to learn about my motivation and starting point. Let’s go down to business…

AMD or Intel

The first question when building a new PC is of course Intel or AMD. This choice is always a combination of CPU and chipset and I tried one from both vendors with comparable specs and prices. I looked for the following in the configs:

Multi-Core – running VMs is a multi-threaded thing, so the more cores the better. SMT is a must as it gives more logical cores.
Integrated graphics – not really needed for this use case, but I didn’t want to struggle with PCIe graphic cards, especially for trouble shooting. And future reuse of the hardware might be better.
ECC memory – I decided against ECC, but having it as an option when I notice I need it might be a good thing to have.
Low TDP – machines run 24×7 so the amount of power they consume and heat they produce should be minimal.
Future proof – I was thinking about upgrading to full NVMe at some point. So best would be to have a bifurcation option for the x16 PCIe slot to run it with 4×4 NVMes
Two PCIe slots – the x16 slot should be empty for the NVMe option, but I need to put a 10G NIC in the system (alternatively I have a 10G NIC on board)
Two M.2 slots on-board – VSAN cache and option for boot disk
Min 4x SATA ports – to connect my SATA SSDs (3x per server) and SATA boot drive

Item	Intel	AMD
Model	Core i5-12600	Ryzen 5700G
Codename / Architecture	Alder Lake-S / Golden Cove	Cezanne / Zen 3
Socket	1700	AM4
Cores/Threads	6/12	8/16
Base/Turbo frequency	3.3 / 4.8 GHz	3.8 / 4.6 GHz
ECC memory	yes	no
TDP	65-117W	65W
PCIe lanes	16x PCIe 5.0, 4x PCIe 4.0	24x PCIe 3.0 (16+4+4)
Price (05/2022)	~250 EUR	~290 EUR
Performance (cpubenchmark.net) (Single-/Multi-Core)	3836 / 21092	3283 / 24549

Compared CPU models

Some more notes:

The Intel 12k CPUs introduced two different kind of cores (BIG / little). Though in ESX (which I want to run), there’s not yet support for this, so to run it properly the little cores need to be disabled. The 12600 is the biggest CPU without little cores and thus it makes no sense to invest more money for CPUs with efficiency cores when they have to be disabled.
Though the Intel CPU supports ECC I did only find super expensive (~500 EUR) Supermicro boards with socket 1700 to support ECC, so basically this is an unusable option.
While the AMD CPU doesn’t provide ECC support, the platform (AM4) does on most boards. So changing to another CPU (no graphics) would be an option to go to ECC.
Both CPUs support 128GB of memory

Chipset wise I looked for boards with the Intel B660 and the AMD B550M on µATX boards. Boards with integrated 10G were too expensive (>500 EUR for AMD) or not even available (Intel). I already have the 10G SFP+ NICs, so I’ll make use of them.

The higher class chipsets like Z690 or X570 increased the board prices without giving me more features I need. Imho the AM4 platform in general gives you more value for your money. You’ll get things like 2.5 Gbit/s networking at lower prices than comparable Intel boards.

Overall platform cost (CPU + Mainboard) was almost the same at the time when I looked (AMD: 500 vs Intel 510 EUR).

Testing

So now I tested both systems with a full 128GB UDIMM config, one Samsung 870 EVO 4TB, a Gigabyte Aorus PCIe 4.0 1TB NVMe and a Sandisk 480GB SSD as boot disk. I set everything up as a one-node VSAN with vSphere 7 and did run some tests.

Intel

The B660 board I checked did not support bifurcation and I did check a lot of BIOS manuals but didn’t find any mention of it.
Power usage for an idling system was at ~50W. Putting 3 VMs on the ESX system brought the consumption up to almost 100W. This was already tested with unnecessary onboard devices (LAN, audio, serial) disabled and power limits properly set for the Intel CPU.
FIO tests were great – almost 40k average write IOPS and 57k avg. read on latencies <100us. Average transfer rates for read/write were above 2GB/s. Peak values were much higher.
I think this comes down to the PCIe 4.0 NVMe in this setup.

AMD

PCIe 4.0 SSD was only connected with PCIe 3.0 (board supports PCIe 4.0 but the 5700G only 3.0)
4×4 bifurcation was not choosable in the bios, only 1×8+2×4 – after clarifing with Gigabyte (mainboard) support, this is a limitation for the AMD CPUs with graphics – graphics are internally connected via x8 PCIe, so there’s no other setup possible. CPUs without graphcis (like 5700X) would support 4×4
Power usage in idle was at 30W, during FIO load tests (50% CPU load on the system) the peak was at 64W with an average at 57W.
FIO performance was lower – “only” 32k read and 53k write iops, latencies were higher (125us vs <100us) and average transfer, too (~1,4GB/s / 1,2GBs/). The peaks were higher, though not so high as on the Intel config.

Decision

I decided for the AMD platform. This might not have been your choice, but here’s why:

Power usage – I calculated the difference in Total Cost of Ownership with my current energy prices (which are still quite cheap but might increase a lot on next provider change). In the end over a 5 year time I’d save about 250 EUR per server when going with AMD. Taking the lower performance into account that would still mean an aveage of 92% of performance (I/O, single & multi-core) for 84% of the price. Point for AMD
Though I liked the idea of latest gen hardware and PCIe 5.0 there’s really no use for it. Even with PCIe 3 I’d be still good – I only have a 10G connection and probably won’t upgrade within the next 5 years to something faster as 10G is today still quite uncommon at home. 10G is about 1Gbyte per second and the AMD platform can handle that. Also DDR5 was not a reasonable choice (price wise).
The AMD platform gives me more options in terms of extendability. I can go ECC when changing the CPU and I can update to PCIe 4.0 (if the 3.0 performance really isn’t enough) on the same board. Also the platform does support bifurcation if I’d like to go to an all NVMe setup.
I get more multi-coring on the AMD platform (8 vs 6 cores)

The BOM

So finally, here’s the BOM with prices I paid beginning of 2022. Links go to vendor sites. Take this info to the price search machine of your choice.

CPU	AMD Ryzen 5700G (8C/16T)	270 EUR
Mainboard	Gigabyte B550M AORUS Pro-P	113 EUR
Memory	4x 32GB DDR4-3200 any vendor (Adata/Transcend in my case) à 110 EUR	440 EUR
Cache NVMe	Gigabyte AORUS Gen4 SSD 1TB (1.8 PB TBW!)	130 EUR
Case	Sharkoon VS4-V	42 EUR
PSU	be quiet! Pure Power 11 400W	44 EUR
Cooler	Arctic Freezer A13 X CO	19 EUR
Sum		1,058 EUR

For memory I don’t have any special requirements. There are specs for it, I don’t overclock, so I ususally just take the cheapest modules which fulfill my requirements.
I reused my Samsung Evo 870 4TB SATA SSDs and my HPE 560SFP+ 10G NICs, so I didn’t include them in the BOM.
I didn’t like the packaged CPU cooler with the AMD CPU (especially the mount mechanism), so I’ve chosen to go for a cheap alternative with the Arctic Freezer A13 X CO.
I went for the AORUS Gen4 SSD as it brought even in PCIe 3.0 mode more performance. And if I’d ever go to another CPU with PCIe 4.0 I can use this. TBW is great, too and might be useful for the cache device.

Conclusion

So with this BOM I get for about 2,000 EUR additional invest on what I have today a high available setup for my home lab. I’d be going through some learnings on building and configuring it in a future post.