Best laid plans

Jun 7, 2024 m73 lenovo homelab kubernetes hardware mpcie ethernet

Putting together a homelab Kubernetes cluster in my own stubborn way. I’m assuming a reader who’s basically me before I embarked on this little expedition, so I won’t go into minute detail about day-to-day Linux setup and administration - only the things that are new to me and have changed since I last encountered them.

Sections added as I proceed with this!

The genesis

I’m mostly an AWS flavoured person at work, but most of our stuff runs under Kubernetes (a mixture of self-managed and EKS now) and I don’t know that as well as I would like to. Of course I’ve played with minikube, and I did set up a tiny cluster in the cloud sandbox on Linux Academy¹ following their CKAD course back before they got bought by ACloudGuru and they in turn by PluralSight.

Still, I had a hankering to run on something a bit more like a real cluster. Plus I want to try things like yanking the network cable out of a worker instance and seeing how well it handles it. People prefer it if you don’t do that sort of thing at work.

Anyway, with these vague thoughts in mind I saw this listing on eBay:

Ten barebones Lenovo m73 Tiny PCs

At £15 per unit … and not a lot more with postage and packaging… that seems like too good a bargain to miss. They’re only populated with 4G of RAM, but I happened to have a small pile of suitable 8G SODIMMs from an earlier project. No disks, but wouldn’t it be fun to have them netboot and use my Synology NAS for their storage. Could I build a little cluster out of these? I mean, it would be rude not to…

They turned up, well packed with no damage, and they’re pretty solidly constructed little machines anyway. What’s inside the tin though?


These are not mighty powerhouses.

The basic upgrades

I had enough 8G RAM SODIMMS kicking around that I could bump eight of them up to 16G RAM and of course that left me with plenty of 4G SODIMMS so the remaining two got 8G apiece.

One of the machines would end up being the “cluster gateway” and would need a disk and two network ports. The disk is easy, I had a suitable 512G SATA SSD of indeterminate speed, that went in no problem.

That second network port… well, there was a blanking plate on the back for something; probably a serial port or something? Surely I could mount an ethernet port there if I had something to connect it to inside?

Note the blanking plate above the VGA connector

Inside there’s an unpopulated Mini PCI Express (mpcie) slot that’s normally occupied by the wifi card. Using this to drive a wired ethernet port appealed to me as it would be capable of supporting fast speeds and I wouldn’t have an annoying USB dongle hanging off the back of the machine. I bought a suitable mpcie Ethernet board on Swedish Amazon and crossed my fingers that it would be ok in my system.

It is possible to upgrade the CPU in these machines (even as far as a Xeon apparently) but for now I’m not looking for a performance beast, and the upgrade cost would more than double the “investment” in them so far - £30 a pop! Perhaps I’ll upgrade that later if I’m still having fun with it.

Uh, yeah, the plan

So, here’s the plan…

A sketch of how I want things to communicate

The Telia Router is my home internet router. We get symmetric gigabit internet out of that thing (yay, Sweden!) which is damned useful as both my wife and I work from home a lot of the time. Anyway that’s our gateway to the internet and it needs to be accessible from the machines in the cluster I’m setting up.

That router’s configurable, but out of the box it provides dhcp for a subnet of and itself sits and listens internally to the subnet on as the gateway. It NATs the internal subnet to the real internet and it does get assigned a public IP address and one can route on a per-port basis to things inside the subnet (not that I need that much). I won’t be changing any of that. What it doesn’t supply is any fancy dhcp flags. This includes options 66 and 67 which are what you need for netboot. So if I want to netboot my cluster, I’m going to need to set up something custom.

That’s where the PC I’m upgrading to have two ethernet ports comes in. This is my “cluster gateway server” and it’s going to look like this:

The mess inside that cluster gateway server

On the built-in Ethernet port it’s going to look like a fairly normal client PC on the Telia router’s subnet.

On the newly added Ethernet port it’s going to host a bunch of services:

As well as those services - which are all primarily visible on the new Kubernetes subnet, the gateway will do two other important things:

Once that’s all set up I’ll do one other tweak; I will need to add the cluster gateway as an entry in my laptop’s route table. Otherwise my laptop will send them to its default route, which means it will pass them to the Telia Router… which knows nothing about that subnet and will reject them.

Basically I will tell my laptop:

“If you want to send traffic to 192.168.0.someting then give it to and that machine knows what to do with it.”

I think this will all work, but there are some big fuzzy patches around the following:

I’m sure there will be other surprises along the way.


Next up, collating all this hardware and making sure it works.

Part 1 - Installing the hardware


¹ I enjoyed that LinuxAcademy course, but I didn't bother to sit for the certification because the proctoring process (install shitty software on your laptop and clear your room of all material) sounds so hideous. At least I can (and do) take the AWS exams at one of their partner's premises using their hardware instead of mine.

² "Pre-Boot eXecution Environment", which used to be called "NetBoot" ... basically it's a bunch of BIOS extensions that knows how to do basic DHCP configuration, how to use that to find a TFTP server on the subnet, and how to use that to find, load, and run a binary.

© 2017 - 2024 Dave Minter