Part 7 — The Slippery Slope from Raspberry Pi to Distributed Infrastructure
At some point during this whole “accidentally building my own cloud” saga, I remembered I had a Mac mini lying around in a drawer.
Now, to be fair, I already had plans for it. Mostly development-related things. Maybe some testing. Perhaps a Linux experiment or two. The usual “this machine will surely be useful one day” kind of reasoning that all developers seem to suffer from.
At the time, though, the Raspberry Pi 5 was already doing an alarming amount of work:
reverse proxy duties
VPN access
media services
backups
Docker containers
and, of course, helping power the ever-growing Needle ecosystem
And honestly? The tiny thing handled it remarkably well.
Modern Raspberry Pis are absurdly capable little machines. The fact that you can build a genuinely useful personal cloud around something barely larger than a sandwich still feels vaguely illegal to me.
But then Needle started evolving.
The local-first music player became:
a backend
a web player
a synchronized playback system
a metadata processing machine
and eventually a full-blown music platform
Which meant the poor Raspberry Pi suddenly found itself crunching through:
large FLAC libraries
artwork processing
metadata indexing
maintenance tasks
file operations
background jobs
And that tiny fan…
Oh, that fan.
At first it would spin up occasionally.
Then more frequently.
Then with increasing urgency.
At some point I realised I had developed an emotional relationship with the sound of a Raspberry Pi entering thermal combat.
You know the sound.
That tiny:
“whrrrrrrrrrr…”
which quietly says:
“Sir, respectfully, I am doing my best.”
And one evening, while listening to the Pi desperately chewing through Needle’s metadata updates, I looked over at the Mac mini sitting there all cool, silent, and criminally underutilized.
And I thought:
“Well… this seems silly.”
Because suddenly the architecture no longer made sense.
The Raspberry Pi was acting as:
reverse proxy
networking gateway
VPN endpoint
media host
infrastructure layer
AND metadata crunching worker
Meanwhile the Mac mini was basically sitting there like:
“You know I have actual horsepower, right?”
So naturally, instead of behaving like a normal human being and keeping things simple, I began splitting workloads across machines.
Which sounds dramatically more professional than it actually was.
In reality, it mostly consisted of:
moving containers around
editing Docker configurations
staring at logs
muttering at networking
and occasionally breaking things in creative new ways
Slowly, though, a structure started emerging.
The Raspberry Pi became more of an infrastructure and networking machine:
reverse proxy through Caddy
VPN access
traffic routing
lightweight services
gateway responsibilities
Meanwhile the Mac mini started taking over the heavier lifting:
Nextcloud
Immich
larger containers
storage-heavy services
media-related workloads
things benefiting from significantly more CPU power
Which, unfortunately, is the exact moment where a homelab quietly evolves into “infrastructure.”
Because your brain starts changing.
You stop thinking:
“Can I run this?”
and start thinking:
“Where should this live?”
That tiny mental shift is dangerous.
Suddenly you catch yourself caring about:
workload separation
storage performance
network paths
service responsibilities
CPU architecture
thermal behaviour
backup topology
reverse proxy routing
And before you know it, you’re troubleshooting CalDAV redirects on a Saturday afternoon because Apple Calendar has decided your infrastructure is personally offensive.
Naturally, there were also some… learning experiences.
For example, I discovered that using .local hostnames between systems is apparently a wonderful way to invite chaos directly into your home network.
Things would work beautifully for days.
Then suddenly:
containers couldn’t see each other
reverse proxies got confused
services pretended other machines no longer existed
and debugging became an archaeological expedition
Eventually I gave up and started using fixed IP addresses like some ancient network wizard who had finally run out of patience.
And honestly? Things immediately became calmer.
That’s the funny thing about this whole project.
The backend became more complex.
But the day-to-day experience became simpler.
Needle became faster.
Media became easier to access.
Sync became more reliable.
Services stopped fighting each other quite so much.
The infrastructure faded further into the background.
Which, ironically, is probably the best compliment infrastructure can receive.
And the weirdest part?
None of this happened because I set out to build a sophisticated multi-machine setup.
I just looked at a tiny overworked Raspberry Pi fan screaming into the night, glanced over at an idle Mac mini sitting in a drawer, and made one innocent little decision:
“Maybe this machine should help.”
As it turns out, that’s a very slippery slope indeed.