homelab
Three machines, one reverse proxy, a private network and nightly Btrfs snapshots: how my family cloud fits together, and what I learned along the way.
· blog
For the past year and a half, my family has been using its own cloud, including photos, files, chat, mail, passwords. This cloud is actually just a shared Linux box and a couple of game servers, all on two machines at home. This is how it's put together, what broke along the way, and what I'd tell someone starting out. Everything is kept in one Git repo of config files, which was a very useful way to jog my memory; the commit history is a decent diary :)
What it looks like
Monitoring dashboard for Atlas
There are three machines. Atlas is the main one and runs nearly everything. Ares is the second, mostly for game servers, so they live apart from the family's data. Lastly, Eros is the third, a Raspberry Pi that maintains the crucial services, specifically Headscale (VPN), Authentik (IdP), and Adguard Home (DNS).
Almost every service is a Docker Compose project in its own folder. Each project is started by a small systemd unit, so Docker comes up with the machine and I can stop and start things with systemctl. The exception is the chat server, which is a pile of containers managed by an Ansible playbook (which needs special handling for backups, how fun).
What's on it:
- Files and documents: Nextcloud, with an office suite and a whiteboard next to it
- Photos: Immich
- Chat: a Matrix server
- Mail: Stalwart, our own mail server
- Passwords: Vaultwarden
- Login: Authentik, an identity provider all other apps use
- Game servers: the Pelican panel on Atlas, its Wings daemon on Ares
- Mischellaneous: a finance app, a trip planner, a flight log, and location history
- Plumbing: Caddy, AdGuard Home, Headscale, ntfy, Beszel, Diun
Crucially, there is a big requirement for all services to be free and open source, which means everything is locally hosted, with no proprietary software. There are countless ethical reasons to prefer open source software, but the most important ones are that it's truly free, with no vendor lock-in.
The front door
Only one thing answers on the web ports: Caddy. Every service listens on localhost only, and Caddy decides by hostname who gets the request, adds compression and security headers, and handles certificates by itself. Adding a service is a Compose file plus a handful of lines in the Caddyfile.
The domain's DNS is on Cloudflare. It was proxied for a while (which is something I reccommend to everyone until you're truly comfortable with the security implications), but today plain A records point at the house and ddclient keeps them current as the public IP changes. The tunnel setup is still active and document, ready to be used at any moment. The firewall is ufw with everything denied by default, and I keep the output of its status command in the repo.
Mail and Matrix are partial exceptions. SMTP can't go through Caddy, nor can Matrix Federation, so those ports are open directly.
Keeping it private
The part of our homelab I like the most is the access control setup. Authentik is an identity provider that handles all authentication, and all services must login through it, making single sign-on easy. Headscale is our control server for Tailscale, so we have a private network that everything joins, with Headplane as a web UI for it. The best part about Tailscale is that through ACLs, members can reach their own devices and the servers, servers can reach each other, and only admins can manage the server tags.
AdGuard Home is our DNS server, responsible for blocking ads and trackers. It also resolves local .lan names, which are accessible only via Tailscale or our local network. These .lan names are useful for quick access to sensitive services like router panels and the AdGuard control panel.
Headscale panel for Atlas (via Headplane)
Since these 3 services are the backbone of user connections, they are treated as critical and kept on a separate machine (Eros). Backups are still sent to Atlas due to size constraints.
Additionally, Eros runs:
- ntfy sends push notifications to our phones.
- Beszel monitors all 3 machines.
Desk
ssh desk
The newest piece is Desk, a shared Linux virtual machine that lives on the tailnet. Each of the four of us has an account, and you reach it by ssh desk, with no keys or passwords, since the tailnet already knows who you are. Files, installed tools and tmux sessions stay on it, so you can pick up where you left off from any device. It syncs a folder both ways with Nextcloud, and each person can install software with a small pkg wrapper around Nix, without root and without affecting anyone else.
It's a container, which means no systemd inside, and tmux sessions die when it restarts. My fix for this has been using scripts in a special user folder that run on every start, and another special user folder that runs before shutdown while sessions are still alive.
The desk has easily been my favourite addition in the past year. I've been using it as a Sandbox for my Claude Code, which through Nextcloud, has access to select folders and files that it can use to help me study for my courses, and having it accessible from any device without any special setup has made it a great tool for learning.
A backup a day...
The disk is Btrfs, split into subvolumes that cover the system, home, Matrix data and the Docker app folders. btrbk takes snapshots at 3am every day of all relevant subvolumes and sends them to a separate backup drive, keeping a week of dailies plus one weekly and one monthly.
A snapshot of a database in the middle of a write is a bad backup, so the job doesn't call btrbk directly. A small script stops the Matrix playbook and every running Docker unit, takes the snapshot (which is nearly instant), and then starts everything again before the slow transfer to the backup drive. It remembers which units were running, so it only restarts those. It also traps exit, so even if the script dies halfway, the services come back. Services are down only for the snapshot, at night, and the slow copy happens with everything back up.
No stale services
Especially in the age of AI, updating software is a necessity. I check for new container versions with Diun. It watches the containers I label, once a day, and sends me a push notification through ntfy when a new image shows up. It can't tell whether a tag is newer, only that it's different, so I have to make use of regex for each service to ensure only relevant tags are matched. Diun doesn't update anything itself, I read the notification and decide (pin your images, and always check the changelog before updating).
Still a long way to go
This homelab has been my entrypoint into infrastructure and DevOps, and I'm still learning a lot. Throughout the past year, a lot of things have changed:
- Services come and go, and thats ok. A budgeting app, a fitness tracker, and more were all on the server at some point and are gone now. Its important to realise when a service simply isn't worth it.
- The repo drifted away from the server. One of my commit is literally named Updated to latest files on Atlas, and it changes 33 files. The server had moved on for months without the repo following. A repo that's only a snapshot of the machine isn't config management, it's a diary.
- Networking. I constantly made changes with Cloudflare tunnels, port forwarding, and other networking configurations. With all these changes, the local network went offline at least 5 times in the past year.
There are plenty of ambitious plans for the future, like offsite backups, router replacements, or even media serving. This all happens with time (and money, but that's a different story), and its never done, ever. There will definitely be a couple more blog posts in the future with who knows what.
If I did it again
- Set up backups before the second service, not after the first thing you'd miss.
- Keep the config in Git from day one, and make that the only way things change. I didn't update the repo for months, and spent days redocumenting.
- Secrets stay out. Compose files should at least be read from
.env files, with the repo only having .env.example copies. Ideally use Docker secrets or a similar solution, but even I'm not there yet. - Pin versions that matter (the database behind the photos, the mail server) and let the rest float, with something watching. It's easy to miss a migration that will break things.
- Plan for being the only person who understands it. Documentation is surprisingly useful, especially if you're the only person managing the servers, its very easy to forget how things work.
It's not a polished product, and it never will be. It's very useful to a handful of people who'd otherwise be paying for a stack of subscriptions. And it makes a good place to learn how fragile the internet actually is.