The best advice I heard about setting up a homelab is to “use what you have”. I don’t remember where I heard that first, but it stuck with me, and I think it’s a great advice and starting point.
The second best advice I learned is to “be curious” and try to learn as much as you can, and while I had a infrastructure experience for the past 10+ years, I wanted to refresh my knowledge about ansible, and learn more about linux, setting up a fully secure, self-hosted, open-source, private homelab with some services that were helpful to me, and my partner.
I had a few old laptops and one raspberry pi that were sitting around from previous projects, so I decided to use them to set up my homelab. Below are the devices I used:
| Device | RAM | Storage | CPU | GPU | OS |
|---|---|---|---|---|---|
| Raspberry Pi 5 | 16GB | 64GB MMC, 1TB SSD | ARM | N/A | Raspberry OS Lite |
| Lenovo Thinkpad T450s | 8GB | 256GB SSD | Intel i5 | Intel HD 5500 | Proxmox VE |
| MacBook Pro 2015 | 16GB | 512GB SSD | Intel i7 | Intel, Nvidia GT750M | Ubuntu, Archlinux |
| Desktop PC | 32GB | 512GB SSD | Intel i5 | Nvidia GTX1060 | Archlinux |
The following services are running on these devices:
On the Raspberry Pi 5:
- DNS Sinkhole (Pi-hole).
- VPN Gateway (Wireguard).
- Backups (Garage S3 Object Store).
On the PVE nodes1:
- Monitoring (Prometheus, Grafana, Loki, …).
- Git Server (cgit).
- CalDav Server (Radicale).
- VPN and Mesh Networking (Headscale, Tailscale).
- Subscription tracking (Wallos).
- Obsidian notes live sync (couchdb).
- WIP: Streaming services (Jellyfin).
The implementation for these services is done in the following projects:
Ansible Distro Install
is a collection of ansible roles to install different Linux distributions on a variety of devices.
It supports:
- Installing Archlinux, Ubuntu, Proxmox VE, and Raspberry OS.
- Configuring dual booting with Archlinux and Ubuntu (and possibly other distros in the future).
- Provisioning a fully functional installation of the desired distro (using the
ansible-distro-configureproject).
For now, I’m trying to give the Macbook Pro a second life by installing linux on it, and it does work, but there are still some issues with the graphics drivers, so I will have to look into that in the future, or use it as a second PVE node.
The project consists of 3 roles:
partition: Responsible for partitioning the disk and creating the necessary filesystems.bootstrap: Responsible for bootstrapping the OS and installing the necessary packages.configure: Responsible for running theansible-distro-configureproject.
While the bootstrap role could theoretically be built to run any distro installation from any Live ISO2, I
opted-in to use more traditional method of using a live USB with the desired distro, and a small commands that needs
to be ran manually to set up the network and ssh access, and then run the ansible playbook to install the distro.
Ansible Distro Configure
is a collection of self-contained, modular ansible roles to configure a bare linux installation to fully customized. It supports configuring Ubuntu and Archlinux setup with desktop environment, dot files, password manager, web-browsers, and applications.
It’s open-source, and doesn’t require the installation to be done via the ansible-distro-install project,
but it integrates easily with it.
The roles are split into 3 categories:
tag: required_for_bootroles required to get default installation to a basic accessible state over network.hostname: Configures the hostname of the machine.network: Configures the NetworkManager and default network settings.sudo: Configures the sudo group root access.user: Create the user configure default user with authorized_keys (for ssh).ssh: Configures and hardens the ssh server settings and adds the known_hosts.password_policy: Configures the password policy for the system.
configure.ymlroles, which configure the system to be fully functional and ready to use with all the goodies.timezone: Configures the timezone and NTP settings.dotfiles: Configures the dotfiles for the user.password_store: Configures the pass password manager and adds the default passwords.hyprland: Configures the hyprland Archlinux desktop environment and adds the default settings.hyprtheme: Configures the default themes for the hyprland desktop environment.bash,vim,alacrity,tmux,steam,gpu_driver, … and many more.
service.ymlroles, which configure the PVE containers and deploy these services.proxy: Configures the reverse proxy and cloudflare tunnel/makeshift-DDNS 😅.headscale: Configure the headscale plane server.tailscale: Configure the tailscale service/container with exit nodes, authkeys, and auth server (headscale).git: Configure the self-hosted cgit server.caldav: Configure the self-hosted radicale server with accounts and shared calendars.subscription: Configure the self-hosted wallos subscription tracking service.notes: Configure the self-hosted couchdb service for obsidian notes live sync.monitor/metrics: Configure the self-hosted prometheus, grafana, and loki monitoring stack.- …
There is also the upgrade.yml playbook, which is used to upgrade the system and the services.
Architecture
Except for the git, caldav, and headscale services, that are publicly accessible -the CalDAV service
doesn’t need to be publicly accessible, but it is added for convenience-, all the other services are only accessible
from the local network, or via VPN (Wireguard, Tailscale) on .internal domain names, that are resolved by the
Pi-hole DNS resolution, while still retaining the same VPN functionality mentioned before in
My Home (VPN) Network Setup.
Here is the almost correct full digram of the homelab setup:
---
config:
theme: redux
look: neo
---
graph TD
Internet[🌍 Internet] <--> ISP[ISP: 2.213.76.?]
Internet <--> VPN[VPN: 149.40.8.?]
ISP <--> SUBNET[Router Subnet: 192.168.0.1]
VPN <--> VPN_SUBNET[VPN Subnet: 10.2.0.2]
subgraph "Local Area Network (LAN)"
SUBNET <--direct mode--> RPI[Raspberry Pi: 192.168.0.254]
VPN_SUBNET <--VPN mode--> RPI
RPI <--> D1[TV: 192.168.0.100]
RPI <--> D3[Desktop: 192.168.0.210]
RPI <--> PROXY[proxy.intenral]
subgraph "Proxmox VE Cluster"
PROXY <--> TS[tailscale]
PROXY <--> MONITOR[monitor.internal]
PROXY <--> SUBS[subscriptions.internal]
PROXY <--> ...
end
end
TS <--> D4[Laptop: 163.116.173.?]
TS <--> D5[Phone: 2a02:3032::1::?]
Closing thoughts
I have been managing my whole homelab setup with this ansible playbook approach for a while now, and it has been working great, I can easily add new services, or change the configuration of existing ones, the data of all these services is backed up to the garage S3 object store, and I can easily restore the backups when needed.
The next steps are to add more nodes to the PVE cluster, investigate the approach of LXC containers vs KVM virtual machines, add more services (ex. jellyfin), add offsite-backup replication, and maybe build a server case for these devices 😅.
Feel free to reach out if you are interested in this project, have questions, recommendations for a follow write-ups, ideas to add, or just want to say “hi”!
-
Right now PVE is deployed on a single node (Lenovo Thinkpad T450s), but I plan to add more nodes for redundancy or reuse the MacBook as a second node in the future. ↩︎
-
The Proxmox VE installation is using a Debian live ISO, because the Proxmox VE ISO which comes already with its own installer, but the installer can’t be automated via a bash script or ansible playbook. ↩︎