VPS public SSH: enable fail2ban with bantime-increment so brute-force
probers get evicted with exponential backoff (1h → 4h → 16h → 2.7d →
10.7d, capped at 30d). Default jail covers sshd; maxretry=5 in 10m.
server-debug-tools: htop, tcpdump, dnsutils, jq, curl. Imported by
sunken-ship + phantom-ship via flake.nixosModules.server-debug-tools.
These are the practical bits we'd otherwise pick up by enabling
clan.core.enableRecommendedDefaults — but the full clan defaults flip
systemd-networkd/resolved on, which broke dnsmasq + navidrome's resolv
.conf bind-mount on the homelab servers, so we cherry-pick instead.
Stage 4.5: declare a Hetzner-hosted reverse-proxy VPS as a clan machine.
- nixos/hosts/vps-relay.nix: Debian→NixOS cx23 in hel1. Caddy at public
80/443 reverse-proxies navidrome.dannydannydanny.me and
bbbot.dannydannydanny.me over ZT to sunken-ship.
- nixos/disko-cloud.nix: simple GPT + ext4 root, no LUKS — cloud provider
has physical disk anyway and there's no operator at boot.
- flake-modules/clan.nix: register vps-relay as an inventory machine,
zerotier peer, internet networking target at its clan-generated ZT
IPv6, and add vps-relay.clan to clanHostsModule /etc/hosts.
- sunken-ship fitness-bot: drop pkgs.cloudflared from PATH + set
WEBAPP_URL=https://bbbot.dannydannydanny.me. Paired with the bbbot
upstream patch (start.py honors env WEBAPP_URL and skips cloudflared
when set) — once the 15-min fitness-bot-pull timer pulls that change,
bbbot will stop churning trycloudflare.com URLs.
Vars (zerotier identity/ip + sops machine key) generated on sunken-ship
because clan's hermetic sandbox on macOS fails to run the zerotier
identity generator (same workaround as for data-mesher earlier).
VPS install flow: Hetzner-created Debian box, then `clan machines
install vps-relay --target-host root@<public-ipv4>` reinstalls to
NixOS; subsequent updates go over ZT.
Stage 4e, take 2. Point the clan-community input at our fork's branch
that sanitizes machine.name for data-mesher's file-name validator
(upstream PR: clan/clan-community#25). Revisit this pin once merged.
- flake.nix: clan-community.url → fork + fix branch
- flake-modules/clan.nix: re-adds meta.domain = "clan",
inventory.instances.data-mesher (sunken-ship bootstrap, both default),
inventory.instances.dm-pull-deploy (sunken-ship push, both default
action="switch"), and clanHostsModule that puts /etc/hosts entries
for <host>.clan → each machine's ZT IPv6 so libp2p multiaddr
resolution works without a clan-domain DNS server.
- Generator vars for data-mesher + dm-pull-deploy signing keys were
regenerated on sunken-ship (data-mesher isn't packaged for
aarch64-darwin, so clan vars generate runs on Linux).
Stage 4e-a of the clan migration. Set up signed-file gossip
(data-mesher, experimental, clan-core) and pull-based NixOS deploy
(dm-pull-deploy, experimental, clan-community) across both servers.
- sunken-ship is the data-mesher bootstrap node + dm-pull-deploy push
role; phantom-ship joins via /dns/sunken-ship.clan/tcp/7946/... — the
hostname resolves via /etc/hosts (clanHostsModule) to sunken-ship's
ZT IPv6 since we don't run a DNS server for the clan domain.
- Both machines run the dm-pull-deploy default role with
action="switch": they watch /var/lib/data-mesher/files/home/
dm_pull_deploy/target and nixos-rebuild switch against the pushed
git+…?rev=…&narHash=… flake ref on each change.
- Signing keys (shared + per-host status) generated via clan vars
generate, ran on sunken-ship because data-mesher isn't packaged for
aarch64-darwin.
The legacy dotfiles-rebuild timer stays installed as a fallback until
dm-pull-deploy is proven; a smart push timer on sunken-ship (calls
dm-send-deploy only when origin/main moves) comes next.
clan-cli's upload / build / copy steps each resolve the SSH target
independently. With `internet.host = "sunken-ship"` (bare hostname),
off-LAN / missing-mDNS cases broke \`clan machines update\` because the
mac couldn't resolve the hostname. Pin both the inventory internet
instance's host AND clan.core.networking.{target,build}Host to each
machine's stable ZT IPv6, so every update path works regardless of
LAN DNS state — and the mac reaches the servers the same way it does
for ssh sunken-ship-zt / phantom-ship-zt.
clan-cli preferred the zerotier networking export (priority 900, user
defaulted to root@) over our clan.core.networking.targetHost setting,
which broke \`clan machines update\` with "Host key verification failed"
against the ZT IPv6 address as root@. Declaring an inventory.instances
.internet instance with priority 2000 makes clan-cli prefer the LAN
hostname and explicit danny@ user, so updates go over the LAN (ZT
stays available for SSH aliases and service-level use).
clan-cli silently ignores the `?dir=` URL parameter when resolving a
flake source, so with the flake at nixos/flake.nix `clan machines
update` fails with "flake.nix does not exist". Move the flake tree up
so the repo root contains flake.nix, flake.lock, flake-modules/, lib/,
modules/, sops/, and vars/. Host-specific NixOS modules stay in
nixos/{hosts,home,fish.nix,neovim.nix,…}; flake-module paths updated
accordingly.
- dotfiles-rebuild flakeRef is now "${dotfilesDir}#<host>" (was
"${dotfilesDir}/nixos#<host>").
- CLAUDE.md build commands + clan section updated. nixupdate fish alias
updated. sunken-ship hostsfile comment updated.
- Existing /etc/dotfiles checkouts on the servers will pick up the new
layout on the next `dotfiles-rebuild` timer tick; the rebuild service
was pre-updated via rsync so its flakeRef matches before the pull.
Also includes 4b follow-through: zerotier identities are now live on
both servers (sunken-ship=d553a2de33 controller, phantom-ship=6c048abbdc
peer) and IPv6 ping across the ZT mesh works.
