mulbo-server's /folders endpoint reads navidrome.db directly because
the Subsonic API's path field is tag-virtual (not real fs paths).
Three pieces:
- services.navidrome UMask = 0027 (force) so future DB writes are
group-readable; default was 0077.
- tmpfiles z-rules to chmod 0640 the existing navidrome.db, -wal, -shm
(created under the old umask).
- mulbo-server gets SupplementaryGroups=[navidrome] so the unit's
process can read those files.
Trade-off: couples mulbo-server to Navidrome's schema (specifically
media_file.id + media_file.path). Acceptable given Navidrome 0.61.1
has been stable on these columns; we'll catch breakage at the /health
navidrome_db_readable probe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds:
- MULBO_MUSIC_ROOT=/srv/music (for the /folders fs walk)
- EnvironmentFile=/home/danny/.secrets/mulbo-server-navidrome (creds
for Subsonic API calls — file is mode 600, owned by danny, not in
source control)
Required for the new /folders endpoint and the upcoming POST /tracks
which needs to call search3.view + startScan.view.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 5 of the mulbo Navidrome-pivot — companion HTTP service co-
located with Navidrome that owns uploads + the dedup index + the
real on-disk folder layout (which Navidrome's tag-virtual API can't
expose). Wire spec lives in the mulbo repo at 20_mulbo/SERVER_API.md.
Runs as `danny` so writes pass through to /home/danny/music/mulbo-
uploads via the existing /srv/music ro bind-mount — no mount changes
needed. Bound to [::]:8091 (8090 was taken by escape-hormuz on
phantom-ship); firewall scopes it to the ZT mesh, same trick bbbot
uses on 8080.
Pulls the python-projects repo via SSH using sunken-ship's id_ed25519
(registered as a read-only deploy key on the repo). Auto-pull timer
runs every 15 min, offset from fitness-bot-pull and dotfiles-rebuild.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hara (openclaw) shipped escape_hormuz imperatively — service runs but
firewall + Caddy vhost weren't declared, so the public URL didn't
resolve and the firewall rule would've been wiped on next
dotfiles-rebuild. Bring it under nix:
phantom-ship.nix
- systemd.services.escape-hormuz on port 8090, binds :: for ZT
- 8090 added to zt+ allowedTCPPorts
- tmpfiles entry for /home/danny/.local/share/escape_hormuz
vps-relay.nix
- Caddy vhost escapehormuz.dannydannydanny.me → ZT [::]:8090
Phase 1 of the de-platform-from-GitHub roadmap (vimwiki/diary/2026-05-03.md).
- phantom-ship: services.forgejo bound to 0.0.0.0:3000, sqlite, lfs on,
registration disabled, sign-in required.
- phantom-ship: add :3000 to the existing zt+ allowedTCPPorts list
(joins shelfish/scuttle — never exposed on WAN/Wi-Fi).
- vps-relay: Caddy vhost git.dannydannydanny.me reverse-proxies over
ZT to phantom-ship:3000.
Manual steps before reachable:
1. GoDaddy A record git.dannydannydanny.me -> 89.167.39.251
2. clan machines update phantom-ship && clan machines update vps-relay
3. On phantom-ship: bootstrap admin (registration is disabled)
KomTolk is the rebranded translate-platform — same Copenhagen
translation gigs Mini App, new name. Service on port 8080, mirrors
shelfish/scuttle/banana setup. New tmpfiles dir + zt+ firewall
opening + caddy vhost.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bananasimulator.service mirrors shelfish/scuttle (fastapi + uvicorn
+ httpx + python-telegram-bot). Port 8083. ENV BS_RIPE_MIN_PER_STAGE=2
in prod (30 min total banana lifetime); preview uses 0.5 for fast
testing.
vps-relay gets a fifth vhost (bananasimulator.dannydannydanny.me)
reverse-proxying to phantom-ship over ZeroTier. The shipyard manifest
has been pointing at this URL as a placeholder since day one — now
it's actually live.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scuttle.service mirrors shelfish — fastapi/uvicorn/httpx/python-telegram-bot
plus websockets, runs uvicorn --host :: --port 8082, DB at
~/.local/share/scuttle/scuttle.db (tmpfiles rule + zt+ firewall port
added alongside shelfish's).
vps-relay gets a fourth virtualHost (scuttle.dannydannydanny.me)
reverse-proxying to phantom-ship over ZeroTier. WebSocket upgrade is
transparent under Caddy's reverse_proxy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds two write-capable IMAP tools:
- gmail_mark_read: sets \Seen flag on a message
- gmail_archive: copies to [Gmail]/All Mail and removes from INBOX
The IMAP connection already used SELECT (read-write mode); this just
exposes the mutation surface through MCP.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
shelfish was only listening on 127.0.0.1 — vps-relay's Caddy
couldn't reach it over the ZT mesh. Bind 0.0.0.0 and allow 8081
inbound on \`zt+\` interfaces (not the global firewall — same
pattern sunken-ship uses for bbbot).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Original commit added Caddy directly on phantom-ship and opened
ports 80/443 — that would have exposed the home connection's
public IP via DNS. Reverting that and using the existing relay
pattern instead: vps-relay (Hetzner) terminates public TLS and
reverse-proxies over ZeroTier to phantom-ship's ZT IPv6 on 8081.
phantom-ship now just runs shelfish.service bound to 127.0.0.1:8081;
it accepts connections only from the ZT mesh interface (since
caddy/firewall changes are gone, the only listeners are the
existing trusted-interface ones plus this loopback).
vps-relay gets a third virtualHost alongside navidrome and bbbot.
DNS: shelfish.dannydannydanny.me → 89.167.39.251 (vps-relay public IP),
NOT phantom-ship's home IP.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Caddy fronts 80/443 with auto-Let's-Encrypt; reverse-proxies
shelfish.dannydannydanny.me to the local shelfish service on
127.0.0.1:8081. ACME issues the cert once the subdomain A-records
to this host's static IP.
Shelfish service mirrors shipyard's pattern: nix-built python env,
SHIPYARD_BOT_TOKEN_FILE pointed at the existing secret, DB stored
outside the rsynced code dir at ~/.local/share/shelfish/ so deploys
don't clobber state.
Code itself is rsync'd from ~/python-projects/27_shelfish/ to
/home/danny/shelfish/ (same convention as shipyard).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an MCP server exposing read tools (list_inbox, search, read_email)
across three personal Gmail accounts using existing app passwords in
/etc/openclaw/. Wired into claude-channels via --mcp-config. Slated for
replacement by an OAuth2 Gmail+Calendar server in path 2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
VPS public SSH: enable fail2ban with bantime-increment so brute-force
probers get evicted with exponential backoff (1h → 4h → 16h → 2.7d →
10.7d, capped at 30d). Default jail covers sshd; maxretry=5 in 10m.
server-debug-tools: htop, tcpdump, dnsutils, jq, curl. Imported by
sunken-ship + phantom-ship via flake.nixosModules.server-debug-tools.
These are the practical bits we'd otherwise pick up by enabling
clan.core.enableRecommendedDefaults — but the full clan defaults flip
systemd-networkd/resolved on, which broke dnsmasq + navidrome's resolv
.conf bind-mount on the homelab servers, so we cherry-pick instead.
Move the imperative SSH-key-related scars accumulated during the
clan/VPS rollout into nix config so future installs and rebuilds
reproduce the same state:
- danny@sunken-ship + danny@phantom-ship: trust the mac admin key
(id_ed25519_<host> on Daniel-Macbook-Air) and the host's own
self-loopback key (used by clan ssh-ng:// nix-copy-closure back
to the same host during `clan machines update`).
- root@sunken-ship + root@phantom-ship: trust the mac admin key so
`clan machines update` can run its SOPS-key upload step that
SSHes to root@<host> to write /var/lib/sops-nix/key.txt.
Existing key files (~/.ssh/id_ed25519 on each host) stay where they
are; the keypair was generated once during initial bootstrap and the
public side is now declared above. Reinstalls would regenerate and
need the pubkey re-pinned here.
Stage 4.5: declare a Hetzner-hosted reverse-proxy VPS as a clan machine.
- nixos/hosts/vps-relay.nix: Debian→NixOS cx23 in hel1. Caddy at public
80/443 reverse-proxies navidrome.dannydannydanny.me and
bbbot.dannydannydanny.me over ZT to sunken-ship.
- nixos/disko-cloud.nix: simple GPT + ext4 root, no LUKS — cloud provider
has physical disk anyway and there's no operator at boot.
- flake-modules/clan.nix: register vps-relay as an inventory machine,
zerotier peer, internet networking target at its clan-generated ZT
IPv6, and add vps-relay.clan to clanHostsModule /etc/hosts.
- sunken-ship fitness-bot: drop pkgs.cloudflared from PATH + set
WEBAPP_URL=https://bbbot.dannydannydanny.me. Paired with the bbbot
upstream patch (start.py honors env WEBAPP_URL and skips cloudflared
when set) — once the 15-min fitness-bot-pull timer pulls that change,
bbbot will stop churning trycloudflare.com URLs.
Vars (zerotier identity/ip + sops machine key) generated on sunken-ship
because clan's hermetic sandbox on macOS fails to run the zerotier
identity generator (same workaround as for data-mesher earlier).
VPS install flow: Hetzner-created Debian box, then `clan machines
install vps-relay --target-host root@<public-ipv4>` reinstalls to
NixOS; subsequent updates go over ZT.
Stage 4f cleanup. The flake moved from ~/dotfiles/nixos/ to ~/dotfiles/
in 88c5139; docs and install scripts hadn't been refreshed. Point all
rebuild / flake references at the new root:
- AGENTS.md, README.md, server-quickstart.md, docs/server-installer-usb.md,
docs/sunken-ship-wifi.md, nixos/readme.md — rebuild command paths.
- scripts/nixos-server-install.sh — auto-detect now looks for flake.nix
at repo root (was nixos/flake.nix).
- scripts/post-install-provision.sh — first-rebuild hint path.
`nixos/hosts/<host>-hardware.nix` and friends stay where they are —
host-specific NixOS modules still live under nixos/; only the flake
entry-points + sops/ + vars/ + lib/ + modules/ + flake-modules/ moved.
nixos/readme.md rewritten to reflect the split (flake at root, per-host
modules under nixos/).
Stage 4d of the clan migration. Navidrome is now reachable only over
the ZeroTier mesh (port 4533 on sunken-ship's ZT IPv6 address, or via
the sunken-ship-zt SSH alias). Dropped:
- systemd.services.cloudflare-tunnel
- clan.core.vars.generators.cloudflare-tunnel
- cloudflared from environment.systemPackages
- vars/per-machine/sunken-ship/cloudflare-tunnel/
Manual follow-ups still needed on sunken-ship:
- rm /home/danny/.secrets/cloudflare-tunnel-token (old unmanaged token)
- delete the tunnel itself in the Cloudflare Zero Trust dashboard
- unlink the DNS record music.dannydannydanny.me if it was separate
Home-manager now writes a drop-in at ~/.ssh/config.d/zerotier with
sunken-ship-zt and phantom-ship-zt aliases pointing at the ZT IPv6
addresses. Useful when off the LAN — the aliases route over the
ZeroTier mesh. Requires a one-time \`Include ~/.ssh/config.d/*\` at
the top of ~/.ssh/config.
Declare a clan.core.vars.generators.cloudflare-tunnel generator that
prompts for the tunnel token on first run and stores it SOPS-encrypted
under vars/per-machine/sunken-ship/cloudflare-tunnel/tunnel-token.
systemd.services.cloudflare-tunnel ExecStart now reads the decrypted
secret at runtime from \${config.clan.core.vars...path} (lives at
/run/secrets/vars/...) instead of the unmanaged
/home/danny/.secrets/cloudflare-tunnel-token file.
Stage 4c of the clan migration. The tunnel itself is slated for
retirement in 4d — ZeroTier-only access after that. Cloudflare token
was rotated during this migration; old value no longer valid.
clan-cli silently ignores the `?dir=` URL parameter when resolving a
flake source, so with the flake at nixos/flake.nix `clan machines
update` fails with "flake.nix does not exist". Move the flake tree up
so the repo root contains flake.nix, flake.lock, flake-modules/, lib/,
modules/, sops/, and vars/. Host-specific NixOS modules stay in
nixos/{hosts,home,fish.nix,neovim.nix,…}; flake-module paths updated
accordingly.
- dotfiles-rebuild flakeRef is now "${dotfilesDir}#<host>" (was
"${dotfilesDir}/nixos#<host>").
- CLAUDE.md build commands + clan section updated. nixupdate fish alias
updated. sunken-ship hostsfile comment updated.
- Existing /etc/dotfiles checkouts on the servers will pick up the new
layout on the next `dotfiles-rebuild` timer tick; the rebuild service
was pre-updated via rsync so its flakeRef matches before the pull.
Also includes 4b follow-through: zerotier identities are now live on
both servers (sunken-ship=d553a2de33 controller, phantom-ship=6c048abbdc
peer) and IPv6 ping across the ZT mesh works.
Stage 4b of the clan migration. Declares a clan.inventory.instances.zerotier
instance with sunken-ship as controller and phantom-ship as peer (controller
is also listed as a peer so it joins its own network). Generates the network
ID, controller identity, and per-peer identities via `clan vars generate`;
all secrets are SOPS-encrypted to the user's age key and the per-machine
age keys.
- nixos/sops/ — clan-managed SOPS state (user + per-machine age keys).
- nixos/vars/ — shared + per-machine zerotier vars; *-identity-secret
files are SOPS-encrypted, *.value files are plain public data.
- clan.core.networking.{targetHost,buildHost} = "danny@<host>" on both
servers so `clan machines update` knows where to push and build.
- mac gets `zerotier-one` installed as a homebrew cask; authorization
on the controller happens manually by node-ID in a follow-up step.
Known rough edges (to chase in later stages):
- zerotier-inventory-autoaccept.service races zerotierone.service on
first activation (connection refused against the local API). Retrying
the unit succeeds; clan upstream bug.
- Deployment must go through `clan machines update`, not plain
nixos-rebuild, or the per-host SOPS age key isn't uploaded and
zerotier-one can't decrypt its identity.
Stage 4a of the dendritic + clan migration. Both servers now live under
clan.machines (via nixos/flake-modules/clan.nix) and clan-core generates
their nixosConfigurations for us; the previous per-host flake-modules
are removed.
Notes:
- clan.core.enableRecommendedDefaults = false on both machines so we
keep the existing dhcpcd / non-networkd / non-resolved stack. Services
like dnsmasq, navidrome, and the existing wireless setup break with
the clan defaults on.
- dotfiles-rebuild timer is untouched (safety net). Replacing it with
clan machines update / dm-pull-deploy comes in 4e.
- mac stays outside the clan as admin only.
Verified: `clan machines list --flake path:…/nixos` returns both hosts;
both servers rebuild cleanly and all services (navidrome, cloudflare-
tunnel, fitness-bot, dnsmasq, openclaw-gateway, sshd) stay active.
Telegram bot hub that lists mini-apps and collects feedback via ForceReply.
Code deployed via rsync to /home/danny/shipyard/; token at
~danny/.secrets/telegram-bot-token-shipyard.
