shelfish was only listening on 127.0.0.1 — vps-relay's Caddy
couldn't reach it over the ZT mesh. Bind 0.0.0.0 and allow 8081
inbound on \`zt+\` interfaces (not the global firewall — same
pattern sunken-ship uses for bbbot).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Original commit added Caddy directly on phantom-ship and opened
ports 80/443 — that would have exposed the home connection's
public IP via DNS. Reverting that and using the existing relay
pattern instead: vps-relay (Hetzner) terminates public TLS and
reverse-proxies over ZeroTier to phantom-ship's ZT IPv6 on 8081.
phantom-ship now just runs shelfish.service bound to 127.0.0.1:8081;
it accepts connections only from the ZT mesh interface (since
caddy/firewall changes are gone, the only listeners are the
existing trusted-interface ones plus this loopback).
vps-relay gets a third virtualHost alongside navidrome and bbbot.
DNS: shelfish.dannydannydanny.me → 89.167.39.251 (vps-relay public IP),
NOT phantom-ship's home IP.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Caddy fronts 80/443 with auto-Let's-Encrypt; reverse-proxies
shelfish.dannydannydanny.me to the local shelfish service on
127.0.0.1:8081. ACME issues the cert once the subdomain A-records
to this host's static IP.
Shelfish service mirrors shipyard's pattern: nix-built python env,
SHIPYARD_BOT_TOKEN_FILE pointed at the existing secret, DB stored
outside the rsynced code dir at ~/.local/share/shelfish/ so deploys
don't clobber state.
Code itself is rsync'd from ~/python-projects/27_shelfish/ to
/home/danny/shelfish/ (same convention as shipyard).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an MCP server exposing read tools (list_inbox, search, read_email)
across three personal Gmail accounts using existing app passwords in
/etc/openclaw/. Wired into claude-channels via --mcp-config. Slated for
replacement by an OAuth2 Gmail+Calendar server in path 2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
VPS public SSH: enable fail2ban with bantime-increment so brute-force
probers get evicted with exponential backoff (1h → 4h → 16h → 2.7d →
10.7d, capped at 30d). Default jail covers sshd; maxretry=5 in 10m.
server-debug-tools: htop, tcpdump, dnsutils, jq, curl. Imported by
sunken-ship + phantom-ship via flake.nixosModules.server-debug-tools.
These are the practical bits we'd otherwise pick up by enabling
clan.core.enableRecommendedDefaults — but the full clan defaults flip
systemd-networkd/resolved on, which broke dnsmasq + navidrome's resolv
.conf bind-mount on the homelab servers, so we cherry-pick instead.
Move the imperative SSH-key-related scars accumulated during the
clan/VPS rollout into nix config so future installs and rebuilds
reproduce the same state:
- danny@sunken-ship + danny@phantom-ship: trust the mac admin key
(id_ed25519_<host> on Daniel-Macbook-Air) and the host's own
self-loopback key (used by clan ssh-ng:// nix-copy-closure back
to the same host during `clan machines update`).
- root@sunken-ship + root@phantom-ship: trust the mac admin key so
`clan machines update` can run its SOPS-key upload step that
SSHes to root@<host> to write /var/lib/sops-nix/key.txt.
Existing key files (~/.ssh/id_ed25519 on each host) stay where they
are; the keypair was generated once during initial bootstrap and the
public side is now declared above. Reinstalls would regenerate and
need the pubkey re-pinned here.
Stage 4.5: declare a Hetzner-hosted reverse-proxy VPS as a clan machine.
- nixos/hosts/vps-relay.nix: Debian→NixOS cx23 in hel1. Caddy at public
80/443 reverse-proxies navidrome.dannydannydanny.me and
bbbot.dannydannydanny.me over ZT to sunken-ship.
- nixos/disko-cloud.nix: simple GPT + ext4 root, no LUKS — cloud provider
has physical disk anyway and there's no operator at boot.
- flake-modules/clan.nix: register vps-relay as an inventory machine,
zerotier peer, internet networking target at its clan-generated ZT
IPv6, and add vps-relay.clan to clanHostsModule /etc/hosts.
- sunken-ship fitness-bot: drop pkgs.cloudflared from PATH + set
WEBAPP_URL=https://bbbot.dannydannydanny.me. Paired with the bbbot
upstream patch (start.py honors env WEBAPP_URL and skips cloudflared
when set) — once the 15-min fitness-bot-pull timer pulls that change,
bbbot will stop churning trycloudflare.com URLs.
Vars (zerotier identity/ip + sops machine key) generated on sunken-ship
because clan's hermetic sandbox on macOS fails to run the zerotier
identity generator (same workaround as for data-mesher earlier).
VPS install flow: Hetzner-created Debian box, then `clan machines
install vps-relay --target-host root@<public-ipv4>` reinstalls to
NixOS; subsequent updates go over ZT.
Stage 4f cleanup. The flake moved from ~/dotfiles/nixos/ to ~/dotfiles/
in 88c5139; docs and install scripts hadn't been refreshed. Point all
rebuild / flake references at the new root:
- AGENTS.md, README.md, server-quickstart.md, docs/server-installer-usb.md,
docs/sunken-ship-wifi.md, nixos/readme.md — rebuild command paths.
- scripts/nixos-server-install.sh — auto-detect now looks for flake.nix
at repo root (was nixos/flake.nix).
- scripts/post-install-provision.sh — first-rebuild hint path.
`nixos/hosts/<host>-hardware.nix` and friends stay where they are —
host-specific NixOS modules still live under nixos/; only the flake
entry-points + sops/ + vars/ + lib/ + modules/ + flake-modules/ moved.
nixos/readme.md rewritten to reflect the split (flake at root, per-host
modules under nixos/).
Stage 4d of the clan migration. Navidrome is now reachable only over
the ZeroTier mesh (port 4533 on sunken-ship's ZT IPv6 address, or via
the sunken-ship-zt SSH alias). Dropped:
- systemd.services.cloudflare-tunnel
- clan.core.vars.generators.cloudflare-tunnel
- cloudflared from environment.systemPackages
- vars/per-machine/sunken-ship/cloudflare-tunnel/
Manual follow-ups still needed on sunken-ship:
- rm /home/danny/.secrets/cloudflare-tunnel-token (old unmanaged token)
- delete the tunnel itself in the Cloudflare Zero Trust dashboard
- unlink the DNS record music.dannydannydanny.me if it was separate
Home-manager now writes a drop-in at ~/.ssh/config.d/zerotier with
sunken-ship-zt and phantom-ship-zt aliases pointing at the ZT IPv6
addresses. Useful when off the LAN — the aliases route over the
ZeroTier mesh. Requires a one-time \`Include ~/.ssh/config.d/*\` at
the top of ~/.ssh/config.
Declare a clan.core.vars.generators.cloudflare-tunnel generator that
prompts for the tunnel token on first run and stores it SOPS-encrypted
under vars/per-machine/sunken-ship/cloudflare-tunnel/tunnel-token.
systemd.services.cloudflare-tunnel ExecStart now reads the decrypted
secret at runtime from \${config.clan.core.vars...path} (lives at
/run/secrets/vars/...) instead of the unmanaged
/home/danny/.secrets/cloudflare-tunnel-token file.
Stage 4c of the clan migration. The tunnel itself is slated for
retirement in 4d — ZeroTier-only access after that. Cloudflare token
was rotated during this migration; old value no longer valid.
clan-cli silently ignores the `?dir=` URL parameter when resolving a
flake source, so with the flake at nixos/flake.nix `clan machines
update` fails with "flake.nix does not exist". Move the flake tree up
so the repo root contains flake.nix, flake.lock, flake-modules/, lib/,
modules/, sops/, and vars/. Host-specific NixOS modules stay in
nixos/{hosts,home,fish.nix,neovim.nix,…}; flake-module paths updated
accordingly.
- dotfiles-rebuild flakeRef is now "${dotfilesDir}#<host>" (was
"${dotfilesDir}/nixos#<host>").
- CLAUDE.md build commands + clan section updated. nixupdate fish alias
updated. sunken-ship hostsfile comment updated.
- Existing /etc/dotfiles checkouts on the servers will pick up the new
layout on the next `dotfiles-rebuild` timer tick; the rebuild service
was pre-updated via rsync so its flakeRef matches before the pull.
Also includes 4b follow-through: zerotier identities are now live on
both servers (sunken-ship=d553a2de33 controller, phantom-ship=6c048abbdc
peer) and IPv6 ping across the ZT mesh works.
Stage 4b of the clan migration. Declares a clan.inventory.instances.zerotier
instance with sunken-ship as controller and phantom-ship as peer (controller
is also listed as a peer so it joins its own network). Generates the network
ID, controller identity, and per-peer identities via `clan vars generate`;
all secrets are SOPS-encrypted to the user's age key and the per-machine
age keys.
- nixos/sops/ — clan-managed SOPS state (user + per-machine age keys).
- nixos/vars/ — shared + per-machine zerotier vars; *-identity-secret
files are SOPS-encrypted, *.value files are plain public data.
- clan.core.networking.{targetHost,buildHost} = "danny@<host>" on both
servers so `clan machines update` knows where to push and build.
- mac gets `zerotier-one` installed as a homebrew cask; authorization
on the controller happens manually by node-ID in a follow-up step.
Known rough edges (to chase in later stages):
- zerotier-inventory-autoaccept.service races zerotierone.service on
first activation (connection refused against the local API). Retrying
the unit succeeds; clan upstream bug.
- Deployment must go through `clan machines update`, not plain
nixos-rebuild, or the per-host SOPS age key isn't uploaded and
zerotier-one can't decrypt its identity.
Stage 4a of the dendritic + clan migration. Both servers now live under
clan.machines (via nixos/flake-modules/clan.nix) and clan-core generates
their nixosConfigurations for us; the previous per-host flake-modules
are removed.
Notes:
- clan.core.enableRecommendedDefaults = false on both machines so we
keep the existing dhcpcd / non-networkd / non-resolved stack. Services
like dnsmasq, navidrome, and the existing wireless setup break with
the clan defaults on.
- dotfiles-rebuild timer is untouched (safety net). Replacing it with
clan machines update / dm-pull-deploy comes in 4e.
- mac stays outside the clan as admin only.
Verified: `clan machines list --flake path:…/nixos` returns both hosts;
both servers rebuild cleanly and all services (navidrome, cloudflare-
tunnel, fitness-bot, dnsmasq, openclaw-gateway, sshd) stay active.
Telegram bot hub that lists mini-apps and collects feedback via ForceReply.
Code deployed via rsync to /home/danny/shipyard/; token at
~danny/.secrets/telegram-bot-token-shipyard.
Claude Code Channels replaces OpenClaw for the @HarakatBot Telegram
bridge. Uses claude.ai subscription auth via long-lived OAuth token
at /etc/claude-channels/env — sidesteps the API rate limits OpenClaw
was hitting.
Runs as danny since plugin + pairing state lives in ~/.claude.
Wraps claude in script(1) because claude needs a PTY for its
interactive session mode.
OpenClaw service disabled but config kept for easy rollback during
validation. Will be fully removed once Channels is proven stable.
Her workspace (SOUL/MEMORY/IDENTITY/etc) is preserved in
vimwiki/openclaw/workspace/.
- Add import-tree input; flake.nix now auto-loads every file under
./flake-modules so new hosts/features drop in without editing flake.nix.
- Extract the duplicated dotfiles-rebuild service, timer, and
safe.directory wiring into nixos/modules/dotfiles-rebuild.nix, exposed
via flake.nixosModules.dotfiles-rebuild.
- sunken-ship and phantom-ship now pull it in from their flake-modules;
hostname-specific flakeRef is derived from config.networking.hostName.
The dotfiles-rebuild service runs as root, but /etc/dotfiles is owned
by `danny`. The GIT_CONFIG_* env vars in the service unit only affect
the git CLI — nix/libgit2 reads safe.directory from /etc/gitconfig.
After a recent nixpkgs bump libgit2 now enforces this strictly, so the
service was failing to evaluate the flake.
Enable programs.git and set programs.git.config.safe.directory =
[ dotfilesDir ] on both sunken-ship and phantom-ship so the trust is
persistent and Nix-managed.
Extract the per-host home-manager block (useGlobalPkgs, useUserPackages,
backupFileExtension, users.<name> with username/homeDirectory/optional
stateVersion/optional imports) into nixos/lib/home-manager-user.nix.
Each flake-module now imports it with its per-host parameters, removing
~40 lines of boilerplate across the four hosts.
- Convert flake.nix to flake-parts.lib.mkFlake; split each host into
its own module under nixos/flake-modules/.
- Replace zen-browser specialArgs plumbing with a nixpkgs overlay so
home.nix can just reference pkgs.zen-browser.
Claude Code Channels will replace OpenClaw for the Telegram bot.
Channels uses claude.ai subscription auth instead of pay-as-you-go
API, sidestepping the rate limits Hara has been hitting.
nixos-rebuild was failing with "fsType accessed but has no value
defined" on newer nixpkgs. Bind mounts need fsType=none explicitly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New fitness-bot-pull service + timer, modeled on dotfiles-rebuild.
Checks origin/main for new commits, pulls + restarts the service if
the HEAD moved. Offset by 7 min from dotfiles-rebuild to avoid
overlap.
Code now lives at github.com/DannyDannyDanny/bigbiggerbiggestbot
(cloned to /home/danny/tg_fitness_bot). workouts.db is gitignored
so it's preserved across pulls.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Exposes navidrome via music.dannydannydanny.me.
Bypasses CGNAT — no port forwarding needed.
Token stored outside repo at ~/.secrets/cloudflare-tunnel-token.
- programs.neovim.extraLuaConfig renamed to initLua in nixpkgs unstable
- uhk-agent is x86_64-linux only, removed from darwin home config;
macOS: download .dmg from ultimatehackingkeyboard.com
