Inference Fleet · atlas

How it's built

Hermes answers through per-host gateways over a Tailscale tailnet. Talos, straylight, and titan primary is cloud xAI grok-4.6. Straylight's cloud fallback is zai glm-5.3-flash; local fallback and delegation go to straylight llama.cpp :11434. Titan keeps local fallback, vision, and delegation on its own llama.cpp :8081. Small-model aux (approval, titles, web extract) goes to dixie:11434; Honcho compute goes through tether on dixie :11435. sift on orion is the fleet URL fetch and search: Tavily first, SearXNG fill-in; Brave LLM available but disabled. tether on dixie :11435 is the stream-converting proxy in front of that llama-server. Six hosts, two local routers, two clouds of compute.

Hosts 6 Structures 21 Decisions 7 Flows 7
Cloud APIs Connection Gateways Local inference Clients & surfaces Memory plane Other hosts flow
01

Decisions locked

the axes the fleet commits to
AxisDecisionADR
Primary modelCloud is the default; local models never carry the main path. Talos, straylight, and titan primary is xAI grok-4.6. Straylight cloud fallback is zai glm-5.3-flash.—
Local routerllama.cpp llama-server on straylight is the main-loop fallback/delegation endpoint. Aux routing (approval/titles/web_extract) is served by dixie:11434.adr/0001-local-router.md
TransportMattermost is the gateway transport and hosts the 'claude' reviewer user.—
MemoryHoncho runs on rift; every model role calls tether on dixie :11435 (upstream llama-server :11434) for gen, embed and dialectic.adr/0002-memory-placement.md
GPU ceilingStraylight (Strix Halo) holds models-max 3 under a ~104 GiB TTM cap. Titan holds models-max 2. Dixie holds two residents on 12 GB.adr/0003-gpu-ceiling.md
Fetchsift on orion is the fleet URL fetch (Firecrawl-compatible /v2/scrape and /v1/fetch) and search (native /v1/search and Firecrawl-compatible /v2/search). Search: Tavily first, SearXNG fill-in; Brave LLM available but disabled.—
Dixie proxytether on dixie :11435 is the stream-converting OpenAI proxy in front of llama-server :11434.—
02

Structures

the 21 things that make up the fleet
03

Representative flows

packet shapes the design implies

Payload shapes are what the design implies, not measured traffic. 7 flows walk the fleet.

04

Context

glossary, one line per noun