General Configuration¶

The platform's core behavior, including LLM providers and infrastructure services, is controlled via environment variables.

LLM Provider¶

The platform supports multiple LLM providers through LiteLLM. Switch providers by setting two environment variables — no code changes needed.

Provider examples¶

MODEL_PROVIDER=gemini
MODEL_NAME=gemini-3.6-flash
# Either Vertex AI:
GOOGLE_GENAI_USE_ENTERPRISE=TRUE
GOOGLE_CLOUD_PROJECT=your-project-id
GOOGLE_CLOUD_LOCATION=us-central1
# Or AI Studio:
GOOGLE_GENAI_USE_ENTERPRISE=FALSE
GOOGLE_API_KEY=your-api-key

MODEL_PROVIDER=anthropic
MODEL_NAME=anthropic/claude-sonnet-4-20250514
ANTHROPIC_API_KEY=sk-ant-api03-...

MODEL_PROVIDER=openai
MODEL_NAME=openai/gpt-4o
OPENAI_API_KEY=sk-...

MODEL_PROVIDER=ollama
MODEL_NAME=ollama/llama3
OLLAMA_API_BASE=http://localhost:11434

Getting API keys¶

Model-call retry and fallback¶

Tool calls have had a circuit breaker and retries for a long time; the model call itself now has two layers of its own (AEP-021):

The two layers answer different failures. Retry absorbs a blip on one model. Fallback covers a model that stays unavailable for longer than the retries last: retrying the same model harder does nothing when that model's capacity is what is gone.

• What moves the chain on: 404 (a retired model does not come back), 408, 429, 5xx, timeouts and dropped connections. Never 400, which fails the same on every model, and never 401/403, which are a configuration error.
• Never mid-answer: once a model has started answering, the turn stays on that model.
• One family per chain: ADK formats tool-call history differently for Gemini and for LiteLLM-routed models, so a chain that mixes them is refused at startup rather than failing on its first failover.
• Cost of a dead primary: retries run inside each link, so with a chain configured, keep ORRERY_LLM_RETRY_ATTEMPTS modest (2–3).
• Observability: orrery_llm_failover_total{from_model,to_model} counts failovers, and each one logs a WARNING. Evals run against the primary only, so keep chains within one capability tier.

Billing labels (Vertex AI)¶

ADK already adds adk_agent_name, so billing splits per agent out of the box. These labels add the deployment dimensions. Values are normalised to GCP's label rules. The Gemini Developer API does not accept labels, and ADK strips them there.

# Gemini Pro, falling back to Flash when Pro is unavailable
MODEL_NAME=gemini-3.6-pro
MODEL_FALLBACK_CHAIN=gemini-3.6-flash
ORRERY_LLM_RETRY_ATTEMPTS=3

Session & Memory Persistence¶

Sessions (conversation state) and long-term memory (cross-session recall) share one store. The platform supports exactly two backends — in-memory (no DATABASE_URL) or PostgreSQL (DATABASE_URL set). SQLite is not supported. Multi-replica deployments require PostgreSQL.

Inspecting the store locally¶

make up starts pgAdmin on http://localhost:5050, alongside Postgres and unprofiled for the same reason kafka-ui is — the thing it inspects is always running. It opens straight into the browser tree with the connection pre-registered (infra/pgadmin/servers.json): no sign-in page, no master password. Enter the database password once on first connect (POSTGRES_PASSWORD, default agents_secret) and tick Save password; it persists in the pgadmin-data volume.

The tables worth knowing, all created by ADK:

It binds to 127.0.0.1 only, and the login page is disabled because it sits in front of a database whose password ships in .env.example — a second login to reach it would buy nothing. Both facts stop being true the moment this is exposed off-host, so don't: it is a local-development container.

Context Caching¶

Context caching reduces token usage and latency by caching static system instructions (agent descriptions, tool schemas, RBAC rules) across requests. This is only effective with Gemini models — when using Claude/OpenAI via LiteLLM, the config is accepted but has no effect.

Context caching is enabled by default in the orrery-assistant agent. You can tune the values via environment variables or disable it by not passing a context_cache_config to run_persistent().

Cache hit/miss events are exposed as the orrery_context_cache_events_total Prometheus counter on the /metrics endpoint.

Context Compaction¶

Caching shrinks what a request costs; compaction shrinks what it contains. Once a session's prompt grows past ORRERY_COMPACTION_TOKEN_THRESHOLD, ADK replaces the older turns with an LLM-written digest and keeps the most recent ORRERY_COMPACTION_RETENTION_EVENTS events verbatim.

Without it a long incident session grows monotonically until the request exceeds the model's window and the turn fails (400 INVALID_ARGUMENT on Gemini). ToolOutputCapPlugin caps a single tool result at 4 MiB, never the accumulated transcript — three capped results in history already approach the ~10 MiB request ceiling. Compaction is the only thing that bounds the conversation as a whole.

Compaction is lossy for the model but lossless for the record. ADK appends the digest as a new event carrying the compacted timestamp range; the original events stay in the session store and are filtered out only when assembling the request. Audit, replay, and GET /session/{id}/activity still see everything.

On by default across every transport. The 250k default is deliberately out of reach for ordinary sessions — enabling compaction should change nothing except for the long investigations it exists to rescue.

Notes

• The summarizer runs on a cheap model. Summarization is plumbing, not user-facing reasoning. Left unset, ADK would derive it from the root agent's own model and bill digests at that rate. On non-Gemini providers there is no cheap default we can assume exists, so ORRERY_COMPACTION_MODEL falls back to the agent's MODEL_NAME — set it explicitly to get the savings.
• The sliding-window backstop cannot be switched off. ADK requires compaction_interval/overlap_size whenever compaction is configured, so it always runs on invocations where the token threshold did not fire. The default interval is set high enough that the token trigger normally does the work.
• Compaction invalidates the cached prefix on the turn it fires, since the history it covers changed. Tune the threshold well below the model's hard limit but high enough that most sessions never compact, and watch the counter below against your cache hit rate.

Compactions are exposed as the orrery_context_compaction_total Prometheus counter on the /metrics endpoint.

Agents¶

Which specialists the coordinator can delegate to, and how to add your own. See Adding an agent.

Process¶

HTTP front door¶

A refused request is a 429 with Retry-After, which the console shows as "retry in Ns".

Daily token budgets¶

A rate limit bounds how often a turn starts, not what it costs: one message can fan out to every specialist, and each model call re-sends the run so far. Two daily ceilings bound the bill. Both are off by default, are counted per UTC day from each model response's token usage (prompt, output and thinking), and apply on every transport, since they are a plugin in default_plugins().

Once a budget is spent, model calls are answered with a short "usage limit reached, resets at 00:00 UTC" reply instead of being sent, so the refusal costs nothing. Each call reserves an estimate of its prompt before it goes out (characters / 4), so calls started at the same moment see each other, and the response's reported usage replaces the estimate afterwards. A budget can still be exceeded by the output of the calls already in flight and by the estimate's error. Counters use ORRERY_RATE_LIMIT_STORAGE_URI: per process with memory:// (they reset on restart, and with ORRERY_MULTI_REPLICA each replica counts its own, which startup warns about), shared with redis://. Refusals are counted in orrery_token_budget_refusals_total{scope="user"|"global"}.

The store is a cost control, not a security gate: if it fails or takes longer than 2 s, the call goes ahead unbudgeted, the error is logged, and orrery_token_budget_store_errors_total counts it. OrreryTokenBudgetStoreFailing (infra/alert_rules.yml) fires on it.

Who is charged:

• Console and A2A: the verified subject. Slack: the Slack user id. Google Chat: the verified email. An agent acting for a user, and every specialist a turn calls, spends that user's budget.
• With authentication off, the console keys on anonymous:<client IP>, which a caller can rotate and which everyone behind one proxy shares; only the deployment budget means much there.
• The scheduled triage sweep runs as batch, so it draws on the deployment budget like everyone else: once that is spent, a sweep's nodes are refused.

Not charged: the compaction summarizer, knowledge embeddings, the onboarding self-test probe (all called outside the plugin chain), and calls that fail or are cancelled before their final response.

A2A exposition (ORRERY_A2A_*) has its own table in A2A.

Chat attachments¶

Images and UTF-8 text files on /chat and /chat/stream (multipart), processed in memory for one turn and never stored (AEP-030).

A proxy in front must not spool bodies to disk: nginx proxy_request_buffering off, or on Ingress-NGINX nginx.ingress.kubernetes.io/proxy-request-buffering: "off" (the Helm chart's default ingress annotations set it, with proxy-body-size: 25m). For a 1 GiB container use MAX_CONCURRENT_DECODES=1, MAX_PIXELS=16000000, MAX_CONCURRENT_UPLOADS=2.

With attachments on, the server defaults ADK's ADK_CAPTURE_MESSAGE_CONTENT_IN_SPANS to false (so does configure_tracing), so prompts, uploaded text and tool results stay out of trace spans. Set it to true explicitly to opt back in.

Distributed Tracing¶

OpenTelemetry tracing follows a single request through the agent hierarchy so you can attribute latency and localize failures. It is an opt-in extra (uv sync --extra otel), off by default, and driven entirely by the env vars below — default_plugins() reads OTEL_TRACING_ENABLED automatically, so one flag turns it on across every transport.

Planning¶

The reasoning-heavy agents in orrery-assistant (the root orchestrator, triage_summarizer, and remediation_actor) accept an optional ADK planner that injects an explicit reasoning step before tool calls. Planning is opt-in and off by default — setting ORRERY_PLANNER is the only knob you need.

Choosing a planner:

• plan_react is provider-agnostic — works with every backend MODEL_PROVIDER supports (Gemini, Claude, OpenAI, Ollama via LiteLLM). The model output is structured into /*PLANNING*/, /*ACTION*/, /*REASONING*/, and /*FINAL_ANSWER*/ phases. Use this if you run on anything other than Gemini, or if you want plan steps you can surface in a UI (the Google Chat progress card already keys off state_delta writes, so the additional structure shows up naturally).
• builtin uses Gemini's native thinking tokens via BuiltInPlanner(ThinkingConfig(...)). Cheaper and lower-latency than plan_react on Gemini, with no output-shape change. Falls back to no planner (with a warning) when MODEL_PROVIDER != gemini, since LiteLLM-routed models do not consume the ADK thinking config.

Tradeoffs to be aware of:

• plan_react makes responses noticeably more verbose and adds an extra reasoning round-trip per turn — A/B-test before flipping it on for latency-sensitive surfaces.
• Planners are intentionally not attached to per-system health checkers, the remediation verifier, or the journal writer. Those agents execute one short tool sequence per turn; planning would add latency without changing the output.
• Reasoning never leaks to users. Whatever planner you choose, the model's reasoning (builtin thought tokens, plan_react /*PLANNING*///*REASONING*/ phases, and provider reasoning surfaced through LiteLLM) is marked as a thought part by ADK. Every user-facing transport — Google Chat, Slack, the HTTP /chat API, and the CLI — funnels event text through extract_reply_text() (orrery_core.serving.events), which drops thought parts and emits only the final answer. So ORRERY_PLANNER_INCLUDE_THOUGHTS=true makes thoughts available for tracing/progress without showing them in the reply.

Infrastructure¶

The included docker-compose.yml starts the local diagnostic stack.

make up starts every profile listed here; make up PROFILES=… narrows it.

Management Commands¶

make up                   # start all services
make down                 # stop all services
make reset                # stop and wipe volumes
make up PROFILES=tracing  # start the tracing stack (Tempo + Grafana)
make down # stop the tracing stack

Docker Compose profiles¶