General Configuration¶

The platform's core behavior, including LLM providers and infrastructure services, is controlled via environment variables.

LLM Provider¶

The platform supports multiple LLM providers through LiteLLM. Switch providers by setting two environment variables — no code changes needed.

Provider examples¶

MODEL_PROVIDER=gemini
MODEL_NAME=gemini-2.5-pro
# Either Vertex AI:
GOOGLE_GENAI_USE_VERTEXAI=TRUE
GOOGLE_CLOUD_PROJECT=your-project-id
GOOGLE_CLOUD_LOCATION=us-central1
# Or AI Studio:
GOOGLE_GENAI_USE_VERTEXAI=FALSE
GOOGLE_API_KEY=your-api-key

MODEL_PROVIDER=anthropic
MODEL_NAME=anthropic/claude-sonnet-4-20250514
ANTHROPIC_API_KEY=sk-ant-api03-...

MODEL_PROVIDER=openai
MODEL_NAME=openai/gpt-4o
OPENAI_API_KEY=sk-...

MODEL_PROVIDER=ollama
MODEL_NAME=ollama/llama3
OLLAMA_API_BASE=http://localhost:11434

Getting API keys¶

Session & Memory Persistence¶

Sessions (conversation state) and long-term memory (cross-session recall) share one store. The platform supports exactly two backends — in-memory (no DATABASE_URL) or PostgreSQL (DATABASE_URL set). SQLite is not supported. Multi-replica deployments require PostgreSQL.

Inspecting the store locally¶

make up starts pgAdmin on http://localhost:5050, alongside Postgres and unprofiled for the same reason kafka-ui is — the thing it inspects is always running. It opens straight into the browser tree with the connection pre-registered (infra/pgadmin/servers.json): no sign-in page, no master password. Enter the database password once on first connect (POSTGRES_PASSWORD, default agents_secret) and tick Save password; it persists in the pgadmin-data volume.

The tables worth knowing, all created by ADK:

It binds to 127.0.0.1 only, and the login page is disabled because it sits in front of a database whose password ships in .env.example — a second login to reach it would buy nothing. Both facts stop being true the moment this is exposed off-host, so don't: it is a local-development container.

Context Caching¶

Context caching reduces token usage and latency by caching static system instructions (agent descriptions, tool schemas, RBAC rules) across requests. This is only effective with Gemini models — when using Claude/OpenAI via LiteLLM, the config is accepted but has no effect.

Context caching is enabled by default in the orrery-assistant agent. You can tune the values via environment variables or disable it by not passing a context_cache_config to run_persistent().

Cache hit/miss events are exposed as the orrery_context_cache_events_total Prometheus counter on the /metrics endpoint.

Context Compaction¶

Caching shrinks what a request costs; compaction shrinks what it contains. Once a session's prompt grows past ORRERY_COMPACTION_TOKEN_THRESHOLD, ADK replaces the older turns with an LLM-written digest and keeps the most recent ORRERY_COMPACTION_RETENTION_EVENTS events verbatim.

Without it a long incident session grows monotonically until the request exceeds the model's window and the turn fails (400 INVALID_ARGUMENT on Gemini). ToolOutputCapPlugin caps a single tool result at 4 MiB, never the accumulated transcript — three capped results in history already approach the ~10 MiB request ceiling. Compaction is the only thing that bounds the conversation as a whole.

Compaction is lossy for the model but lossless for the record. ADK appends the digest as a new event carrying the compacted timestamp range; the original events stay in the session store and are filtered out only when assembling the request. Audit, replay, and GET /session/{id}/activity still see everything.

On by default across every transport. The 250k default is deliberately out of reach for ordinary sessions — enabling compaction should change nothing except for the long investigations it exists to rescue.

Notes

• The summarizer runs on a cheap model. Summarization is plumbing, not user-facing reasoning. Left unset, ADK would derive it from the root agent's own model and bill digests at that rate. On non-Gemini providers there is no cheap default we can assume exists, so ORRERY_COMPACTION_MODEL falls back to the agent's MODEL_NAME — set it explicitly to get the savings.
• The sliding-window backstop cannot be switched off. ADK requires compaction_interval/overlap_size whenever compaction is configured, so it always runs on invocations where the token threshold did not fire. The default interval is set high enough that the token trigger normally does the work.
• Compaction invalidates the cached prefix on the turn it fires, since the history it covers changed. Tune the threshold well below the model's hard limit but high enough that most sessions never compact, and watch the counter below against your cache hit rate.

Compactions are exposed as the orrery_context_compaction_total Prometheus counter on the /metrics endpoint.

Distributed Tracing¶

OpenTelemetry tracing follows a single request through the agent hierarchy so you can attribute latency and localize failures. It is an opt-in extra (uv sync --extra otel), off by default, and driven entirely by the env vars below — default_plugins() reads OTEL_TRACING_ENABLED automatically, so one flag turns it on across every transport.

Planning¶

The reasoning-heavy agents in orrery-assistant (the root orchestrator, triage_summarizer, and remediation_actor) accept an optional ADK planner that injects an explicit reasoning step before tool calls. Planning is opt-in and off by default — setting ORRERY_PLANNER is the only knob you need.

Choosing a planner:

• plan_react is provider-agnostic — works with every backend MODEL_PROVIDER supports (Gemini, Claude, OpenAI, Ollama via LiteLLM). The model output is structured into /*PLANNING*/, /*ACTION*/, /*REASONING*/, and /*FINAL_ANSWER*/ phases. Use this if you run on anything other than Gemini, or if you want plan steps you can surface in a UI (the Google Chat progress card already keys off state_delta writes, so the additional structure shows up naturally).
• builtin uses Gemini's native thinking tokens via BuiltInPlanner(ThinkingConfig(...)). Cheaper and lower-latency than plan_react on Gemini, with no output-shape change. Falls back to no planner (with a warning) when MODEL_PROVIDER != gemini, since LiteLLM-routed models do not consume the ADK thinking config.

Tradeoffs to be aware of:

• plan_react makes responses noticeably more verbose and adds an extra reasoning round-trip per turn — A/B-test before flipping it on for latency-sensitive surfaces.
• Planners are intentionally not attached to per-system health checkers, the remediation verifier, or the journal writer. Those agents execute one short tool sequence per turn; planning would add latency without changing the output.
• Reasoning never leaks to users. Whatever planner you choose, the model's reasoning (builtin thought tokens, plan_react /*PLANNING*///*REASONING*/ phases, and provider reasoning surfaced through LiteLLM) is marked as a thought part by ADK. Every user-facing transport — Google Chat, Slack, the HTTP /chat API, and the CLI — funnels event text through extract_reply_text() (orrery_core.serving.events), which drops thought parts and emits only the final answer. So ORRERY_PLANNER_INCLUDE_THOUGHTS=true makes thoughts available for tracing/progress without showing them in the reply.

Infrastructure¶

The included docker-compose.yml starts the local diagnostic stack.

Management Commands¶

make up                   # start all services
make down                 # stop all services
make reset                # stop and wipe volumes
make up PROFILES=tracing  # start the tracing stack (Tempo + Grafana)
make down # stop the tracing stack

Docker Compose profiles¶