Changelog¶

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased¶

0.4.0 - 2026-09-11¶

The agent can now read what humans wrote. Until this release Orrery saw live infrastructure through ~70 tools and recalled its own past sessions, but had no access to a runbook, a postmortem or an ADR — every incident started from zero institutional knowledge. search_knowledge closes that, on the tool path where the injection screen, credential scrubbing, output cap and audit trail all apply, because a retrieved document is attacker-reachable text like any other tool result. Around it: the platform is itself on-callable (twelve runbooks, a runbook_url on every alert), the confirmation gate reports what it decides and who it refuses, conversation history lives in the session store instead of the browser, and CI is green again after two rounds of transitive-dependency advisories.

Minor rather than patch: a new tool (search_knowledge), three new HTTP endpoints (GET /sessions, GET|DELETE /session/{id}), new metrics and alert rules, a new configuration surface (ORRERY_KNOWLEDGE_*, KNOWLEDGE_*, EMBEDDING_*) and a new build-time action (make knowledge-sync). Nothing existing changed shape; a 0.3.x deployment upgrades with no config edits and knowledge retrieval stays off until ORRERY_KNOWLEDGE_BACKEND is set.

Added¶

• The agent can read what humans wrote (AEP-025, core/orrery_core/knowledge/). Orrery could read live infrastructure through ~70 tools and recall its own past sessions; it had no access to a runbook, a postmortem or an ADR, so every incident started from zero institutional knowledge. search_knowledge closes that. Scoped as two seams rather than one "RAG provider", because a managed vendor owns both halves of the problem and a self-hosted store owns neither: Vertex AI Search ingests through its own connectors and answers queries, while Elasticsearch and pgvector do nothing until you write both sides. KnowledgeSource (filesystem / git / Confluence → Document) and KnowledgeRetriever (query → Passage) are therefore independent, with KnowledgeIndex implemented only by backends we populate ourselves — a retrieve-only backend is declaring "my ingestion is somebody else's job", and make knowledge-sync skips it rather than failing. Adding Notion is one source; standardising on Azure AI Search is one retriever; neither touches an agent. The read end is a real tool, and that is the safety argument, not a detail. Subclassing ADK's BaseRetrievalTool puts the result on the after-tool chain, so SafetyScreenPlugin neutralizes injected spans, PIIRedactionPlugin scrubs pasted tokens out of postmortems, ToolOutputCapPlugin bounds a chatty retrieval and AuditPlugin records the query. This matters because a retrieved document is attacker-reachable text — a Confluence page or git-hosted runbook is editable by anyone with write access to the source, the same threat model as a pod annotation. ADK's VertexAiSearchTool would bypass all four: despite the name it is model built-in grounding, appending a types.Retrieval to the request config so the model retrieves server-side with no after_tool_callback and no audit entry. agents/orrery-assistant/tests/test_knowledge_wiring.py walks both roots and fails the build if a grounding tool is ever attached. Two backends behind resolve_retriever(), which mirrors resolve_model(). Elasticsearch shipped first because make up already starts the container and BM25 alone is a large improvement over the any-single-word match in DatabaseMemoryService — no image swap, no embedding provider, and it proved the seams before phase 2 paid for semantics. pgvector is deliberately hybrid: semantic search is weakest exactly where SRE queries are strongest — an exact identifier like CrashLoopBackOff or a consumer-group name — where a nearest-neighbour search returns three plausible pages that never mention it. Both rankings are computed and fused with Reciprocal Rank Fusion, which combines ranks rather than scores because cosine distance and ts_rank_cd share no scale and any weighted sum would be dominated by whichever has the larger range. Provenance is a required field on every Passage, with age_days and a stale flag surfaced in the result: an operator at 03:00 cannot otherwise tell a retrieved fact from a hallucination, and that is the failure mode that makes teams abandon retrieval. "No matches" and "backend unreachable" are deliberately different results — if they looked alike, a broken index would quietly become "we have no runbook for that". Misconfiguration fails fast at startup; a runtime outage does not, because knowledge is an augmentation and the platform diagnosed incidents without it before this existed. Indexing is a build-time action (make knowledge-sync), never lazy at request time — a first query that silently triggers a full Confluence crawl is an outage waiting for its moment. Sync is revision-driven and idempotent, prunes stale chunks after a shrinking edit (nine chunks becoming four would otherwise keep answering from five slices that no longer exist), and skips deletion-pruning entirely when a run had any error, because a source that failed halfway looks identical to one whose documents were all deleted. Retrieval is viewer-level and not ACL-aware, so the rule is stated rather than faked: index only what every viewer may read. ConfluenceSource refuses to auto-discover spaces — a constructor without an explicit space list raises, and a 403 fails loudly.
• On-call runbooks, and runbook_url on every alert (AEP-017, runbooks/). A platform that restarts pods, rolls back deployments and resets consumer-group offsets is itself on-callable; until now every incident meant re-deriving the architecture from source, which does not happen at 03:00. Twelve pages: a first-five-minutes checklist, an escalation policy with evidence-preservation commands, a template, and nine incident runbooks. The set is organised around a question that turned out to come first in nearly every scenario — is Orrery broken, or is Orrery correctly reporting that something else is? A circuit breaker opening because Kafka is genuinely down is the system working, and mistaking that for a fault is the most likely way to make an incident worse. It is a fixed section in the template rather than prose buried in each page. All fourteen Prometheus rules now carry a runbook_url annotation. Two planned alerts were not shipped as originally drafted: the availability alert's expression matched up{job=~"orrery.*"} when the scrape job is actually agents, and prompt-injection detection had no metric to alert on at all (see below).
• orrery_safety_screen_total{direction, source}. Prompt-injection screening was log-only: MetricsPlugin bounds the tool status label to four values to cap cardinality, so a BLOCKED result recorded as ok and no expression over orrery_tool_calls_total could find a detection. The only alert that could be written was one that never fires, which is worse than none because it reads as coverage. direction keeps the two halves apart, and summing them would be meaningless: direct means a user message was refused before it cost a token — someone is probing the agent; indirect means text inside a tool result was neutralized — attacker-reachable content is sitting in the monitored infrastructure. Indirect counts spans, not events, because two injected lines in one payload is a worse finding than one; direct counts one, because the screen stops at the first match and the run never learns how many others the message held.
• Confirmation lifecycle events (AEP-024, core/orrery_core/security/, observability/). The gate was already correct and entirely silent. A successful approval was only derivable — correlate two tool_attempt entries and know the invariant — and a refused one left no trace anywhere, even though a second person attempting to approve someone else's destructive action is the most interesting event the subsystem can produce. Four events on the audit stream via a shared audit_event() helper: confirmation_raised (emitted inside raise_pending(), so no transport can forget it), confirmation_decided, confirmation_refused and confirmation_expired (both store backends — an expired pending and one nobody saw were previously identical). Every event carries mode, closing the provenance gap where an auditor could not distinguish a human-approved production change from a model re-call on a dev surface. Four counters plus a decision-latency histogram, which skips observations with no known pending creation time rather than inventing a latency that would poison the data the TTL should be tuned from. No confirmation_id column was added: PendingConfirmation.action_id is already a uuid4 primary key on both backends, so a second identifier would be a redundant column that could drift. The gate's and chain became staged guards so a refusal can name why; ordering, short-circuiting and the single consume_pending() call are unchanged, and test_guardrails.py passes untouched — which was the requirement, since this is observability only.
• OrreryUnauthorizedApprovalAttempt (critical) and OrreryUnattributableApprovals (warning), with a runbook. The first is the only alert in the file about a person rather than a system. The second catches the quieter failure: a transport not stamping the turn's actor means nobody can approve anything and guarded remediation silently stops working.

• AEP-024, AEP-025 and AEP-026 written up in enhancements/. AEP-026 (experience capture / REX) remains proposed: mine the platform's own incident history into after-action documents, with ADK's BigQueryAgentAnalyticsPlugin as one optional sink behind an ExperienceStore seam rather than the system of record, deterministic clustering, and generated documents reaching the corpus only through a merged pull request — without that gate the loop closes on itself and model error becomes indexed institutional fact.

• pgAdmin in the local compose stack (docker-compose.yml, infra/pgadmin/servers.json). make up now serves it on 127.0.0.1:5050 for browsing the session store — unprofiled alongside Postgres for the same reason kafka-ui is unprofiled alongside Kafka. Opens directly into the browser tree: desktop mode disables both the sign-in page and the master-password prompt, and the connection is pre-registered. The database password is entered once and saved into the volume — pgAdmin only accepts a saved-password file at mode 0600 owned by its own uid, which a host bind mount cannot provide. PGADMIN_CONFIG_ALLOW_SPECIAL_EMAIL_DOMAINS is set because the default account's .local address is a reserved TLD that the image's email validation rejects, restart-looping the container.

• Conversation history is server-backed (core/orrery_core/serving/server.py, web/). Three endpoints — GET /sessions (the caller's conversations, newest first, titles only), GET /session/{id} (one conversation with its transcript) and DELETE /session/{id} — and the console now reads its sidebar from the session store instead of localStorage. The transcripts were already in Postgres: create_app builds the gateway with DatabaseSessionService whenever DATABASE_URL is set. What was missing was any way to read them back, so the browser-local sidebar was the only index of which sessions belonged to whom — a different machine, or cleared site data, left every past conversation stranded in the store as unreachable rows. Two further consequences of that split are also gone: the per-conversation 200-message persistence cap (which made the visible transcript disagree with what the agent still remembered) and the 50-conversation eviction (which hid sessions that still existed). The cap now lives on the response, where the data is. Transcripts are rebuilt by build_transcript() in serving/events.py, which keeps only what a person said and what the agent said back — function calls, planner thoughts and compaction digests are dropped, and each turn's text events are merged the way POST /chat concatenates them, so a reopened conversation reads as it did live. The compaction check is explicit rather than incidental: an AEP-020 digest is authored "user" and keeps its content in actions.compaction, so a text-only filter happens to skip it today, but nothing would stop a future ADK from replaying an LLM summary of the conversation into the conversation as something the user said. Titles are written into session state on a conversation's first turn rather than derived from the transcript, because list_sessions returns state but never events — deriving them would cost one query per listed session on every load. Sessions opened by another transport carry none and render as "New chat". Every endpoint pins user_id to the verified subject, so another user's session id is a plain 404, and DELETE is checked before it is executed. Transcripts loading lazily means opening the console costs one request regardless of how much history exists. CORS allow_methods gained DELETE — without it the delete worked same-origin and failed only in cross-origin dev.

Changed¶

• Google ADK 2.5.0 → 2.8.0, alongside two dependency sweeps (litellm 1.98.0, uvicorn 0.52.4, pydantic 2.13.5, pydantic-settings 2.15.0, google-auth 2.57.1, ty 0.0.75, ruff 0.16.6, hypothesis 6.168.0; react 19.2.8, vite 8.2.2, vitest 5, typescript-eslint 8.70.0, eslint 10.8.1 and the testing/globals groups; node 26 base digest; actions/setup-uv 10.0.1). PyArrow left the gcp extra for the new bigquery-analytics extra upstream, which is unused here and cuts ~50 MB from the install. google-genai moved 2.11.0 → 2.19.0 transitively. core/pyproject.toml had floored google-adk[eval,db]>=2.5.0 while the workspace root moved to >=2.7.1. The lock resolved 2.7.1 either way, but orrery-core is published as its own distribution and a consumer installing it outside this workspace would have been allowed an ADK two minors behind what the code targets.
• docker-compose.yml uses pgvector/pgvector:pg16 in place of postgres:16-alpine. It is the stock Postgres image plus the vector extension — same data directory, same defaults — so an existing volume keeps working and sessions, memory and confirmations are unaffected. The Helm chart takes an external DATABASE_URL and needed no change.
• AgentGateway.session_service is optional in the type, not just at runtime. __init__ always assigns a service while from_runner() may assign None, so the attribute inferred as non-optional and the two constructors disagreed; the old # type: ignore[assignment] silenced the report rather than the mismatch. Declaring it BaseSessionService | None then surfaced seven call sites in the HTTP server and Slack handler dereferencing an optional. They now narrow through a sessions property that raises with a message naming run_in_session(), instead of an AttributeError three frames down. Found by ty 0.0.73, which flags what 0.0.63 let through.
• JSONFormatter's field allowlist covers the confirmation lifecycle. Extended rather than replaced with a sweep over record.__dict__ — logging puts a lot of machinery on a record and a caller can attach anything through extra, so an unfiltered merge would leak both into the log stream and make the output shape unstable.

Fixed¶

• CI had been red on main for over two weeks — every run since 2026-08-06, and every open Dependabot PR with it. The Security Scan job's Trivy filesystem scan runs with severity: HIGH,CRITICAL / exit-code: 1 / ignore-unfixed: true, so a fixed HIGH advisory is a hard failure by design; cryptography was pinned at 49.0.0 with CVE-2026-69247 against it. Clearing that exposed three more HIGH advisories against sqlparse 0.5.5 that had landed in Trivy's database since. Both are transitive, so both were lock-only upgrades (cryptography 50.0.0, sqlparse 0.6.0).
• CI red again on main and on all eleven open Dependabot PRs, same shape. nltk 3.10.0 (transitive via google-adk[eval] → rouge-score) picked up one CRITICAL and three HIGH advisories (CVE-2026-79675, CVE-2026-71513, CVE-2026-72818, CVE-2026-78680), fixed upstream in 3.10.3 — a three-line uv lock --upgrade-package nltk. The bump then tripped Dependency Review on a newer advisory, GHSA-8mgp-746c-j5xp, which covers every published nltk release (<= 3.10.3) with no patched version yet. Trivy already skips it via ignore-unfixed; dependency-review-action has no equivalent, so without an allow the lockfile could never again be touched where nltk is concerned. The one GHSA is allow-listed in ci.yml with an inline justification (nltk is only used for eval scoring; the affected model-artifact APIs are never called) and a re-check pointer — the entry is to be deleted the moment a patched release exists. A serial rebase-and-merge then landed the Dependabot backlog: the branch ruleset requires an up-to-date branch, so each merge knocked the rest BEHIND and they went one at a time.
• vite.config.ts imports ./src/api/paths.ts with its extension. Vite 8.2.1 warns that the extensionless form is unsupported by configLoader: 'native', planned to become the default in a future major — the native loader resolves the config through Node, which has no extensionless resolution. Both tsconfigs already set allowImportingTsExtensions, so the explicit form type-checks unchanged.
• The pgvector backend validates its table identifier. A table name cannot be a bound parameter — SQL binds values, not identifiers — so KNOWLEDGE_PG_TABLE is interpolated into every statement. It comes from configuration rather than a request, but "config is trusted" is exactly the assumption that ages badly once values arrive from a Helm chart, a ConfigMap or an operator CR. Validated once at construction against a plain-identifier pattern, which makes the interpolation provably safe rather than conventionally safe.

Security¶

• .trivyignore.yaml — triaged findings, with mandatory expiry dates. The Release & Publish workflow's image scan had been failing on every push since 2026-08-06, independently of CI: eight HIGH advisories against the Go standard library (v1.26.5) linked into usr/local/bin/docker, the Docker CLI bundled so the docker-agent can inspect local containers. There is no upgrade to take. Every published Docker CLI tag — 29-cli, cli, latest — is still built with Go 1.26.5, verified by scanning each one rather than assumed, so re-pinning the digest changes nothing. Removing the CLI would disable the docker-agent in-container, and rebuilding it from patched Go is disproportionate for a binary invoked as a subprocess against a local socket. All eight are denial-of-service or content-handling issues in a client that serves no HTTP, renders no templates to a browser and parses no untrusted DNS; the worst realistic outcome is the subprocess crashing, which the tool layer already handles as a tool error. Every entry carries expired_at (2026-10-21) and a statement saying why. The expiry is the review mechanism and it is self-enforcing — Trivy stops honouring an expired entry and the gate goes red again, so an ignore cannot silently become permanent, and a permanent ignore is indistinguishable from not scanning. Wired through the trivyignores input on all three Trivy steps, because Trivy auto-loads a plain .trivyignore but not the YAML form, and only the YAML form supports expiry.

0.3.1 - 2026-07-26¶

A security release. Two external reviews and one internal audit found four ways a guarded action or a credential could slip past a gate that was supposed to hold — all four are closed here, each with a test that fails if it comes back. Nothing in the platform's API changed; the dependency majors below are build-time only and invisible to anyone consuming the image or the library.

Security¶

• The closed-loop remediation actor ran mutating tools unattended (agents/orrery-assistant/orrery_assistant/remediation.py). remediation_actor was the only agent in the tree holding @confirm/@destructive tools without before_tool_callback=require_confirmation(), and GuardrailsPlugin enforces RBAC only — the human-in-the-loop gate is per-agent wiring. run_triage.py pins the batch session to operator, which RBAC lets past @confirm tools, so scale_deployment had nothing standing in its way: a scheduled overnight sweep could rescale a deployment the model chose, with no human in the conversation. restart_deployment and rollback_deployment were refused, but by role, not by the mechanism run_triage.py's own docstring credited. The actor is now gated like every other specialist, and its instruction covers the blocked outcome so a confirmation_required is reported as waiting, never as done. The cause was the invariant test, not the missing line. tests/test_confirmation_wiring.py — written after the docker specialist regressed identically — walked only orrery_chat_agent, and the remediation actor lives in the orrery_triage_workflow graph, which no AgentTool edge reaches. It now walks both roots (handling Workflow.graph.nodes) and carries a guard-the-guard test, because a walker that reaches nothing passes vacuously.
• Indirect prompt injection: tool results are now screened (core/orrery_core/plugins/safety_plugin.py). SafetyScreenPlugin screened only the user's message, which is the less important half for an infrastructure agent: a pod annotation, container log line, Kubernetes event, Kafka topic config or Elasticsearch document is attacker-reachable text that reaches the model wearing a tool result's authority. Matched spans in tool output are now neutralized in place rather than the payload being dropped — that payload is the evidence the agent was asked to read, so refusing it would break the diagnosis. Same in-place/return-None contract as PII redaction (a returned copy early-exits ADK's after-tool chain and silences every later observer), and the same worker-thread hop above OFFLOAD_THRESHOLD_CHARS. ORRERY_SAFETY_SCREEN=false disables both directions.
• Tool results that are not dicts escaped both content plugins (core/orrery_core/plugins/pii_plugin.py, core/orrery_core/payload.py, core/orrery_core/persistence/memory.py). Flagged by an external audit as a hypothetical; it was live. ADK does not require a dict — FunctionTool.run_async returns the function's value verbatim and the {"result": …} normalization happens in __build_response_event, after the after-tool chain — so a dict/list-only walk skipped whatever it was handed. Of the 120 registered tool functions, 119 return dicts; the exception is load_memory, wired into the shipped chat root, returning a Pydantic LoadMemoryResponse. The write side did not compensate: SecureMemoryService carried its own shorter pattern list (key=value pairs and PEM blocks) while the tool path also caught bare provider tokens, so a ghp_/AKIA/JWT pasted into chat was stored verbatim. Three gaps in a line — paste a token, it is stored unredacted, load_memory recalls it, neither plugin touches it, and it lands in the model context and the audit log. Both walks now traverse object attributes (payload.mutable_attributes), so a Pydantic result is scrubbed in place with the chain intact; a bare str/bytes cannot be mutated at all, so it is scrubbed by returning the replacement, logged at warning level because that early-exits the remaining observers (losing one audit outcome line beats writing a credential into it). security/redaction.py now holds the one pattern set both paths share, with a test asserting the parity structurally.
• The autonomy level could be promoted from session state (core/orrery_core/plugins/autonomy_plugin.py). _active_level() honoured any autonomy_level it found in state, and session state is not a trust boundary — tools write to it, and tool output is attacker-reachable text. An L2 read-only deployment was therefore one state write away from L4. Now honoured only when written through the new set_autonomy_level(), which stamps _autonomy_set_by_server; an unlocked value is ignored with a warning. This is the defence RBAC already had via _role_set_by_server — the two axes now match.
• /docs is off by default wherever auth is on (core/orrery_core/serving/server.py). A browser navigation cannot carry a bearer token, so the interactive schema cannot be gated behind the API's own auth, and it enumerates every route and request shape. ORRERY_DOCS_ENABLED overrides in either direction; local no-auth runs still get it.

Added¶

• make lock-check gates lockfile drift, in CI and locally. Every CI job opens with uv sync --all-extras, which silently rewrites a stale lock in the runner — so drift never failed a build. The check runs uv lock --check before the sync in the lint job, the only position where it can still see the problem, and make lock regenerates after a manifest edit.
• .github/workflows/dependabot-relock.yml commits the missing half of a Python update back onto Dependabot's branch, so the new gate doesn't turn every dependency bump into manual work. Prefers a DEPENDABOT_RELOCK_TOKEN Dependabot secret (an Actions secret is unreadable in a dependabot-triggered run) and warns explicitly when it is absent, because a push made with the default GITHUB_TOKEN does not re-trigger workflows.
• Tests that compose the real plugin chain, rather than one plugin at a time: both content defenses applied to a single payload with nothing cutting the chain short, the output cap proven to be the only plugin permitted to early-exit, and a denied call proven to be audited because audit precedes the gates. Every prior plugin test checked a plugin in isolation or the list of names, which is how an ordering violation could ship.

Changed¶

• Python updates move to Dependabot's uv ecosystem (.github/dependabot.yml). The pip ecosystem only edits pyproject.toml, so every Python update landed with uv.lock still recording the old constraint — pytest-cov merged green that way. The uv ecosystem understands the lock; make lock-check remains the backstop, since it can still leave one file behind.
• npm updates are grouped by what npm's peer ranges force to move together — react, vite, typescript, testing. Three open Dependabot PRs had failed at npm ci with ERESOLVE, and not one was a bad update: each was correct but proposed alone when its peer required a companion to move with it.
• TypeScript is held at 5.x. TS 7 cannot be installed here: typescript-eslint's latest and canary both peer typescript >=4.8.4 <6.1.0, and the package hard-fails at runtime with "typescript-eslint does not support TS 7.0", so --legacy-peer-deps only moves the error from install to lint. No 6.x stable exists to step through. The hold is on the major only, and the config names the command that says when to lift it. The console itself is already TS 7-clean — tsc --noEmit under 7.0.2 reports zero errors.
• React 19, Vite 8, ruff 0.16. React moves as a set (react, react-dom, and both type packages); vite moves with @vitejs/plugin-react 6. Nothing in the console touches an API React 19 removed — no ReactDOM.render, findDOMNode, defaultProps on a function component, propTypes, string ref or forwardRef — which was grepped rather than assumed, since the test suite would not have caught most of them. The vite bump also cut the build from ~1.3 s to ~0.25 s and the bundle from 593 kB to 573 kB. ruff 0.16 formats Python code blocks inside Markdown, so the 24 affected files are docs, not source.
• AutonomyPlugin is registered before GuardrailsPlugin. ADK's before-tool chain early-exits on the first non-None return, so the previous order let an L2 deployment raise a confirmation prompt for a mutation that L2 refuses the moment it is approved. The level is a property of the process — decide it before asking a human anything.
• A paused L4 confirmation reports AWAITING_CONFIRMATION, not BLOCKED. The model acts on that string, and "blocked" reads as a dead end when the action is one human answer away.
• Sessions in a shared thread are keyed per participant (core/orrery_core/serving/gateway.py, agents/slack-bot/slack_bot/session_map.py). ADK scopes every session by (app_name, user_id, session_id), so handing the first speaker's session id to a second speaker never joined them to that conversation — the lookup missed and ADK created a fresh empty session behind the same id. Keying by thread alone only hid that threads are per participant; the mapping now says so, and forget/remove clear the whole thread by default.
• All 9 dev-only npm advisories cleared by upgrading the toolchain (eslint 10, typescript-eslint 8, vitest 4, jsdom 29, globals 17). npm audit --omit=dev was already clean so nothing shipped to a browser was affected, but the noise on every make install hides the one that will eventually matter. Pinning the patched brace-expansion via overrides was tried first and breaks the build — 5.x changed its export shape and minimatch@3.x still calls the old default export.
• The docs home page now leads with the web console running a real triage rather than a Google Chat screenshot: five systems checked in parallel, a Critical verdict, and the recorded findings in the side panel. That is the claim the page makes two paragraphs later, so it may as well be the first thing a visitor sees.

Fixed¶

• uv.lock disagreed with pyproject.toml on main — the manifest asked for pytest-cov>=7.1.0 while the lock still recorded >=6.0.0. The lock is tracked precisely so Docker builds are reproducible, which a lock that disagrees with the manifest is not. Repaired, with the gate above to keep it that way.
• An explicit compaction=None was silently overridden (core/orrery_core/serving/{server,runner}.py). create_app and run_persistent fell back with arg or create_events_compaction_config(); since None is falsy and the only way to disable compaction in code, passing it explicitly re-enabled compaction — the opposite of the request. Env-based disabling always worked, which is why it went unnoticed: the documented off-switch was never the broken one.
• Plugin-composition tests read the developer's .env (core/tests/). Any agent module imported during collection calls load_agent_env(), whose load_dotenv() searches the CWD and its parents, so a perfectly legitimate local ORRERY_AUTONOMY_LEVEL=L3 failed two tests in a file with no connection to it — and only under a full run, since core-only runs never import an agent module.
• An unresolved dotted role claim is now logged (core/orrery_core/security/auth.py). It already failed closed on every malformed claim shape — ten hostile variants all return viewer, none raise — but the fallback was silent, so a mistyped JWT_ROLE_CLAIM demoted every caller to viewer with nothing in the logs pointing at the cause.
• The docker CLI wrapper left a zombie on timeout (agents/docker-agent/docker_agent/tools.py): kill() only delivers the signal, so the child stayed unreaped until the loop's watcher happened to collect it. A tool that times out tends to do so repeatedly.
• Limiter(default_limits=…) was inert (core/orrery_core/serving/server.py). slowapi only applies those through SlowAPIMiddleware, which this app does not install, so the argument read as blanket coverage while doing nothing. Limits are now declared per route, where the cost actually differs — /chat buys tokens; /confirmations/pending is a cheap read the console polls on a timer and must not be throttled into failure.
• ConfirmationStore.add documents why it is not keyed by args_hash, with tests pinning the behaviour. An audit recommended adding it so parallel same-tool calls stop superseding each other; that would also let a card the requester scrolled past authorize an execution minutes later. The supersede costs liveness only — consume_pending matches args_hash exactly, so an evicted call re-prompts and can never be authorized by the survivor's approval.

0.3.0 - 2026-07-25¶

Added¶

• Conversation context compaction (AEP-020), on by default. A long incident session grew its transcript monotonically until the request exceeded the model's window and the turn failed. ToolOutputCapPlugin caps a single tool result at 4 MiB against Gemini's ~10 MiB request ceiling — so three capped results in history were already enough to break the next turn; the cap deferred the failure rather than preventing it, and persistent Postgres sessions made it worse. Implemented by configuring ADK's native EventsCompactionConfig rather than the hand-rolled context engine the AEP originally specified: native compaction refuses to separate a function_call from its function_response (the proposed slice-based design would have split tool pairs and produced provider 400s on exactly the tool-heavy sessions this targets), reads real prompt_token_count, and appends the digest as an event carrying the compacted timestamp range — so the originals stay in the session and are filtered only at request assembly. Lossy for the model, lossless for the record, by construction. create_events_compaction_config() (core/orrery_core/serving/runner.py) is threaded through every App/AgentGateway site; ORRERY_COMPACTION_TOKEN_THRESHOLD defaults to 250k, chosen to sit out of reach of ordinary sessions so enabling it changes nothing else. The summarizer is always passed explicitly — ADK otherwise derives it from the root agent's model, which bills digests at the agent's rate and raises outright for a non-LlmAgent root, which the batch triage Workflow is. Compactions are exported as orrery_context_compaction_total; the hook lives in a summarizer subclass because compaction events bypass on_event_callback entirely (the Runner appends them after the agent generator is exhausted).
• Single sign-on for the web console (OIDC Authorization Code + PKCE). The server perimeter was already OIDC-ready — RS256/JWKS with configurable audience, issuer and role claim — so the missing half was purely in the browser, as TokenGate said in its own comment. Provider-agnostic via oidc-client-ts (Keycloak, Authentik, Auth0, Okta, Entra ID, Google), deliberately a dependency rather than hand-rolled: the failure modes of DIY OAuth are subtle and security-critical. Enabled by setting VITE_OIDC_ISSUER; unset keeps the paste-a-token gate, so make dev-token, CI and offline work are unaffected. The access token is held in memory with silent renew instead of localStorage, and signing out ends the provider session too. The redirect URI is the console root, not /auth/callback — the front door serves the bundle with StaticFiles(html=True), which 404s unknown deep paths, so a sub-path callback would work in dev and break in production.
• A local Keycloak under an sso compose profile, with a pre-imported realm and three demo users (viewer / operator / admin, password same as username) — which makes the RBAC tiers demonstrable in the browser for the first time. make up PROFILES=sso then make run-api SSO=1. No healthcheck on the container by design: the image is distroless, so a CMD-SHELL probe can never run and leaves it permanently unhealthy; make up polls the realm's discovery document from the host instead, which also covers realm import finishing after the server starts listening.
• Role claims may now be dotted paths (JWT_ROLE_CLAIM=realm_access.roles, and the console's matching VITE_OIDC_ROLE_CLAIM). Both sides previously read a flat claim, but the providers people actually deploy nest their roles — Keycloak uses realm_access.roles for realm roles and resource_access.<client>.roles for client roles, neither reachable by a flat lookup, so every SSO user silently resolved to viewer. An unresolvable path still yields viewer: it fails closed.
• Web console: copy-to-clipboard and syntax highlighting. Copy buttons on messages, on code blocks, and on a failed environment check's detail (long connection errors that could not be selected cleanly out of a narrow panel). Reachable by keyboard, not hover-only, with an execCommand fallback for the insecure origins a console on a bastion host is often reached over. Highlighting is restricted to nine languages so the bundle stays inside its 700 kB budget.
• Web console: AEP-019 Milestone 3 — the first-run environment check. POST /onboarding/selftest runs a one-token model round-trip plus every registered integration probe (Kafka, Kubernetes, Elasticsearch, Prometheus, Docker — each specialist's cheapest read-only tool), concurrently, and reports per-check pass/fail with the reason and what to configure when it fails. This is the AEP's own "highest-leverage single feature": nearly every first-run failure is a credential or endpoint that was never wired, and until now the only feedback was a stack trace buried in a tool result several turns into a conversation. Surfaced in the console as a System tab plus a sidebar Check my environment button. Probes are supplied by the app (create_app(integration_probes=...)) so core keeps no dependency on any agent package; they must be read-only, since the self-test does not consult RBAC. A probe that hangs, raises, or returns an error all become one red row rather than breaking the page.
• Web console: the role + autonomy badge is now complete. GET /me returns the server-resolved role and the autonomy level actually in force (L2/L3/L4, or none). The console previously showed only its own decode of the JWT — the browser's reading of a signature it cannot verify. The server's answer now wins in the badge, and the System tab spells out what the active level permits, so a viewer learns why mutating tools are unavailable up front instead of when one is refused.
• Web console: per-system triage chips (AEP-019 Milestone 2). A triage verdict now renders a chip per system that was actually consulted, derived from the recorded tool calls rather than parsed out of the model's prose — the report is free text that changes wording run to run, the activity log is structured. A system that was never called shows no chip: "we didn't ask" and "healthy" are different answers.

Changed¶

• The Makefile is one target per job, with variants as flags (34 targets → 28). make up / make down now act on everything, with PROFILES= to narrow; make check runs the whole gate across both toolchains (ruff + ty + pytest + the web gate), and install and fmt likewise cover both. Variants that used to be near-duplicate targets are now flags: run-cli PERSIST=1, run-api SSO=1, run-slack MODE=socket, run-chat MODE=pubsub, dev-token ROLE=viewer, up PROFILES=tracing. run-api builds the console before serving it, so it is the single command for "the product". npm is optional throughout — Python-only contributors get a warning and a working setup, not a failure. The demo and slack compose profiles are deliberately excluded from make up: they run the agent in Docker on the same ports as make run-api / make run-slack. This renames the developer-facing targets — run-assistant*, infra-*, tracing-*, sso-* and web-* are gone; make help lists the current set.
• Dependabot now covers the web console (.github/dependabot.yml). The pip ecosystem never saw web/, which sits outside the uv workspace, so the console's dependencies — React, and now the OIDC client that handles sign-in — were getting no security updates at all.
• Web console UX: a turn can be stopped, and a failed one retried. A triage sweep fans out to five specialists and can run for a minute with no partial output (streaming is AEP-009), and the only escape used to be reloading the page. Send becomes Stop while a turn is in flight; stopping is reported as stopped rather than failed, and notes that the server may still finish. A failed turn offers Retry, which replays the message without duplicating it in the transcript, and errors can be dismissed. A 429 from the new chat rate limit is explained with its Retry-After instead of surfacing as a bare error.
• Web console: the transcript no longer yanks you back to the bottom. Auto-scroll now only follows the newest message when the reader is already at the bottom — scrolling up to re-read an earlier answer while a sweep is still landing used to fight the user. The transcript also gained role="log" and an accessible name.
• Web console: the composer grows with the message up to a bound, instead of showing a multi-line prompt through a one-line peephole.
• Docs caught up with the 21 tools added in 0.2.3 (agents-overview.md, CLAUDE.md): the at-a-glance table still read 19/18/12/10 tools for kafka/k8s/observability/docker (actually 24/26/16/17) with stale guarded counts, and none of the new tools appeared in the per-agent role tables that the site presents as the authoritative catalog.

Fixed¶

• The web console's environment check returned "Not Found" under make run-web (web/vite.config.ts). The Vite dev proxy listed only /chat, /session, /confirmations, /healthz and /readyz, so the System pane's two endpoints never reached the API. One root cause produced two symptoms because the verbs differ: POST /onboarding/selftest plainly 404'd, while GET /me was rewritten by Vite's history fallback to the SPA shell and returned 200 HTML that then died inside res.json() — which is why Role (server) and Model rendered as "—" with nothing reported. The comment above that list already warned about this exact failure, so the fix is structural rather than two more strings: API_PREFIXES (web/src/api/paths.ts) is now the single source the proxy derives from, guarded by a test that drives every ApiClient method and fails if a path escapes it. Removing a prefix reproduces the original bug as a test failure.
• make clean was destroying node_modules. Its find . -type d -name 'build' matched every npm package that ships its dist in a directory called build — pretty-format, jwt-decode and others — silently gutting them; the web suite then failed much later with Cannot find module …/build/index.js, with no obvious connection to the command that caused it. It now prunes node_modules, .venv and .git before matching.
• Stopping one compose profile tore down the whole stack. docker compose --profile X down removes every service in the file — a profile filter only adds services, it does not scope down — so stopping the tracing or SSO stack also stopped Kafka, Postgres and Prometheus. Both targets now remove their own containers by name.
• make reset deleted local volumes without confirmation. It now lists the volumes it will destroy and requires typing yes (FORCE=1 to skip).
• A non-JSON API response no longer fails silently (web/src/api/client.ts). A 200 that would not parse threw a bare SyntaxError no caller interpreted; it now reports what actually arrived and that the request likely never reached the API. /me failures surface in the System panel instead of being swallowed, and errors distinguish network / auth / rate-limit / 5xx rather than echoing a raw message.
• Web console: an assistant reply could be lost to a same-tick New-chat or delete (web/src/conversations/useConversations.ts). Removing the setState-inside-updater anti-pattern in 0.2.3 replaced it with a stale-snapshot read: newConversation/deleteConversation built a whole new array from the render closure, so a patchActive queued earlier in the same tick — the arriving assistant message — was silently overwritten. The list and the active selection are now one useReducer: they are a single piece of state (every create/delete/select moves both), so React replays each dispatch against the freshest state and neither anti-pattern is reachable. Two regression tests reproduce the lost reply.
• Web console: conversation history is now genuinely bounded (web/). The 50-conversation cap didn't bound storage — one long incident thread grows without limit — and the quota failure it was meant to prevent was swallowed silently. Persisted transcripts are trimmed to the most recent 200 messages (the in-memory transcript is untouched), a rejected write retries against progressively smaller slices instead of giving up, and eviction now drops the least-recently-updated conversation rather than the oldest-inserted — matching the order the sidebar shows.
• Web console: the side-pane refresh guard compared the active conversation with itself (web/src/chat/useChat.ts), so it could never actually fire. The refresh now takes the conversation that owns the session it is fetching for, which is what the guard was meant to compare against.
• A specialist's model failure no longer reads as an answer (core/orrery_core/reliability/error_handlers.py). When an agent reached through an AgentTool hit a model error, graceful_model_error() returned "I encountered an error… Please try again" — and because that text becomes the tool's result in the coordinator's transcript, a coordinator cannot tell a polite apology from a finding: it treats the step as done and can summarize an incident whose Kafka check never ran. The response now states plainly that the step failed and that no result was produced, and calls out quota exhaustion (429) by status code walked up the __cause__ chain rather than by matching an error string.
• k8s tools: input validation gaps (agents/k8s-health). label_selector was passed to the API server unvalidated on list_pods/list_services (the repo's rule is that every tool validates at entry); describe_service/get_configmap accepted namespace="all", which is a legal namespace name, and then 404'd confusingly instead of saying what was wrong; and top_pods summed unparseable CPU/memory quantities as 0, making "this pod is idle" and "we couldn't measure this pod" indistinguishable — it now reports the affected containers instead.

Security¶

• An unknown-kid bearer token returned 500 with a traceback instead of 401 (core/orrery_core/security/auth.py). PyJWT's PyJWKClientError is not an InvalidTokenError, so it escaped the handler on the RS256/JWKS path — meaning essentially every forged or garbage token became an unhandled server error on an unauthenticated request path, burning a JWKS lookup and emitting a stack trace for each one. Now a clean 401. A genuinely unreachable IdP (PyJWKClientConnectionError) still surfaces as a server error, since that is not the caller's fault and a new token would not fix it. This became reachable in practice with the SSO work above, which makes RS256 the recommended production configuration.
• POST /chat is now rate limited (core/orrery_core/serving/server.py). The Slack bot had a limiter; the main front door — the one that spends LLM tokens and can fan a single turn out to six specialists plus a triage sweep — had none, so one leaked credential was unbounded spend. Keyed on the verified JWT subject rather than the source address: behind an ingress every caller shares an IP, and keying on the raw token would hand out a fresh quota on every refresh. Configurable via ORRERY_CHAT_RATE_LIMIT (default 30/minute).
• Elasticsearch search now bounds the query on the cluster (agents/elasticsearch). The HTTP client timeout only bounded our wait — when it fired, a model-authored deep aggregation or leading-wildcard query kept running on the cluster with nobody left to read the result. A timeout (elasticsearch_search_timeout, default 10s) now travels with the body, and a partial result surfaces its timed_out flag so the model can't mistake it for a complete answer.
• A wildcard CORS origin no longer allows credentials (core/orrery_core/serving/server.py). ORRERY_CORS_ORIGINS=* combined with allow_credentials=True makes Starlette echo back whatever origin asked, turning every site into a permitted credentialed caller. The API authenticates with a bearer header rather than a cookie, so credentials buy it nothing — they are now dropped (with a warning) when the origin list contains *.
• pyasn1 0.6.3 → 0.6.4 (uv.lock): CVE-2026-59885 and CVE-2026-59886, two HIGH denial-of-service issues via crafted ASN.1 OBJECT IDENTIFIER / REAL values. Transitive via google-adk → google-auth → pyasn1-modules, so a lock-only bump — nothing declares it directly and pyasn1-modules already allows <0.7.0.
• Requester-verified confirmation: a stale approval could authorize a later action (core/orrery_core/security/guardrails.py, core/orrery_core/serving/gateway.py). The gateway only wrote _confirmation_decision on turns whose text was a decision word, and the gate only checked that the decision was fresh (TTL 300s) and that a pending matched the tool + args-hash. So an approve typed while nothing was pending — for something else, or for nothing at all — stayed in session state and authorized whichever guarded call came next: the model raised the pending, re-called the tool, the args-hash matched, and a @destructive tool executed on an approval no human had given for it. The model-mediated path had a different_invocation check against exactly this; strict mode had dropped it. Fixed from both ends — the gateway now rewrites the decision key every turn (writing None when the message is not a decision), and the gate requires decision.timestamp >= pending.created_at plus decision.by == requester, so an approval that predates the action it would authorize is refused and the action re-prompted. Applies to every shipped exposition (HTTP front door, persistent runner, Slack, Google Chat), all of which arm verified_confirmation=True. Two regression tests cover the bypass and the honest flow.
• Tool exception text no longer reaches the model context (core/orrery_core/reliability/error_handlers.py): graceful_tool_error() returned f"Tool '{name}' failed: {error}", and client exception strings from HTTP/Kubernetes/Kafka embed internal hosts, URLs, and token paths (CWE-209) — which then flowed into the transcript, the session store, and whatever the model said next. The full traceback now stays in the server log and only the exception class is named. The one exception is an HTTP 4xx, whose body describes what was wrong with our own request and is what lets the model correct and retry.
• Namespace scope guard (NamespaceScopeGuard in core/orrery_core/security/rbac.py): RBAC decided which tool a role could run but never where, and restart_deployment is the same @confirm tool whether it targets payments or kube-system. Set ORRERY_PROTECTED_NAMESPACES (comma-separated globs) to refuse non-admin mutations in infrastructure namespaces; reads are never scoped, so triage still sees everything. The effective namespace is resolved from the call's argument or the tool's signature default, and a namespace that cannot be resolved (required-but-missing, or a non-string) fails closed. Opt-in — unset leaves the guard inert, so existing deployments are unchanged.
• alter_topic_config re-gated as @destructive (agents/kafka-health): it was @confirm (operator), but retention.ms=1 or moving cleanup.policy off compact destroys a topic's data as surely as deleting it — a data-loss path behind a non-destructive gate that autonomy L3 also allowed. It is now admin-only and restricted to the four data-affecting keys; the everyday tuning it used to cover (message size, compression, min ISR) moved to a new @confirm tune_topic_config, which refuses the destroying keys.

Performance¶

• PII redaction no longer holds the event loop, and does ~2x less work (core/orrery_core/plugins/pii_plugin.py). It was the dominant cost of the whole after-tool chain — measured ~60 ms per MiB, 1.28 s on a 20 MiB get_pod_logs result — in an async callback whose body is pure CPU, so every other in-flight request stalled for the duration. It also runs on the uncapped result, since ToolOutputCapPlugin has to stay last in the chain (it returns a replacement, which early-exits ADK's after-tool chain). Two fixes: each secret-value pattern now declares the case-sensitive literals it cannot match without (AKIA, ghp_, eyJ, -----BEGIN, …) and is skipped outright when they are absent — 2x faster on clean log text with byte-identical output — and redaction above 256 KiB moves to a worker thread. Measured on 20 MiB: chain 1445 ms → 785 ms, peak loop stall 1200 ms → 515 ms. (Combining the patterns into one alternation was tried first and measured slower, so it was dropped.)
• Audit no longer writes the entire tool response (core/orrery_core/observability/audit.py). Running before the output cap meant a 20 MiB result became a 20 MiB log line, per tool call, with a +45 MiB allocation spike — the largest driver of log-ingestion cost on a busy agent, and a copy of infrastructure output into a store with far wider readership than the agent. Responses over MAX_AUDIT_RESPONSE_CHARS (4096) are now recorded as their status plus measured size; smaller ones (the overwhelming majority) stay verbatim so the trail is still debuggable. The status is kept whatever the size. Measured on 20 MiB: 42 ms → 0.9 ms, +45.3 MiB → +0.1 MiB.
• session_log is bounded (core/orrery_core/observability/activity.py). The log was rewritten whole on every tool call, and ADK's State.__setitem__ copies the assigned value straight into the event's state delta — so event N persisted a list of N entries and a session's write volume grew with the square of its length (a 50-tool triage sweep wrote ~1,275 entries' worth of deltas). Capped at the most recent MAX_SESSION_LOG_ENTRIES (200), which is what the log is actually read for.
• Consumer-lag calculation is no longer quadratic (agents/kafka-health). get_consumer_lag scanned the committed-offset list once per partition. Measured lookup cost: 0.59 ms at 200 partitions, 16 ms at 1000, 152 ms at 3000 — against 0.56 ms through a dict index.
• Memory recall is bounded (core/orrery_core/persistence/memory.py). search_memory had no LIMIT and no recency cut-off, and every matching row is JSON-parsed into a Content and then sent to the model — so a recall on a common word ("error", "pod") got slower and more expensive for the life of the deployment, since memory is append-only and every session of ≥4 events is saved. Now returns the newest MAX_SEARCH_RESULTS (200), flipped back to chronological order for the reader.
• The blocking-tool thread pool is sized from the container's CPU quota (core/orrery_core/concurrency.py, new). Every tool offloads to the loop's default executor, which asyncio sizes at min(32, os.cpu_count() + 4) — and in a pod os.cpu_count() reports the host's cores, so a 64-core node built a 32-thread pool for a container the Helm chart limits to 1000m. configure_default_executor() reads cgroup v2 cpu.max (then v1, then the stdlib) and installs a proportional pool with a floor of 8, called at startup by the HTTP front door and the persistent runner. Override with ORRERY_MAX_WORKER_THREADS.
• GET /confirmations/pending no longer blocks the event loop (core/orrery_core/serving/server.py). The console polls it every 2.5 s while a request is in flight, and the postgres confirmation backend is a synchronous engine — so each poll was a blocking database round-trip on the loop, per client, on a timer. Now offloaded.

Documentation¶

• integrations/web-console.md documents both auth modes and the two Keycloak pitfalls that each present as a generic "Invalid or expired token": realm roles are nested at realm_access.roles, and access tokens carry aud: account unless an audience mapper is added. config/security.md covers dotted role-claim paths; config/general.md gains a Context Compaction section covering the interaction with context caching (a compaction invalidates the cached prefix, so a threshold tuned too low quietly erodes the cache-hit rate).
• Docs caught up with the Makefile rename, which surfaced references to targets removed long ago — run-kafka-health, run-devops, run-docker, run-journal, run-elasticsearch, run-observability, run-k8s and their -cli variants, all in agent READMEs telling readers to run agents standalone. They now point at the orchestrator, which is how the platform is meant to be run.
• integrations/web-console.md (new) — enabling, auth posture, what each panel is for, the endpoints it uses, and the deliberate non-goals. Linked from the integrations overview (now listed first, ahead of the developer-facing adk web) and from getting-started. AEP-019 is marked complete, with the two things it deliberately does not do recorded rather than left as open checkboxes: token-by-token streaming (AEP-009) and the remediation act→verify→retry trace, which lives in the batch workflow the console does not host.

0.2.3 - 2026-07-19¶

Fixed¶

• Web console reliability cleanups (web/): four issues from the app-shell review — (1) a side-pane refresh race where a slow activity/triage/pending fetch for a previous conversation could clobber the one now on screen (useChat now tags each refresh with the conversation it was for and drops stale results); (2) removed the dead sessionId localStorage key (no longer written — sign-out still clears it and any legacy keys); (3) the setState-inside-updater anti-pattern in useConversations (newConversation/deleteConversation no longer call setActiveId inside the setConversations updater — impure under StrictMode); (4) capped conversation history at 50 so localStorage can't grow unbounded and silently stop persisting. 8 new hook tests (useConversations, useAuth sign-out); tsc / eslint / prettier / build all clean.

Added¶

• 21 new tools across four specialist agents (fully validated, guarded, and tested — 45+ new unit tests, all suites green):
• k8s-health (+6, read-only): list_services / describe_service (reports ready vs not-ready Endpoints — the usual cause of "connection refused" when pods look fine), list_configmaps / get_configmap (values truncated), and top_nodes / top_pods (live CPU-millicores / memory-MiB via the metrics API, with a clear message when metrics-server is absent).
• kafka-health (+4): get_topic_config (splits overridden vs default, masks sensitive), alter_topic_config (@confirm, incremental — other configs untouched), reset_consumer_group_offsets (@destructive, earliest/latest), delete_consumer_group (@destructive).
• docker-agent (+7): list_networks / inspect_network, list_volumes / inspect_volume, system_df (disk usage by type), and prune_images / prune_containers (@destructive, dangling/stopped by default).
• observability (+4, read-only): list_prometheus_metrics (discover metric names by substring), get_prometheus_metadata (type/help), get_prometheus_rules (alerting + recording rules and their state), and query_loki_range (LogQL over a relative last-N-hours window).
• Each agent's instruction gained a short routing note so the model reaches for the new tools appropriately; all mutating tools flow through the existing @confirm/@destructive + requester-verified confirmation gate.

0.2.2 - 2026-07-19¶

Changed¶

• Docs review: fixed staleness, tightened onboarding docs, and polished the MkDocs site (CONTRIBUTING.md, SECURITY.md, `,mkdocs.yml):CONTRIBUTING.mdwas rewritten — it pointed contributors at a non-existentmake run-kafka-healthtarget (nowmake run-assistant), the wrong validator import (orrery_core.validation→orrery_core.security.validation), and a stale project tree; it now documents the realfmt/lint/test/evalflow, Conventional Commits, the CHANGELOG expectation, the AEP process, and the mock-every-operator-client testing rule.SECURITY.mdnow references the shipped runtime defenses (SafetyScreenPlugin,PIIRedactionPlugin) and the0.xpre-release support line. Brand consistency pass (AI Agents platform→Orrery) across getting-started/integrations/enhancements. MkDocs style: Inter + JetBrains Mono fonts,navigation.prune/content.code.annotate` / instant-progress, and a light CSS polish for tables and code blocks. Strict build passes.

Added¶

• Four new enhancement proposals from a Hermes-architecture benchmark (enhancements/aep-020..023): comparing Orrery against the mature Hermes agent architecture surfaced four production-readiness gaps worth adapting to the ADK/Postgres model — AEP-020 conversation context compaction (a pluggable summarizing context engine so long incident sessions don't overflow the model window, with lossless lineage), AEP-021 LLM provider fallback chain (FallbackLlm around resolve_model() reusing the existing circuit breaker so a provider outage/quota isn't a full platform outage), AEP-022 trajectory capture (export real runs → *.test.json eval scenarios + a fine-tune corpus, downstream of PII redaction), and AEP-023 first-class scheduled agent tasks (persisted recurring triage sweeps with run history, L2 read-only by construction). All proposed; wired into the enhancements index, roadmap, and nav.

Fixed¶

• Agent evals were failing because the harness didn't mock operator clients (agents/{kafka-health,k8s-health}/tests/test_*_eval.py): the kafka and k8s eval runners mocked only the primary client (Kafka AdminClient / K8s core+apps APIs) but not the operator client (strimzi._custom_objects_api / operators._custom_objects_api). On runs where the model reached for an operator tool, the call hit a live cluster, errored, and polluted the tool trajectory — so exact-match (tool_trajectory_avg_score must be 1.0) collapsed to 0.0 non-deterministically. Both harnesses now mock the operator client too, making the evals hermetic. (Elasticsearch already mocked its ECK client; docker and observability had no operator layer.)

Changed¶

• Agents constrained to deterministic tool choice so exact-match evals stay green (agents/{kafka-health,k8s-health,observability}/*/agent.py): the current model (gemini-3.1-pro-preview) legitimately varied its tool set on broad prompts, which strict trajectory matching can't tolerate. Sharpened three instructions — a better product and a stable eval: kafka now uses Strimzi operator tools only when the user explicitly asks about the operator/CRs/connectors/rebalances/MM2/users (not on plain health/topic/lag questions), plus a "be surgical" rule; k8s defines a cluster "overview" as exactly get_cluster_info + get_nodes (namespaces only when asked); observability builds Loki LogQL directly ({job="…"} |= "…") and calls query_loki_logs once instead of a non-deterministic get_loki_labels/get_loki_label_values discovery preamble. Kafka's consumer-lag scenario dataset was updated to include the deliberate describe_consumer_groups member check (the instruction has always paired it with lag — "lag with no members is an outage, not slowness"). All 5 agent eval suites pass. New guide: Agent Evaluations.
• Cross-session memory is now model-invoked (LoadMemoryTool) instead of preloaded (agents/orrery-assistant/orrery_assistant/agent.py): the chat root swapped PreloadMemoryTool — which searched memory on every turn (query = raw user message) and auto-injected the hits — for LoadMemoryTool, exposing a load_memory function the coordinator calls only when it judges past context useful, with a query it crafts. This keeps trivial turns (greetings, one-off status checks) cheap and gives targeted queries, at the cost of relying on the model to look — so the instruction was updated to say explicitly when to call it (before diagnosing a symptom/alert, or when the user references an earlier incident), when to skip it, and how to query (short system + symptom, at most once per turn). Same memory_service backend and MemoryPlugin write path — no service or storage change. Docs (memory.md, README, agents-overview.md, CLAUDE.md) updated; the ADR-003/AEP-003 records keep their original PreloadMemoryTool wording as point-in-time history.
• Web console redesigned into an app shell (Tailwind CSS v4) (web/): the single-column layout is now a left sidebar (brand, New chat, Run triage, conversation history, identity/role footer) + chat column + a toggleable right inspector panel with two tabs — a clean tool-calls table (time / tool / agent / status, replacing the inline list) and the triage report rendered as prose (auto-opens when a verdict lands). Adds client-side conversation history: since the server has no "list sessions" endpoint, each conversation (full transcript + server sessionId) is kept in localStorage, titled from its first message, and switchable from the sidebar — selecting one reloads its tool-call/triage panes from the server. Styling moved from ~640 lines of hand-written CSS to Tailwind v4 (@tailwindcss/vite) with @tailwindcss/typography for markdown; dark mode still follows prefers-color-scheme. No API or server changes — same endpoints, same requester-verified confirmation flow.

Added¶

• Web console: triage view — AEP-019 Milestone 2 (core/orrery_core/serving/server.py, web/): a Run triage header button sends the canned sweep prompt to the incident_triage_agent, and a new owner-scoped GET /session/{id}/triage endpoint returns the recorded verdict (incident_severity + triage_report — ADK's AgentTool forwards sub-session state deltas to the parent session, so the chat-root triage lands in the HTTP session's state). The console renders it as a severity banner (healthy / degraded / critical, theme-aware) with the full report collapsed inside as markdown, and the tool-call timeline is now polled every 2.5s while a request is in flight, so multi-specialist sweeps become visible as each specialist completes — the closest thing to progress until streaming (AEP-009). 5 new server tests + 2 console tests.
• Web console: tool-call timeline + confirmation UI — AEP-019 Milestone 1 complete (core/orrery_core/serving/server.py, web/): two new read endpoints and their renderers. GET /session/{id}/activity returns the session's tool-call log (recorded by ActivityPlugin under session_log); the lookup pins user_id to the verified JWT subject, so another user's session id is a plain 404. The console renders it as a collapsed timeline under the transcript — which specialist ran which tool, with what outcome. GET /confirmations/pending surfaces the caller's own guarded action awaiting a decision (strict-mode pendings are requester-scoped, so the endpoint can only ever show your own); the console renders an Approve/Deny panel whose buttons send the literal words approve/deny through the normal POST /chat flow — rendering only, the requester-verified gate remains the sole authority on who may approve. New public accessor latest_pending_for_scope() in guardrails; 11 new server tests + 6 new console tests (incl. an end-to-end approve flow).
• Runtime security hardening — AEP-013 completed (core/orrery_core/plugins/safety_plugin.py, pii_plugin.py, agent/base.py): three content-level defenses, all on by default in default_plugins(). SafetyScreenPlugin blocks prompt-injection messages ("ignore previous instructions", "reveal your system prompt", "bypass the guardrails", ...) in before_run_callback — the one plugin hook whose non-None return halts the runner — so a screened message never reaches the model, costs no tokens, and cannot influence a tool call (ORRERY_SAFETY_SCREEN=false disables). PIIRedactionPlugin scrubs credentials from every tool result — credential-named dict keys replaced outright, plus value-pattern scanning for password=... pairs, PEM blocks, AWS/GitHub/Slack/OpenAI token shapes, and JWTs — by mutating the result in place and returning None, because ADK's after-tool chain early-exits on the first non-None return and a returned copy would silence every later observer; it registers before AuditPlugin so the audit log records redacted values too (ORRERY_PII_REDACTION=false disables; ORRERY_REDACT_IPS=true additionally redacts IPv4s — off by default since an SRE agent that can't see pod IPs can't diagnose much). Gemini content-safety filters: create_agent() attaches GenerateContentConfig safety settings (dangerous content / harassment / hate speech / sexually explicit) to Gemini models at BLOCK_ONLY_HIGH (GEMINI_SAFETY_THRESHOLD overrides; GEMINI_SAFETY_FILTERS=false disables; LiteLLM providers unaffected). 41 new tests.
• Supply-chain hardening — AEP-014 completed (Dockerfile, .github/workflows/{ci,release}.yml, deploy/k8s/imagepolicy.yaml, supply-chain.md): all three base images are now pinned by digest (tag@sha256:...; Dependabot's docker ecosystem bumps them), a CycloneDX Python SBOM of the frozen dependency graph is generated on every CI build (artifact) and attached to each GitHub release (asset), the pushed image is Trivy-scanned before the release job runs — HIGH/CRITICAL fixable CVEs fail the workflow so no release is cut for a vulnerable image, with SARIF uploaded to GitHub Code Scanning — and PRs gain a dependency-review gate (fail-on-severity: high). An opt-in Sigstore ClusterImagePolicy (deploy/k8s/imagepolicy.yaml) lets clusters refuse any ghcr.io/bahalla/orrery* image not signed by this repo's release workflow. The verification flow (cosign verify, SBOM download/scan) is documented in Supply Chain Security. Cosign keyless signing and buildx SBOM/provenance attestations had already landed with AEP-011.

Fixed¶

• Web console leaked conversation history across sign-out and trusted stored data (web/src/auth/useAuth.ts, web/src/conversations/useConversations.ts): sign-out cleared the token but left the conversations/activeConversation localStorage keys, so on a shared browser the next user saw the prior user's full transcripts — sign-out now clears all user-scoped keys. Separately, load() blindly cast stored data (as Conversation[]); a corrupt or old-schema entry could throw during render and white-screen the app, so it now validates each entry against the current shape and drops anything malformed.
• PII redaction missed camelCase credential keys (core/orrery_core/plugins/pii_plugin.py): SENSITIVE_KEY_PATTERN required a _/-/. separator before the credential word, so dbPassword / accessToken / AccessToken leaked while db_password was caught. Keys are now normalized to snake_case (camel boundaries → underscores) before both the sensitive-match and the allowlist check — which also keeps nextPageToken mapping onto the allowlisted next_page_token instead of being redacted as a token (the naive fix of a case-insensitive camel lookaround in the regex would have broken exactly that). Also: token now matches singular-only — plural endings (total_tokens, maxTokens) are LLM usage counts, not credentials, and were being scrubbed.
• Security: docker specialist's destructive tools ran without confirmation on the HTTP/console path (agents/docker-agent/docker_agent/agent.py): the GuardrailsPlugin enforces RBAC only — human-in-the-loop confirmation comes from each agent wiring before_tool_callback=require_confirmation(), and four of the five specialists did; docker never had it, so an admin asking the console to delete an image got an immediate unconfirmed remove_image (Slack/Google Chat were unaffected — those transports walk the whole agent tree and wire their own gate). The docker agent now wires the gate like its peers, and a new structural test walks the chat root's AgentTool tree and fails if any agent exposes a @confirm/@destructive tool without a before-tool gate, so this class of gap cannot ship again.

Changed¶

• Web console renders assistant replies as markdown (web/src/components/MessageList.tsx): agent replies are markdown (lists, bold, inline code, tables) and were displayed raw — literal ** and * bullets. Assistant bubbles now render through react-markdown + remark-gfm (React elements, no dangerouslySetInnerHTML, so no sanitizer needed) with theme-aware typography for lists, code blocks, and scrollable tables; user messages stay plain text.

0.2.1 - 2026-07-18¶

Added¶

• Web console MVP (AEP-019, Milestone 1) (web/, core/orrery_core/serving/server.py): a Vite + React + TypeScript single-page console served by the existing FastAPI front door — bearer-token gate (token persisted locally, sent as Authorization on POST /chat), chat threading the server-issued session_id, an identity + role badge (decoded client-side for display only; the server remains authoritative for RBAC), and loading/error/network/auth-expiry states with theme-aware, accessible CSS. Opt-in via ORRERY_WEB_CONSOLE_ENABLED (default off): the built bundle mounts at / after the API routes so explicit routes always win, and a missing bundle is warn-and-skip, not a crash — the static shell is public, /chat stays JWT-gated. The Docker image builds the bundle in a node:22 stage and copies only dist/ into the Python image (no Node in the runtime image); the Node toolchain is isolated from the uv workspace, so make test stays Node-free. Dedicated web CI job (eslint + prettier + tsc strict + vitest).
• One platform-wide confirmation store and flow for guarded tools (core/orrery_core/security/confirmation_store.py + confirmation_flow.py): the pending-approval handshake was implemented three times — core strict mode, the Google Chat bot, and the Slack bot — with drifting semantics (Slack had no TTL, no args-hash pinning, no approval-validity window, and a session-state retry flag that let any retry through once a pending was raised; the Postgres backend existed twice). All transports now share one PendingConfirmation record and one store (memory | postgres via ORRERY_CONFIRMATION_BACKEND, over the existing DATABASE_URL), scoped per transport (requester for HTTP/CLI strict mode; thread/space for Chat; channel/thread for Slack), plus shared flow primitives: raise_pending, approval_refusal (requester-only, fail-closed), blocked_payload, hash_args, and the agent-tree walker wire_before_tool_callback. One-shot guarantees are uniform and atomic (single DELETE … RETURNING on postgres — racing replicas cannot both consume a decision; a lock on memory), approvals are args-hash-pinned with a 120s validity window everywhere, and AgentGateway(verified_confirmation=True) resolves the backend eagerly so a misconfigured postgres backend fails at startup (same fail-fast contract as the session store). Slack hardening: the bot inherits TTL, args-hash pinning, the approval-validity window, the AgentTool-boundary-safe handshake, and the optional Postgres backend (multi-replica) for free. Breaking (unreleased surface only): GOOGLE_CHAT_CONFIRMATION_BACKEND is replaced by ORRERY_CONFIRMATION_BACKEND (the Helm value pubsubWorker.confirmation.backend is unchanged and now exports the new env var), and the unreleased orrery_gchat_confirmations / orrery_pending_confirmations tables are superseded by one orrery_confirmations table (entries are 5-minute-TTL ephemera; abandoned tables can be dropped). The multi-replica motivation: a process-local pending store quietly contradicted both AEP-018's backlog HPA on the Pub/Sub worker and any multi-replica HTTP deployment — a pending raised on pod A was invisible to the pod that received the operator's approval (or the LLM's consuming retry), so the flow died. With postgres, the handshake also survives pod restarts, and the Helm chart still refuses to render a multi-replica Pub/Sub worker while the backend is memory, mirroring the idempotency guard.

Changed¶

• All agent prompts standardized on a shared SRE operating doctrine (core/orrery_core/agent/prompts.py + every agent's agent.py): all 13 instruction sets now compose a shared OPERATING_PRINCIPLES block — verdict-first answers, report only what a tool returned this turn (a failed call is itself a finding; unknown is never healthy), minimal sufficient calls for targeted questions, and read-back verification after any state-changing action — plus a CONFIRMATION_RULE block (on a confirmation_required result: relay to the user and stop, never self-approve). Triage checkers gained a fixed report contract (STATUS: healthy | degraded | critical | unknown; missing data is never reported healthy), the summarizer enforces strict verdict rules (critical if any system is critical; degraded on any degraded or unverified system; healthy only when everything affirmatively reports healthy; record_triage_verdict exactly once), and the remediation actor/verifier were hardened (one action per iteration, smallest blast radius first; the verifier checks the original symptom is gone, not merely that the action ran). _PROBLEM_SIGNALS now also flags status: unknown / unverified / unreachable reports.
• Env var renamed: CONTEXT_CACHE_MIN_TOKENS → CONTEXT_CACHE_MIN_LENGTH (core/orrery_core/serving/runner.py, .env.example, Helm values, docs): Trivy's KSV-0109 check flags ConfigMap keys containing TOKEN as potential secret leaks; renaming the key beats carrying a permanent ignore rule. The old name (shipped in 0.2.0) is still honored as a fallback with a deprecation warning, so existing deployments keep their setting — but update your env files and Helm values.
• Dependencies: google-adk >= 2.4.0 (Postgres-backed tests now skip when the database is unreachable instead of failing), litellm >= 1.91.2 (root floor and the uv override-dependencies pin aligned — the override silently replaced the floor before), uvicorn >= 0.51.0, confluent-kafka >= 2.15.0, slack-bolt >= 1.29.0, google-auth >= 2.55.2; dev tooling ruff >= 0.15.21, ty >= 0.0.58, hypothesis >= 6.156.6; CI astral-sh/setup-uv 8.3.2; uv.lock refreshed.

Fixed¶

• Destructive-tool confirmation looped forever through an AgentTool (core/orrery_core/security/guardrails.py, agents/orrery-assistant/orrery_assistant/agent.py): in requester-verified (strict) mode the pending confirmation was stored in tool_context.state, but guarded tools are reached through an AgentTool (the chat root delegates to a specialist) and every AgentTool call runs the specialist in a fresh, throwaway sub-session. The pending written during the request turn was therefore gone by the turn the human's approve arrived, so the approved branch was structurally unreachable — the gate re-prompted every time and the root model spun, re-calling the specialist with fabricated approvals ("approve", "Force restart…", "I explicitly approve…") until it hit its iteration cap. Two fixes: (1) strict-mode pendings now live in a process-level PendingConfirmationStore keyed by requester (the one identity the gateway forward-propagates into every sub-invocation and that is stable across turns), so a real approval in a later, different session resolves the pending raised earlier; the "only the original requester may approve" rule is now enforced by the key itself (fail-closed on an unknown requester). (2) The chat-root instruction now mandates: on a confirmation_required result, relay it to the user and stop — never self-approve or re-call the specialist in the same turn. Model-mediated (non-strict) mode is unchanged. Regression test drives a @destructive tool across two distinct sub-sessions and asserts the approval resolves. Note: PendingConfirmationStore is process-local; multi-replica HTTP deployments need a shared backend, mirroring the Google Chat store above.

0.2.0 - 2026-07-11¶

Added¶

• Autonomy levels (L2/L3/L4) (core/orrery_core/plugins/autonomy_plugin.py): a third access-control axis, orthogonal to RBAC — RBAC answers who may act, autonomy answers which mode this process runs in. L2 is read-only fail-closed (only unguarded tools + an explicit whitelist run), L3 allows mutations but blocks @destructive (+ blacklist), L4 allows destructive tools behind ADK-native request_confirmation. Classification comes from the same @confirm/@destructive metadata RBAC uses. Opt-in: registered only when ORRERY_AUTONOMY_LEVEL (or default_plugins(autonomy_level=...)) is set, with a per-request session.state["autonomy_level"] override. Blocked calls return a structured deny dict so audit/metrics still observe them.
• Tool output cap (core/orrery_core/plugins/output_cap_plugin.py): ToolOutputCapPlugin bounds every tool result to max_tool_result_bytes (default 4 MiB, 0 disables). One chatty logs call or wide Elasticsearch result is otherwise re-sent with the whole session every turn until the Gemini/Vertex ~10 MiB request limit kills the run with 400 INVALID_ARGUMENT. Trimming is structure-preserving — the longest string field or list is trimmed element-wise so the JSON stays parseable, small status fields survive — with a truncation note telling the model to narrow its query. Runs last among the after-tool observers (ADK's after-tool chain early-exits on the first non-None return).
• Requester-verified confirmation (core/orrery_core/security/guardrails.py, serving/gateway.py): the @confirm/@destructive gate no longer trusts a model re-call as proof a human said yes. With AgentGateway(verified_confirmation=True) — now enabled on every shipped exposition (HTTP server, persistent runner, Slack bot, Google Chat bot) — the gate requires an explicit decision stamped by the gateway from the transport's verified sender: a deliberate approve word (approve/confirm/proceed/go ahead; a casual "ok"/"yes" doesn't authorize) by the same user who triggered the pending action (fail-closed: unknown requester, second-person approvals, stale decisions, and changed args all refuse), while deny is broad. Dev surfaces (adk web, CLI, evals) keep the model-mediated flow.
• Per-turn caller identity (core/orrery_core/agent/base.py): create_agent() wraps every instruction in an identity_aware_instruction provider that appends "who you are talking to THIS turn" whenever a transport stamped the turn's actor (the gateway stamps msg.user_id automatically; _auth.subject is the fallback) — so in shared threads the model acts for the current sender and never reports a tool's service account as the user. Side effect: prompts are used verbatim (no {var} state templating; literal braces are safe). base_instruction() unwraps the provider for tests.
• Audit attempt records (core/orrery_core/observability/audit.py): AuditPlugin now emits a tool_attempt event in before_tool — registered ahead of the gate plugins — so a call that a gate denies, or that crashes mid-tool, still leaves an audit record of what was attempted; the outcome (including a gate's deny status) is audited after the call as before.
• AgentGateway — one shared turn pipeline (core/orrery_core/serving/gateway.py): every transport (HTTP, Slack, Google Chat, CLI) used to re-implement the same five steps — build a Runner, resolve identity, map a conversation to an ADK session, run the agent, extract the reply. A ports-and-adapters gateway now owns that pipeline (InboundMessage/OutboundReply, ChannelAdapter, pluggable SessionResolvers), with sessions in PostgreSQL (DATABASE_URL) or in-memory.
• Pub/Sub worker idempotency (AEP-018) (agents/google-chat-bot/google_chat_bot/idempotency.py): Google Chat's Pub/Sub transport is at-least-once, so a redelivered event (lost ack, pod OOM, handler-timeout nack after side effects) would double-run @destructive tools — restart/scale/rollback a deployment, increase Kafka partitions, silence Alertmanager. The worker now claims each event id before dispatching and drops duplicates (ack without re-executing); a failed handler releases the claim so a legitimate redelivery still retries (at-most-once side effects without losing failed work). New IdempotencyStore protocol with two backends: InMemoryIdempotencyStore (bounded + TTL, single-replica) and PostgresIdempotencyStore (INSERT … ON CONFLICT DO NOTHING, shared across replicas via the existing DATABASE_URL — no new infrastructure). Dedup key is the Chat eventId, falling back to a stable content hash. 33 unit tests including concurrent-claim races on both backends.
• Backlog-based autoscaling for the Pub/Sub worker (AEP-018) (deploy/helm/orrery-assistant/templates/pubsub-worker-hpa.yaml): an HPAv2 scaling on the subscription's num_undelivered_messages External metric (CPU/memory are useless for an LLM-I/O-bound worker), behind pubsubWorker.autoscaling.enabled. The chart fails to render a multi-replica worker (replicaCount > 1 or autoscaling on) while idempotency.backend == memory, making the split-brain misconfiguration impossible to ship.

Changed¶

• Google Chat confirmations decide by thread reply; buttons are opt-in (agents/google-chat-bot/): a card button click is resolved by Google's Workspace add-ons runtime with a synchronous HTTPS round-trip that a Pub/Sub pull worker cannot answer — the click fails on Google's side with error code 3 ("The Chat app didn't respond or its response was invalid") before the worker sees it. The confirmation card now asks the operator to reply approve or deny in the card's thread (a plain MESSAGE event, which Pub/Sub always delivers with the thread attached); the handler resolves the reply against the thread's pending confirmation with the same semantics as a click — requester-only approve (fail-closed), broad deny, deliberate approve words only, synthetic re-run + one-shot consume_approved. Inline ✅/❌ buttons (and the triage card's 🔧 Run-remediation button) render only with GOOGLE_CHAT_INTERACTIVE_BUTTONS=true, for HTTP-endpoint deployments where clicks can complete.
• Plugin registration order (core/orrery_core/plugins/__init__.py): default_plugins() now registers Audit before the gates (ADK's before-tool chain early-exits on the first non-None return, so anything after a deny never runs) and the output cap last among after-tool observers; the canonical order is documented in the module docstring.
• Core reorganized into aspect subpackages (core/orrery_core/{agent,observability,persistence,plugins,reliability,security,serving,tools}/), with re-exports preserved at orrery_core.*. Database sessions now fail fast when DATABASE_URL is set but unreachable (opt-out via ORRERY_DB_ALLOW_INMEMORY_FALLBACK), and the Slack bot resolves the sender's role per turn instead of per session.
• Build & CI: explicit build backends + unified package discovery across the workspace; CI reuses the Makefile targets so the quality gate is defined once; release checks gate on tag pushes only; kubeconform actually validates the rendered Helm manifests; make gains a verify-only check (lint + ty + test) with derived ty search paths, and the per-agent runner targets were dropped (run the orchestrator — it composes every specialist).
• Google Chat bot config: google_chat_interactive_buttons (default false) plus three worker settings — google_chat_pubsub_idempotency_backend (memory | postgres) and ..._ttl_seconds — plus Helm pubsubWorker.idempotency.* / pubsubWorker.autoscaling.* values. Documented in Google Chat: Pub/Sub Setup.
• Dependencies: google-adk >= 2.3.0, ruff >= 0.15.20, ty >= 0.0.56, hypothesis >= 6.156.1, actions/checkout@v7; uv.lock refreshed.

Removed¶

• Google Chat Quick Commands: all APP_COMMAND handling is gone (event detection, both app-command handlers, appCommandPayload extraction fallbacks). No commands need to be configured in the Chat API console anymore — decisions are thread replies (or opt-in buttons).

Fixed¶

• Pub/Sub worker: Helm now refuses to render a multi-replica worker with the in-memory idempotency backend (HPA/replica conflict guard), and the malformed-payload catch is scoped to decode errors instead of swallowing everything.

Security¶

• Approvals are requester-only on every surface: the Slack and Google Chat click/reply handlers refuse an approval from anyone but the verified user who triggered the guarded action (fail-closed — an unidentifiable decider is refused too; the pending survives so the requester can still decide), while deny stays open to anyone. Previously any channel member could approve someone else's destructive action.

Documentation¶

• AEP index gains colored status/priority badges; ADR-003 and guardrails schemas converted to Mermaid; diagrams and wide content restyled for legibility; images centralized under images/; guardrails and Google Chat integration docs rewritten for the two confirmation modes (including why buttons can't work over Pub/Sub).

0.1.11 - 2026-06-21¶

Added¶

• Distributed tracing with OpenTelemetry (AEP-010) (core/orrery_core/tracing.py): New configure_tracing() installs a process-global TracerProvider exporting to an OTLP collector (Tempo, Jaeger, Cloud Trace, …) with a console fallback for local dev — idempotent and gated by OTEL_TRACING_ENABLED. ADK 2.0 already emits native spans for agent / tool / LLM calls under the gcp.vertex.agent tracer, so the new TracingPlugin enriches the current span (orrery.request_id, orrery.user_role, orrery.tool.status / result_size, exception recording) rather than creating duplicate spans; after_model only bridges token counts into track_llm_tokens() since ADK already records gen_ai.usage.*. default_plugins(enable_tracing=None) resolves the flag from OTEL_TRACING_ENABLED and prepends the plugin first, so a single env var turns tracing on across every transport (Google Chat, Slack, HTTP server, persistent runner) with no per-agent wiring — a missing [otel] extra is a skip-with-warning, not a crash.
• Log ↔ trace correlation (core/orrery_core/log.py): JSONFormatter now stamps request_id (a dependency-free ContextVar) plus trace_id / span_id (lazy OpenTelemetry lookup, omitted when the extra isn't installed or no span is active) onto every log record, so a log line can be pivoted straight to its trace.
• Local tracing stack (docker-compose.yml, infra/tempo.yml, infra/grafana-datasources.yml, infra/grafana-dashboards.yml, infra/dashboards/orrery-observability.json): make tracing-up brings up Tempo (OTLP ingest + storage) and Grafana under the tracing compose profile, with provisioned datasources (Tempo + Prometheus, trace→metric linked) and an Orrery — Agent Observability dashboard (tool call rate, p95 latency, error rate, LLM tokens/s, circuit-breaker state, and a live trace table). New tracing-down target.
• [otel] extra on orrery-core (OpenTelemetry SDK + OTLP gRPC exporter), surfaced at the workspace root as orrery[otel]. Imported lazily — the rest of the package never requires it.
• Tests: 11 new cases in core/tests/test_tracing.py (configure idempotency + disabled path, span enrichment via an in-memory exporter, token→metrics bridge, log/trace correlation), with a hermetic autouse fixture that resets the global provider and the ambient OTEL_TRACING_ENABLED so the suite stays deterministic.

Fixed¶

• make install resolution failure: google-adk[eval] transitively caps litellm < 1.86.0 (via google-cloud-aiplatform[evaluation]), which is unsatisfiable against the litellm >= 1.89.3 floor — no version pair satisfies both. Added a uv override-dependencies = ["litellm>=1.89.3"] entry so the workspace resolves; the cap only guards aiplatform's evaluation path (exercised by make eval), and AgentEvaluator imports cleanly on litellm 1.89.3.

Changed¶

• Pinned GitHub Actions bumped alongside the dependency upgrades: astral-sh/setup-uv v5 → v8.2.0 (ci, docs, release), docker/build-push-action v6 → v7, sigstore/cosign-installer v3 → v4.1.2. setup-uv and cosign-installer are pinned to full versions because they don't publish moving major tags for v8 / v4.
• Docker image now installs the otel extra (Dockerfile): the runtime image is built with --extra postgres --extra server --extra otel, so OTEL_TRACING_ENABLED=true actually emits traces from the shipped container — previously OpenTelemetry was absent from the image and tracing silently no-op'd in deployment.
• Release pipeline hardening (.github/workflows/release.yml): the GitHub Release body is now built into a file and passed via body_path, fixing a long-standing bug where the inline $(cat docker_info.md) was never shell-evaluated and the Docker-images section rendered literally. The Helm kubeconform step now actually validates the rendered manifests (it previously only installed kubectl and validated nothing). Trivy renders the Helm chart against TRIVY_KUBE_VERSION=1.28.0 so it's no longer skipped for misconfiguration scanning.

Security¶

• Three transitive HIGH advisories cleared from the lock (caught by the Trivy CI gate): cryptography 46.0.5 → 49.0.0 (GHSA-537c-gmf6-5ccf — vulnerable OpenSSL bundled in the wheels, fixed in 48.0.1), python-multipart 0.0.28 → 0.0.32 (CVE-2026-53539 — quadratic-time querystring parsing with semicolon separators causing CPU DoS, fixed in 0.0.30), and starlette 1.2.1 → 1.3.1 (CVE-2026-54283 — request.form() size limits silently ignored for application/x-www-form-urlencoded, enabling DoS). pyopenssl 26.2.0 → 26.3.0 came along transitively. Lock-only change; 817 tests still pass.

Documentation¶

• Observability page consolidated (metrics.md, retitled Observability in the nav): metrics and distributed tracing now live on one page — the three-signal overview, metrics reference, and the full tracing guide (the Grafana Tempo trace-waterfall screenshot, what the spans carry, log↔trace correlation, and the make tracing-up local stack + dashboard). config/general.md keeps a compact tracing env-var table that links to it, so tracing is discoverable under Platform Features → Observability rather than buried in configuration. The homepage Observability card and "Observable by Design" principle now mention traces, the CLAUDE.md architecture section gains a tracing bullet, and AEP-010 is marked completed (with an implementation note on how the final design diverged from the original proposal and why exemplars were deferred).
• mkdocs navigation polish: dropped navigation.expand for a cleaner collapsed sidebar and added navigation.footer (prev/next), navigation.instant(+.prefetch), toc.follow, and search.share.

0.1.10 - 2026-06-14¶

Added¶

• Two-root ADK 2.0 architecture (ADR-003): The orchestrator is split into two entrypoints that reuse the same node agents. The interactive root is orrery_chat_agent, a chat-mode LlmAgent that keeps conversation history and routes free-form queries to the six specialist AgentTools + memory, plus an incident_triage_agent AgentTool for single-turn full sweeps. The batch root is orrery_triage_workflow, a deterministic graph Workflow (make run-triage) — parallel health checks → health_join → triage → conditional remediation. A chat-mode agent must be a root (ADK forbids it as a routed graph node) and a Workflow can't be an AgentTool, so the two are separate by design.
• Remediation Subgraph: The self-healing loop is now expressed via explicit graph edges (remediation_actor → remediation_verifier → verify_route), strictly bounded by the MAX_REMEDIATION_ITERATIONS state counter instead of the legacy LoopAgent. The verifier signals success via the mark_remediation_resolved tool (replacing exit_loop / actions.escalate).
• Fail-safe triage routing: triage_route infers severity from the per-system status reports and flags triage_verdict_missing when the LLM skips record_triage_verdict, so a degraded system is never silently routed to "resolved".
• End-to-end graph flow tests: agents/orrery-assistant/tests/test_graph_flow.py drives the real routing nodes through InMemoryRunner (parallel fan-out/join, intent dispatch, bounded remediation loop, missing-verdict fallback) with no LLM credentials required.
• Confirmation walkers traverse graph nodes: the Slack and Google Chat confirmation wiring now walks Workflow.graph.nodes, so guarded destructive tools on graph-node agents still fire interactive approvals.
• Shared reply-text extraction helper: New extract_reply_text() in core/orrery_core/events.py (exported from orrery_core) builds user-facing text from a runner event while skipping ADK thought parts. All four transports (Google Chat, Slack, HTTP, CLI) now funnel through it instead of each iterating content.parts — one place to maintain the thought-filtering rule, so a new transport inherits it for free.
• create_agent(mode=...) passthrough: create_agent now accepts ADK 2.0's mode (chat / task / single_turn). The interactive root orrery_chat_agent sets mode="chat" explicitly so it retains conversation history across turns — previously it reverted to greeting-style replies on follow-up questions because the root's delegation mode was inferred rather than pinned.

Fixed¶

• Planner reasoning leaked into user replies: With a planner active (Gemini builtin thinking, plan_react, or a reasoning model via LiteLLM), the model's thought/reasoning parts were concatenated into the answer shown to users — so Google Chat, Slack, the HTTP /chat API, and the CLI all rendered the chain-of-thought ("Checking Kubernetes Status…") above the actual response. ADK normalizes every form of reasoning onto part.thought = True (native thinking, PlanReActPlanner /*PLANNING*/ phases via _mark_as_thought, and LiteLLM Anthropic/OpenAI reasoning via _convert_reasoning_value_to_parts); all four transports now skip thought parts, so reasoning never reaches the user regardless of provider.
• Stale ADK 1.x references in docs: core/README.md documented the removed create_sequential_agent() / create_parallel_agent() factories — replaced with the graph Workflow composition pattern. agents/orrery-assistant/README.md was rewritten for the ADR-003 chat-root + triage-Workflow architecture (was still showing the old SequentialAgent/ParallelAgent sub-agent tree and missing the Elasticsearch checker and remediation loop). agent-design-patterns.md and a few internal docstrings updated likewise.
• Whole-workspace test collection: switched pytest to --import-mode=importlib and added agents/google-chat-bot/tests to testpaths. The Google Chat bot's 93 tests were previously excluded from the suite (its test_auth.py / test_app.py / test_handler.py basenames collided with other packages under the legacy prepend import mode), so they never ran in CI. The full suite is now 806 tests collected in a single run.

Removed¶

• Deprecated Agent Factories: Removed create_sequential_agent, create_parallel_agent, and create_loop_agent from orrery_core.base as they are deprecated in ADK 2.0 in favor of the Workflow API.
• Legacy Routing Evals: Removed the planner_routing eval that asserted the old root-level LLM routing. The orrery-assistant root no longer has a routing eval — the chat root's specialist dispatch and the triage Workflow's deterministic FunctionNode routing are exercised by unit tests instead.

Security¶

• CVE remediation across the dependency graph: Bumped litellm 1.82.6 → 1.83.14 to address 7 advisories — 2 CRITICAL (CVE-2026-35030 OIDC auth bypass / privilege escalation, CVE-2026-42208 SQL injection) and 5 HIGH (CVE-2026-35029 RCE via unrestricted proxy config, CVE-2026-40217 arbitrary code execution via bytecode rewriting, CVE-2026-42203 SSTI in /prompts/test, CVE-2026-42271 authenticated command execution via MCP stdio test endpoints, GHSA-69x8-hrgq-fjj8 password hash exposure / pass-the-hash bypass). Five transitive HIGH-severity bumps came along for free: mako 1.3.10 → 1.3.12 (CVE-2026-44307 Windows path traversal via backslash URI), pyasn1 0.6.2 → 0.6.3 (CVE-2026-30922 unbounded-recursion DoS), pyopenssl 25.3.0 → 26.2.0 (CVE-2026-27459 DTLS cookie callback buffer overflow), python-multipart 0.0.22 → 0.0.28 (CVE-2026-42561 unbounded multipart-header DoS), and urllib3 2.6.3 → 2.7.0 (CVE-2026-44431 cross-origin sensitive-header forwarding, CVE-2026-44432 decompression-bomb safeguard bypass).
• Trivy filesystem scan in CI (.github/workflows/ci.yml — security job): New aquasecurity/trivy-action@0.28.0 step runs scan-type: fs with scanners: vuln,misconfig,secret against the workspace, gating on HIGH,CRITICAL with ignore-unfixed: true and exit-code: 1. Catches the vulnerable-dependency class that Bandit (a Python static analyser) doesn't see, plus IaC misconfig in Dockerfile / deploy/terraform/, plus committed secrets.
• Bandit now scans every file: Pinned the uvx invocation to --python 3.14 so the AST parser matches our requires-python floor. Before this, GitHub's runner Python (3.12) silently skipped core/orrery_core/rbac.py, core/orrery_core/runner.py, and agents/google-chat-bot/google_chat_bot/pubsub_worker.py with "syntax error while parsing AST" — they parse Python 3.14 syntax — meaning those modules weren't being security-scanned at all. The skip is now caught by the report (0 files skipped) and the modules participate in the scan.
• Hardcoded /tmp removed from the test suite (core/tests/test_secrets.py:117): test_filebackend_satisfies_protocol now takes pytest's tmp_path fixture instead of passing "/tmp" to FileBackend, resolving the Bandit B108 (hardcoded_tmp_directory) Medium finding that was breaking the bandit -ll gate.

Changed¶

• ADK 1.x → 2.0 upgrade: Bumped google-adk to >=2.2.0 and added the [db] extra (ADK 2.0 moved sqlalchemy/DatabaseSessionService out of the base package). run_persistent() / create_app() now accept an Agent | Workflow root. litellm is re-pinned to >=1.83.14,<1.86.0 (ADK 2.2.0 transitively caps it below 1.86 via google-cloud-aiplatform[evaluation]; resolves to 1.85.4, retaining the CVE fixes in this release).
• Workspace pinned to Linux + macOS resolution environments (pyproject.toml — new [tool.uv] block with environments = ["sys_platform == 'linux'", "sys_platform == 'darwin'"]): The project depends on Docker daemon, Kubernetes client, and confluent-kafka — none of which are supported on Windows in the form we consume them. Restricting the lock-file environments lets uv resolve a coherent dependency graph instead of failing on phantom Windows-only conflicts (the litellm bump above surfaced one). No runtime change; this only affects what platform combinations uv considers when locking.
• Relaxed two version floors to satisfy litellm's strict transitive pins: pydantic>=2.13.3 → pydantic>=2.12.5 (litellm 1.83.14 hard-pins pydantic==2.12.5) and aiohttp>=3.13.5 → aiohttp>=3.13.3 on agents/slack-bot (litellm 1.83.10–13 pin aiohttp==3.13.3, 1.83.14 pins ==3.13.4). Resolver picks the highest version that satisfies every constraint; in practice the lock holds at the litellm-mandated floors. Tests at 693/693 after the changes — if you adopt pydantic 2.13-specific behavior later, this floor should be raised back.

Added¶

• Authentication layer for the HTTP front door (AEP-013, first slice) (core/orrery_core/auth.py, core/orrery_core/server.py, core/orrery_core/secrets.py): New verify_token() supports both HS256 (shared secret, for dev/gateway-fronted deployments) and RS256/ES256 via JWKS (production IdPs — Auth0, Keycloak, Okta, Google IAP, GitHub OIDC). JWTConfig.from_env() reads JWT_ALGORITHM / JWT_SECRET / JWT_JWKS_URL / JWT_AUDIENCE / JWT_ISSUER / JWT_ROLE_CLAIM / JWT_LEEWAY_SECONDS. extract_role() maps claim lists or space/comma-separated strings to viewer / operator / admin with namespace aliases (orrery-admin, orrery_operator). New AuthPlugin reads the verified _auth payload from session state and calls set_user_role() so the existing RBAC enforcement becomes trust-rooted. Opt-in via default_plugins(enable_auth=True) — zero behavior change for existing deployments.
• Authenticated FastAPI runner (core/orrery_core/server.py): New create_app() mounts the ADK Runner behind a HTTPBearer dependency. POST /chat requires a valid JWT, seeds new sessions with the verified _auth payload, and re-stamps long-lived sessions so a re-minted token with a downgraded role applies immediately. GET /healthz and GET /readyz are auth-free. auth_enabled=False is supported for local dev with an explicit warning and pins anonymous callers to viewer so RBAC still gates writes. Eager cfg.jwt.validate() at startup fails fast on misconfiguration rather than at first request.
• Pluggable secrets manager (core/orrery_core/secrets.py): SecretsManager resolves in priority order — explicit backends → FileBackend(ORRERY_SECRETS_DIR) → env vars → caller default. Auto-installs the file backend when the env var points at a real directory, so Kubernetes Secret volumes work with no code changes. SecretsBackend Protocol leaves Vault / GCP Secret Manager / AWS Secrets Manager adapters as a future PR.
• Tests: 60 new cases across core/tests/test_auth.py (extract_role matrix, JWTConfig.validate, HS256 round-trip + signature / audience / issuer / expiry / leeway / missing-sub / missing-exp failure modes, RS256/JWKS happy path with a mocked PyJWKClient, AuthPlugin role-application and forced-viewer fallback), core/tests/test_secrets.py (env fallback, file backend trailing-newline handling, backend ordering, misbehaving-backend isolation, auto-registration via ORRERY_SECRETS_DIR), and core/tests/test_server.py (FastAPI TestClient — /healthz is auth-free, missing/invalid/expired tokens return 401 with WWW-Authenticate: Bearer, valid tokens dispatch to the runner with the JWT subject as user_id and the _auth payload seeded, existing sessions re-stamp _auth on every request, anonymous-mode pins to viewer, eager misconfiguration fails fast). Suite at 693 unit tests (was 633), lint + format + ty clean.
• Deps: New optional extras on orrery-core — [auth] (PyJWT 2.10+ with crypto for RS256/ES256) and [server] (auth extras plus FastAPI + Uvicorn). Both surface at the workspace root as orrery[auth] / orrery[server]. Core itself adds no new required dependencies — anyone using only the Slack / Google Chat transports stays on the existing footprint.

Documentation¶

• config/security.md — new page with the threat model, HS256 vs. RS256/JWKS setup recipes, JWKS endpoints for Auth0 / Keycloak / Okta / Google IAP / GitHub OIDC, role-mapping rules, and the ORRERY_SECRETS_DIR Kubernetes Secret-volume pattern. Wired into the mkdocs nav under Getting Started → Configuration.
• .env.example — Security / Auth block filled in with the full variable matrix (AUTH_ENABLED, JWT_ALGORITHM, JWT_SECRET, JWT_JWKS_URL, JWT_AUDIENCE, JWT_ISSUER, JWT_ROLE_CLAIM, JWT_LEEWAY_SECONDS, ORRERY_CORS_ORIGINS, ORRERY_SECRETS_DIR) replacing the previous "coming soon" stub.
• README.md — Safety & Governance bullets added for JWT authentication and the secrets manager.

Added (previously)¶

• Opt-in ADK planners on reasoning-heavy agents (core/orrery_core/base.py, agents/orrery-assistant): New resolve_planner() helper reads ORRERY_PLANNER (none | plan_react | builtin) and create_agent() accepts a planner= kwarg. Three agents in orrery-assistant opt in — the root orchestrator, the triage_summarizer, and the remediation_actor — sharing a single planner instance resolved once at import time. plan_react works across Gemini/Claude/OpenAI/Ollama via the existing LiteLLM integration; builtin consumes Gemini's native thinking tokens (ORRERY_PLANNER_THINKING_BUDGET, ORRERY_PLANNER_INCLUDE_THOUGHTS) and falls back to no planner with a warning when MODEL_PROVIDER != gemini. Default ORRERY_PLANNER=none is a no-op — zero behavior change for existing deployments. Per-system health checkers, the remediation verifier, and the journal writer intentionally skip the planner; they execute one short tool sequence per turn so an extra reasoning pass would only add latency. Tool-leaf agents (e.g., kafka_health_checker, k8s_health_checker) remain planner-free.
• Tests: 10 new cases in core/tests/test_base.py::TestResolvePlanner covering default-off, both planner choices, the Gemini-only fallback for builtin, the ORRERY_PLANNER_THINKING_BUDGET / ORRERY_PLANNER_INCLUDE_THOUGHTS knobs, unknown-value warning, and case-insensitivity. Plus 5 deterministic wiring tests in agents/orrery-assistant/tests/test_planner_wiring.py that reload the agent module across env-var permutations to assert (a) which three agents pick up the planner under plan_react / builtin, (b) that all seven tool-leaf agents — the five health checkers, the journal writer, and the remediation verifier — stay planner-free, and (c) that builtin falls back to no planner under non-Gemini providers. Suite at 633 unit tests (was 618), lint + format clean.
• Agent-level eval scaffolding for the planner-enabled root (agents/orrery-assistant/tests/evals/planner_routing.test.json, agents/orrery-assistant/tests/test_orrery_eval.py): Two routing scenarios — narrow Kafka query and narrow Elasticsearch query — that exercise the full ORRERY_PLANNER=plan_react → root → AgentTool → mocked client path against a real LLM. Gated behind make eval; skips when no Gemini credentials are present. Mocks kafka_health_agent.tools._get_admin_client and elasticsearch_agent.tools._get_session at the same layer the per-specialist evals use. The accompanying test_config.json ships with smoke-test thresholds (0.0) because the AgentTool injects an LLM-generated request arg that defeats strict tool_trajectory_avg_score matching — the eval still proves the agent loads with the planner, the LLM picks a specialist, and the full path runs without errors. Tighten thresholds once an arg-matching strategy is adopted.

Changed¶

• Type hints / lint hygiene to clear make ty after the ty>=0.0.34 bump: agents/elasticsearch/elasticsearch_agent/tools.py adds a # ty: ignore[invalid-assignment] on requests.Session.verify (the type stub narrows it to bool, but at runtime the attribute accepts bool | str | None — a string is treated as a CA bundle path); core/tests/test_operators.py annotates the FakeStrimzi test double with explicit tuple[str, ...] / tuple[CRDRef, ...] types so it satisfies the OperatorDetector protocol; core/tests/test_base.py::TestResolvePlanner.test_builtin_include_thoughts_false narrows via isinstance(result, BuiltInPlanner) before reading thinking_config.

Documentation¶

• config/general.md — Planning section: Env-var matrix and decision guide for plan_react vs. builtin, including the latency tradeoff and the rationale for keeping tool-leaf agents planner-free.
• agent-design-patterns.md — Planning paragraph under Iterative & Feedback Patterns with the list of opted-in agents and a link to the config reference.
• core/README.md — resolve_planner() reference alongside the existing create_agent() table; planner= is now a documented kwarg.
• README.md — feature bullet in Intelligence & Orchestration announcing opt-in planning with the env-var on-ramp.
• .env.example — Planning block documenting ORRERY_PLANNER, ORRERY_PLANNER_THINKING_BUDGET, ORRERY_PLANNER_INCLUDE_THOUGHTS with inline guidance on when to use each value.

0.1.9 - 2026-04-26¶

Added¶

• Progressive progress cards in Google Chat (agents/google-chat-bot): The async response path now posts a live "🔍 Investigating…" Card v2 immediately and PATCHes it in place as the agent run progresses. Operators see the currently executing sub-agent (friendly label — Checking Kafka, Synthesizing findings, …), a tool-call breadcrumb, subsystem health chips (✅/⚠️/❌/⏳) derived from state_delta writes on kafka_status / k8s_status / docker_status / observability_status / elasticsearch_status, an optional remediation panel populated by remediation_action / verification_result / remediation_summary, and an elapsed-seconds footer. Updates are debounced at 800ms and force-flushed on status transitions.
• Structured triage result card: When an incident_triage_agent run completes, the progress card is replaced with a structured Triage Report — severity badge (🟢 healthy / 🟡 degraded / 🔴 critical), one section per subsystem, the triage_summarizer output, and a role-gated Run Remediation instruction (operator/admin only, only shown when overall severity is not healthy) inviting the user to send the Remediate Quick Command, which dispatches the remediation_pipeline in the same session so it reuses the triage report already in state.
• ChatClient.update_message(): New async PATCH helper on the Chat REST client with dynamic updateMask. Tolerates 404/410 (message deleted) by returning None so the progressive-card loop can stop updating silently.
• progress.ProgressTracker: New module that consumes ADK runner events (author, state_delta, function calls) and drives a debounced async update callback. Threaded through _run_agent via an optional tracker= kwarg so the sync path is unchanged.
• Quick Command-based confirmation flow (agents/google-chat-bot): Confirmation cards no longer embed inline confirm_action / deny_action buttons. Operators send the Approve or Deny Quick Commands (appCommandId=1 / appCommandId=2) configured in the Chat API console; the handler resolves the latest pending confirmation in the thread/space and routes accordingly. Quick Commands ride the standard MESSAGE delivery path, sidestepping the inconsistent invokedFunction-button delivery seen in some Workspace Add-on configurations.
• Tests: +6 test_chat_client.py (create/update mask variants, 404 tolerance, 500 raises), +16 cases in test_cards.py (status classification, progress-card shape, remediation panel, result-card severity, role-gated instruction), +7 cases in test_handler.py (progress posted then updated, final card is triage result when chips landed, role-gated remediation instruction, update failure falls back to new message, runtime error replaces progress with error card, run_remediation click dispatches a new run), +5 cases covering the new approve handshake (parent session id reconstructed from state, store-based approved-pending consumption, stale-approval rejection beyond the validity window, unapproved-pending blocks LLM auto-retry, args embedded in synthetic prompt). All 618 unit tests + lint + format clean.

Changed¶

• handler._handle_message_async rewired: Posts the initial progress card before invoking the runner, attaches a tracker that PATCHes the same message on each significant event, and on completion replaces it with either the structured triage card (when chips landed) or the original reply. Non-triage queries retain the previous text + buffered-confirmation-card behavior.
• _handle_app_command_async and _handle_card_click_async rewired: Both Approve/Deny dispatch paths now post a progress card before re-running the runner and replace it with the final reply on completion, so operators get live feedback during the post-approve LLM run instead of a silent ~30s wait.
• _post_async_error now replaces an existing progress card with an error card (build_error_card) instead of appending a new message, so a crashed run never leaves operators staring at a stuck "Investigating…" frame.
• Card-click dispatch recognizes a third invokedFunction: "run_remediation" alongside confirm_action / deny_action. The remediation branch bypasses the pending-confirmation store and spawns a new _run_agent turn with a remediation prompt.
• Approve handshake is now ConfirmationStore-driven (agents/google-chat-bot/google_chat_bot/confirmation.py): The previous design wrote a _gchat_pending_<tool> fingerprint into tool_context.state on the first call and consumed it on the LLM's retry. That state lives on whatever ADK session ran the tool, which for tools invoked through an AgentTool is the sub-agent's ephemeral session — it doesn't propagate back to the gchat parent session. The callback now consults the store directly, matching (thread_or_space_key, tool_name, args_hash) against entries the click handler has marked approved=True. mark_latest_approved_for_thread flips the flag without popping; consume_approved pops on the LLM's retry. A 120s validity window after Approve prevents a stale approval from auto-executing a fresh request later.
• Synthetic Approve prompt embeds the original arguments verbatim: Replaces the previous "Yes, proceed with <tool>." (which forced the LLM to reconstruct args from chat history — fragile when the call originated inside a sub-agent whose args the parent session never saw) with "The operator (<name>) approved the previous <tool> request. Re-issue the same call now with arguments <k=v, …> and report the result." PendingConfirmation now carries the original args dict, an args_hash, created_at, approved, and approved_at fields.

Fixed¶

• patch_deployment (and any other guarded tool on a sub-agent) detoured through remediation_pipeline after Approve (agents/google-chat-bot): When a guarded tool fired from inside an AgentTool (e.g., k8s_health_agent.patch_deployment), the confirmation callback captured tool_context.session.id — which for an AgentTool is the ephemeral inner session, not the gchat parent session keyed gchat:<thread-or-space>. On Approve, the handler re-entered the runner with that ephemeral id and the LLM, finding no conversation history and only the synthetic "Yes, proceed with patch_deployment.", hallucinated a remediation against an unrelated deployment (default/frontend was the most-frequent miss). The callback now reconstructs the parent session id from state["gchat_thread"] / state["gchat_space"] (which the handler writes at the start of every turn) so Approve always re-enters the conversation the operator was in. Verified end-to-end with restart_deployment and patch_deployment against gatekeeper-system/gatekeeper-audit.
• Re-prompting loop on every Approve retry (agents/google-chat-bot): The per-context _gchat_pending_<tool> flag the callback set on the first call lived on the AgentTool's child session and was invisible to subsequent calls (each AgentTool invocation gets a fresh sub-session). On retry the callback never saw a matching pending and posted another approval card — the operator could be stuck approving forever. Resolved by moving the handshake into ConfirmationStore (see Changed above).
• APP_COMMAND payloads with empty message: {} resolved to the wrong pending in shared spaces: Quick Command events don't carry a thread name, so _extract_thread_name returns None and the handler falls back to matching by space name. The handler now logs a warning when this fallback fires so operators can see when concurrent destructive actions in a shared space could collide on the resolution.

0.1.8 - 2026-04-23¶

Added¶

• Elasticsearch Agent (agents/elasticsearch): New specialist agent for Elasticsearch cluster operations, exposing 19 read-only REST tools — cluster health/stats/nodes/pending tasks/settings, indices listing/stats/mappings/settings, shard allocation + explain diagnostics, search + count, index templates, aliases, ILM policies + explain_ilm_status, and snapshot repositories/snapshots. HTTP session is pooled as a module-level singleton with API-key / basic-auth / CA bundle support via ELASTICSEARCH_* env vars.
• ECK Operator Tools: Five Kubernetes control-plane tools complementing the REST surface — list_eck_clusters, describe_eck_cluster, list_kibana_instances, describe_kibana, get_eck_operator_events. Wired through the shared orrery_core.default_registry.ECKDetector for interpreted healthy / phase / warnings on each CR.
• Orrery-assistant integration: elasticsearch_agent is now a sibling AgentTool on the root orchestrator, and a new elasticsearch_health_checker joins health_check_agent as the fifth parallel branch of the incident triage pipeline (writes elasticsearch_status to session state for the triage_summarizer).
• Compose profile + Makefile targets: docker-compose.yml gains an elastic profile with single-node Elasticsearch 8.13.4 + Kibana (security disabled for dev); make run-elasticsearch / make run-elasticsearch-cli launch the agent standalone.
• Tests & evals: 36 new unit tests (25 REST + 11 ECK) with mocked requests.Session / CustomObjectsApi / CoreV1Api, plus 6 new eval scenarios (cluster_and_indices.test.json, eck.test.json) — bringing the suite to 608 tests / 28 eval scenarios.

0.1.7 - 2026-04-19¶

Added¶

• Zero-clone Docker quick-start: The README and getting-started.md now lead with docker pull ghcr.io/bahalla/orrery:latest + docker run for a 30-second single-container test drive, followed by a curl-the-compose-file path for the full Kafka/Postgres/Prometheus stack — no repository clone required.
• ORRERY_IMAGE override: docker-compose.yml honours ORRERY_IMAGE so users can pin a specific release tag (e.g. ORRERY_IMAGE=ghcr.io/bahalla/orrery:v0.1.7 docker compose --profile demo up -d).
• Best-practices & scaling guide: agent-design-patterns.md gains three new sections — tool/agent sizing budgets (sweet 5–15 tools, 3–7 direct children, depth ≤3), framework/model-specific limits (Gemini/Claude/OpenAI/Ollama caps, ADK LoopAgent.max_iterations discipline, context-window budgeting), and a decision guide for when to reach for the A2A protocol (referencing AEP-005) with a four-stage scaling playbook.

Changed¶

• Python 3.14 upgrade: Bumped requires-python from >=3.11 to >=3.14 across the root and all nine workspace packages (core, docker-agent, google-chat-bot, k8s-health, kafka-health, observability, ops-journal, orrery-assistant, slack-bot). Ruff target-version updated to py314. Dockerfile base images switched to ghcr.io/astral-sh/uv:python3.14-bookworm-slim (builder) and python:3.14-slim-bookworm (runtime). CI and release workflows now install Python 3.14. uv.lock regenerated — all C-extension wheels (confluent-kafka, psycopg2-binary, asyncpg, pydantic-core, numpy, tiktoken) resolved to prebuilt 3.14 wheels with no source-build fallbacks. Full test suite (572 tests) passes on 3.14.
• Single production Dockerfile: Merged Dockerfile.prod into Dockerfile so the repository ships one production-ready image. The consolidated Dockerfile adds UV_COMPILE_BYTECODE=1, UV_LINK_MODE=copy, PYTHONUNBUFFERED=1, PYTHONDONTWRITEBYTECODE=1, a BuildKit --mount=type=cache for uv, --extra postgres by default, and folds ownership into COPY --chown= to drop a redundant chown -R layer.
• Compose services pull by default: orrery-assistant and slack-bot in docker-compose.yml now use image: ${ORRERY_IMAGE:-ghcr.io/bahalla/orrery:latest} with build: . retained as a local-dev fallback — first-run users no longer wait for a local build.
• CI/release type-check coverage: .github/workflows/ci.yml and .github/workflows/release.yml now include --extra-search-path agents/google-chat-bot in the ty invocation so that agent is type-checked alongside the others (it was already shipping in the image and in runtime CI).
• Documentation refresh: SECURITY.md, deployment.md, troubleshooting.md, enhancements/aep-011-deployment-hardening.md, and enhancements/aep-014-supply-chain-security.md updated to reference the single Dockerfile after the consolidation.

Fixed¶

• Broken HEALTHCHECK on the default image: The Dockerfile HEALTHCHECK and the orrery-assistant compose healthcheck both probed http://localhost:8080/healthz, but HealthServer is only started by run_persistent() and the Pub/Sub worker — not by the default adk web CMD. Probes are now owned by the orchestrator (docker-compose / Helm values in deploy/helm/orrery-assistant/values.yaml) rather than baked into the image.
• Latent port conflict in compose: orrery-assistant and kafka-ui both bound host port 8080. The unused 8080:8080 mapping on orrery-assistant was removed (the healthz server doesn't run for that CMD anyway).

Removed¶

• Dockerfile.prod: replaced by the consolidated, production-ready Dockerfile. The release.yml workflow now builds from ./Dockerfile.

0.1.6 - 2026-04-19¶

Added¶

• Operator Registry (orrery_core.operators): Pluggable registry for Kubernetes operator detection and CR status interpretation. Ships with built-in detectors for Strimzi (kafka.strimzi.io — 9 kinds incl. Kafka, KafkaTopic, KafkaConnector, KafkaRebalance) and ECK (*.k8s.elastic.co — 7 kinds incl. Elasticsearch, Kibana, ApmServer, Beat). New detectors can be registered via default_registry.register().
• Structured Tool Results (orrery_core.ToolResult): Pydantic model with ok() / error() / partial() factories and remediation_hints for cross-agent composition. Flattens to a backward-compatible dict via .to_dict(), so adoption is gradual and existing tests/tools keep working.
• k8s-health Operator-Aware Tools: Six new tools on the k8s-health agent — detect_operators, list_custom_resources, describe_custom_resource, get_owner_chain, describe_workload, get_operator_events. describe_workload walks ownerReferences from a Pod up to its root CR (e.g., Pod → StatefulSet → Kafka) and returns the operator's interpreted status (healthy/phase/warnings) instead of raw pod info.
• kafka-health Strimzi Tools: Ten new tools that complement the Kafka-protocol tools with a view into the Strimzi control plane — list_strimzi_clusters, describe_strimzi_cluster, list_strimzi_topics, list_kafka_users, list_kafka_connectors, get_kafka_connect_status, get_mirrormaker2_status, get_kafka_rebalance_status, plus the guarded approve_kafka_rebalance (patches strimzi.io/rebalance: approve) and restart_kafka_connector (patches strimzi.io/restart: true). Uses the shared operator registry for status interpretation.
• Property-Based Guardrail Tests: Integrated hypothesis and added exhaustive tests for tool argument hashing in core/tests/test_guardrails.py, ensuring deterministic and order-invariant hashes for stable confirmation matching.
• Pub/Sub Worker Health Probes: The worker now exposes /healthz and /readyz via the shared HealthServer. Readiness flips to 503 if the streaming-pull future dies, so kubelet restarts the pod automatically.

Changed¶

• ADK Upgrade: Upgraded google-adk to v1.31.0 across the workspace.
• Experimental Warning Suppression: pytest is now configured to suppress all experimental feature warnings from google.adk.features, keeping test output clean and focused.
• Helm: Liveness/Readiness + PDB: pubsubWorker deployment now configures liveness, readiness, health port, and an optional PodDisruptionBudget.
• Terraform: DLQ Triage Access: New dlq_subscribers variable grants roles/pubsub.subscriber on the DLQ subscription to configured SRE/on-call groups, plus a dead_letter_subscription_name output.
• Docs: integrations/google-chat-pubsub.md now documents every Terraform variable (chat_publisher_email, enable_vertex_ai, vertex_ai_project_id, tuning knobs) and the timeout-alignment rule for Pub/Sub ack deadlines.
• AEP-018: Proposal for Pub/Sub idempotency (dedup store on eventId) and HPA-on-backlog for the pubsubWorker to remove the single-replica SPOF during incidents.

0.1.5 - 2026-04-18¶

Added¶

• Pub/Sub Diagnostics: Added verbose trace logging for message receipt, parsing, and agent execution to simplify troubleshooting.
• Heartbeat Monitor: Implemented a 60-second background heartbeat log in the Pub/Sub worker to provide "proof of life" in container logs.
• Cross-Project Support: Explicitly documented and enabled support for subscriptions living in different GCP projects than the agent runner via GOOGLE_CHAT_PUBSUB_PROJECT.

Fixed¶

• Cleanup: Ensured proper cancellation of background heartbeat tasks during worker shutdown.

0.1.4 - 2026-04-18¶

Added¶

• Google Chat Pub/Sub Transport: Added support for private GKE clusters via Pub/Sub connection type.
• Terraform Module: New module in deploy/terraform/google-chat-bot for automated GCP infrastructure setup.
• Helm Expansion: Added pubsubWorker deployment and Workload Identity support to the Helm chart.
• Overridable Publisher IAM: Added chat_publisher_email variable to handle Domain Restricted Sharing (GCP Org Policy) for Workspace Add-ons.

Changed¶

• Unified Handler: Refactored Google Chat bot to use a transport-agnostic handler shared between HTTP and Pub/Sub.
• GKE Deployment Story: Removed legacy Kustomize manifests in favor of a unified Helm + Terraform production pattern.

Fixed¶

• Poison Message Handling: Implemented robust ack/nack logic in the Pub/Sub worker to prevent infinite redelivery of malformed payloads.

0.1.3 - 2026-04-13¶

Changed¶

• Rebranding: Project renamed from "AI Agents for DevOps" to Orrery.
• Package Renaming: ai-agents-core is now orrery-core.
• Agent Renaming: devops-assistant is now orrery-assistant.
• Infrastructure: Updated Kubernetes manifests, Helm charts, and Docker images to use the orrery namespace and naming.
• Observability: Prometheus metrics renamed from ai_agents_* to orrery_*.

0.1.2 - 2026-04-12¶

Added¶

• Google Chat Async Mode: Added GOOGLE_CHAT_ASYNC_RESPONSE and GOOGLE_CHAT_SERVICE_ACCOUNT_FILE to .env.example for long-running agent support.
• Enhanced Documentation: Added Google Chat to the main README and expanded the integration guide with troubleshooting for ADC, scopes, and 401/404/403 errors.

Changed¶

• Google Chat Roadmap: Moved Google Chat from "Upcoming" to "Current Integrations" in the documentation.
• API Reference: Registered google-chat-bot for automatic API documentation generation in mkdocs.yml.

Fixed¶

• Google Chat Event Parsing: Implemented robust multi-path parsing for space names, emails, and thread names to prevent 404 errors during asynchronous replies.
• Async Auth Scope Guidance: Added detailed troubleshooting and configuration for 403 Forbidden errors caused by missing chat.bot scopes when using Application Default Credentials (ADC).
• ADC-First Auth Pattern: Updated Google Chat bot to prioritize Application Default Credentials, enabling seamless Workload Identity support on GKE and Cloud Run.

0.1.1 - 2026-04-11¶

Added¶

• Google Chat Bot Integration: A new integration bringing autonomous DevOps agents to Google Workspace with support for thread-based sessions and interactive Card v2.
• Workspace Add-ons Compatibility: Implemented strict hostAppDataAction (DataActions) schema support, enabling the bot to run behind the Google Workspace Add-ons pipeline.
• Dual-Path Event Detection: Added logic to seamlessly handle interaction events from both standard Google Chat API and the nested Workspace Add-ons event structure.
• Interactive Guardrails for Chat: Wired @confirm and @destructive tools to post interactive Cards v2 with Approve/Deny buttons, allowing operators to authorize dangerous actions directly from the chat.
• Configurable Identities: Added GOOGLE_CHAT_IDENTITIES to allow dynamic verification of multiple signing service accounts (e.g., standard Chat vs Add-ons service agents).
• Kafka KRaft Migration: Removed Zookeeper dependency. Kafka now runs in KRaft mode for improved startup reliability and simplified architecture.
• PostgreSQL Service: Added a dedicated PostgreSQL container to docker-compose.yml for persistent session storage.
• Centralized Configuration: Merged per-agent .env files into a single root .env file. Updated core library to prioritize the root configuration while maintaining legacy override support.
• Cross-Session Memory: Enabled MemoryPlugin in the orrery-assistant agent, allowing it to remember past interactions and save session highlights to the persistent store.
• Kafka Partition Scaling: Added update_kafka_partitions tool to the Kafka health agent with full unit test coverage.
• Production deployment hardening (AEP-011) — complete Kubernetes deployment story
• Kustomize manifests under deploy/k8s/ (Deployment, Service, HPA, PDB, NetworkPolicy, ServiceAccount with scoped ClusterRoles)
• Helm chart under deploy/helm/orrery-assistant/ with configurable values, NOTES, and existingSecret support for out-of-band secret management
• GHCR CD pipeline (.github/workflows/docker-publish.yml) publishing multi-arch (amd64/arm64) images with SBOM and provenance attestation on merges to main and v*.*.* tags
• PostgreSQL session store support — runner.py honors DATABASE_URL (async driver postgresql+asyncpg://…) for multi-instance deployments; core[postgres] extra adds asyncpg and psycopg2-binary
• Rate limiting on the Slack bot /slack/events webhook via slowapi (configurable via SLACK_RATE_LIMIT, default 60/minute)
• Root-level .env.example documenting every required and optional variable across agents and deployment manifests
• deployment.md — end-to-end production deployment guide (Postgres setup, Helm install, rolling updates, troubleshooting)

Changed¶

• SlackBotConfig.resolve_db_url() prefers DATABASE_URL env var over the legacy slack_db_url default — enables sharing Postgres between the Slack bot and the ADK web UI workloads
• load_agent_env() and load_config() now search for a .env file at the project root by default.

Fixed¶

• OIDC Token Verification: Fixed a bug where tokens from Google's migrated OIDC flow (iss=accounts.google.com) were rejected. Added proper identity verification against the email claim.
• Dynamic Session Handling: Fixed SessionNotFoundError in the ADK Runner by enabling auto_create_session=True for thread-based chat integrations.
• JSON Schema Validation: Resolved "Failed to parse JSON" errors in Google Chat by ensuring all responses (including errors and added-to-space events) strictly follow the Add-ons response schema.
• Metrics Callback Signature: Fixed a TypeError in MetricsPlugin where incorrect keyword arguments were passed to the internal callback.
• Kafka Tool Imports: Fixed a NameError in kafka_health_agent/tools.py caused by using decorators before they were imported.
• Database URL Masking: Ensured DATABASE_URL is masked in all log outputs and console prints to prevent credential leaks.

0.1.0 - 2026-04-09¶

First public release of the AI Agents for DevOps & SRE platform.

Added¶

• Multi-agent orchestrator — orrery-assistant root agent delegates to 5 specialist agents via AgentTool and deterministic sub-agent workflows (ADR-002)
• Specialist agents — Kafka health, K8s health, Observability (Prometheus/Loki/Alertmanager), Docker, and Ops Journal
• Slack bot — Thread-based sessions with interactive Approve/Deny buttons for guarded operations
• Incident triage pipeline — SequentialAgent + ParallelAgent for parallel health checks across all systems, triage summary, and journal recording
• Closed-loop remediation (AEP-004) — LoopAgent-based pipeline: act (restart/scale/rollback) → verify → retry up to 3 iterations, with exit_loop tool for early termination
• Context caching (AEP-007) — ADK ContextCacheConfig for Gemini models, reducing token usage for repeated requests. Configurable via CONTEXT_CACHE_MIN_TOKENS (renamed to CONTEXT_CACHE_MIN_LENGTH in 0.2.1), CONTEXT_CACHE_TTL_SECONDS, CONTEXT_CACHE_INTERVALS env vars
• Cross-session memory (AEP-003) — SecureMemoryService with automatic PII redaction and size limits
• Agent evaluation framework (AEP-002) — 22 eval scenarios across 4 agents verifying correct tool routing via ADK's AgentEvaluator. Run with make eval
• RBAC — 3-role hierarchy (viewer/operator/admin) enforced globally via GuardrailsPlugin (ADR-001)
• Safety guardrails — @destructive and @confirm decorators gate dangerous operations with args-hash + TTL confirmation tracking (AEP-001)
• Authentication enforcement — set_user_role() marks server-trusted roles; ensure_default_role() forces viewer for unset roles
• Input validation — 5 reusable validators (validate_string, validate_positive_int, validate_url, validate_path, validate_list) applied across 30+ tool functions
• ADK Plugins — cross-cutting concerns as BasePlugin subclasses: GuardrailsPlugin, ResiliencePlugin, MetricsPlugin, AuditPlugin, ActivityPlugin, ErrorHandlerPlugin, MemoryPlugin
• Prometheus metrics — tool call counts, latency histograms, error rates, circuit breaker state, LLM tokens, and context cache events on /metrics
• Resilience — per-tool circuit breaker via ResiliencePlugin, @with_retry decorator with exponential backoff and jitter
• Structured JSON logging — setup_logging() with JSONFormatter, audit trail via AuditPlugin, activity tracking via ActivityPlugin
• Multi-provider LLM — Gemini (default), Claude, OpenAI, Ollama via resolve_model() + LiteLLM
• Persistent runner — run_persistent() with SQLite-backed sessions, health probes, graceful shutdown
• Agent factory functions — create_agent(), create_sequential_agent(), create_parallel_agent(), create_loop_agent()
• Docker deployment — multi-stage builds, non-root user, docker-compose.yml with demo/slack profiles
• 468 unit tests — all async, all mocked, no running infrastructure required
• CI pipeline — lint (ruff), type check (ty), security scan (bandit), tests, evals

Security¶

• Input validation at tool boundaries prevents injection attacks
• Path traversal prevention in Docker and K8s tools
• URL scheme allowlisting rejects javascript:, data:, file: URIs
• Docker container inspection redacts sensitive environment variables
• Guardrail confirmation bypass fixed with args-hash + TTL tracking
• Server-side role enforcement prevents privilege escalation