Changelog

Notable changes per release. Versions follow the vX.Y.Z tags on main.

Releases v0.1.4 through v0.2.7 are not itemized here: the file went unwritten while they were cut. Their contents are in the releases page and in the commit log.

Unreleased

Changed

• The PDF export renders strictly, as every other Jinja template here already did. A name the context does not carry raises instead of rendering as an empty string, and make lint fails on a Jinja Environment built without undefined=StrictUndefined.
• make lint compares a render call's variables with the names its template reads, for the calls where the template name and the variables are both literals. It prints how many calls it could not read that way.
• A scenario's probe and protocol-explanation prompts are rendered by glossogen validate and by the conformance suite. Probe prompts render with each question's own inputs, which is the pairing the metric makes, and explanation prompts render with no variables, as the metric renders them.
• ProbeQuestion and load_probe_questions moved to glossogen.evaluation.metric_core.protocol_probe_config, beside the config that points at the bank.
• The frontend image is built on node:26-alpine instead of node:22-alpine, and CI checks the frontend on Node 26.
• A scenario's round-detail panel receives the round's verdicts as roundResults, the run detail's round_results for that round. SpotTheDifferenceRunExtras.team_results is gone, since it repeated them; read round_results on the run detail instead.
• A scenario declares its own metrics and run-detail extension from its class. scenario_metrics() returns its metric classes and run_detail_extension() its extension. get_available_metric_names is gone, as are _SCENARIO_METRICS and SCENARIO_METRIC_REGISTRY, and a run_detail_extension.py is no longer found by file name, so no platform module names a scenario's parts. A scenario's own metric is now unknown to evaluate on another scenario's run rather than reporting nothing. For a scenario in another package: import RoundResult and PrimaryChannel from glossogen.scenario_base, return any extension from run_detail_extension() and drop its scenario_name class variable, import the extension contract from glossogen.scenario_run_detail_extension, and import AgentDetail and ChannelMessage from glossogen.run_detail_types. An extension module the hook does not return is never imported, so nothing reports it: the run page shows no scenario data.
• Metric.compute takes scenario: ScenarioBase, the part of the scenario contract metrics read. A metric in another package annotates its override the same way. available_metrics takes the scenario class.
• mean_chars_per_round now includes silent rounds represented in the event log as zero-valued rounds. Re-evaluating an existing run can therefore lower this score compared with reports that averaged only rounds containing messages.
• Agents call every tool in-process. The runner builds each agent's tools as pydantic-ai function tools, with the agent's id bound into each, and no MCP server is started for a run. A scenario tool's executor takes agent_id as its first parameter, which the runner supplies, and glossogen validate checks it; get_mcp_tools is get_tools and ScenarioMcpTool is ScenarioTool. A refused call (ValueError) reaches the model as the tool's error however many times it happens; a cycle used to abort after the second. The recorded tool schemas on agent_registered are pydantic-ai's.
• read_notifications is executed by the agent runner instead of the MCP server, so an MCP client other than the platform's runner no longer lists it. A call issued alongside other tool calls is detected from the response that issued it, rather than from how recently another call was dispatched, so a call made on its own right after another tool's result parks.
• Every scenario's agents are offered the changed read_notifications description and its wait_for and timeout_s arguments. An agent parked on any kind of wait counts as idle; a next_round wait is the agent declaring itself done, so a round can end around it with teammates' messages unread.
• A forced send_message over messages the sender has not read leaves them unread, so they still reach the sender.
• PrimaryChannel takes a required includes_direct_channels; a scenario installed from another package must pass it.
• resume-at-round is now fork-at-round, and every fork boundary is the end of a round. The command clones a finished run into a new directory rather than continuing it in place, so it is named as the fork it is. Its boundary moved with the rename: --after-round N keeps rounds 1..N complete, verdict and postmortem included, and plays round N+1 onward. replace-agent and cross-run-replace-agent take the same flags. Old-to-new mapping: --round-start R is --after-round R-1, --rounds-after-swap J (and --rounds-after-resume J) is --rounds-after J+1, and --source-b-round-end now defaults to min(after_round, B_max_round), the same round as before. A completed run can also be forked after its final round by passing an explicit --rounds-after; the resumed clock records the advance as RoundAdvanced(trigger="fork_after_round").

The rename goes through the API and exports: ResumeAtRoundSource is ForkAtRoundSource { source_run_id, after_round, rounds_after, target_event_id, forked_at } under fork_at_round_source, ReplaceAgentSource and CrossRunReplaceAgentSource carry after_round instead of round_start, DerivedRunReference carries after_round / rounds_after and the derivation type fork_at_round, and the CSV lineage columns follow (lineage.after_round and lineage.rounds_after replace lineage.round_start, lineage.rounds_after_swap and lineage.rounds_after_resume). The on-disk replace_manifest.json schema is unchanged, so every previously recorded run reads back under the new API, and the metric name round_success_after_resume stays as recorded in historical reports. Summary caches written before the rename are translated on read rather than invalidated, so listing an existing runs directory does not rescan every JSONL.

Forking got stricter and safer at the edges. A source whose last end marker is not scenario_complete cannot be forked after its final round, since a kill can land mid-round; a scenario_complete that itself landed mid-round (a scenario's is_finished_early hook) is caught the same way, because the boundary round must carry its round_ended (logs recorded before that event existed carry none anywhere and skip the check). Fork clones carry no simulation_ended lines and no inherited derivation manifests, so a fork of a crashed-then-recovered source never trips the simulation_ended evaluation gate early. A round_count carried by --knobs (every shipped preset has one) sets the fork's total rounds when --rounds-after is omitted and must agree with it when both are given, on all three fork flows; it was silently overwritten before, so a full preset passed to replace-agent --knobs alongside --rounds-after now errors unless the numbers line up.

Sources whose log cannot rebuild a seat's history are refused. A cross-run run cannot be forked: its log holds the replaced-away agent's turns before the import boundary, so a fork would seed the seat with the wrong agent. A replace-agent run accepts only a re-replacement of the same seat, because the filters that hid its predecessor's turns live only in its manifest, which clones do not inherit. A boundary behind an already-fired scheduled_events swap is refused for the same reason: the swap's filters and swapped-in model live only in its config and AgentSwappedMidRun event, so the fork would rebuild the seat with its predecessor's turns under the pre-swap model. The same tests guard source B of a cross-run import: a seat that B itself replaced, imported, or swapped cannot be imported from B's log.

--resume on a fork that grew past its boundary anchors at the log's end, keeping every advance, injection, and verdict the crashed launch already recorded, so a crash between the clock's bookkeeping and the first message, or after a round that played with no channel messages, no longer duplicates round events on recovery. A fork with only agent re-registrations past its boundary resumes at the boundary like a first launch. Play means the agents' own event types; scenario and world events the clock flushes while opening a round (a case announcement from on_round_advanced) count as recoverable progress, so scenarios that log round-open events keep their crash recovery. A cross-run fork recovers the same way unless it actually played, in which case it is refused, since the imported agent's post-boundary turns cannot be rebuilt from source B. Crash recovery of a replace-agent fork bounds the seat's seeding filters to the rounds before the fork's entry round, so the replacement keeps its own text, thinking, and channel traffic instead of being stripped like its predecessor, and re-anchors its hidden channels at the boundary rather than the crash point, so the post-boundary messages it had already seen stay visible.

Saved dashboards keep answering: the stores translate pre-rename lineage terms on read (derivation_type = "resume_at_round" to fork_at_round, lineage.round_start to lineage.after_round with numeric bounds shifted to match, the two window columns to lineage.rounds_after, and lineage.resumed_at to lineage.forked_at) and write the translated form back, as does the summary-cache reader, so a legacy file is translated once rather than on every read. - Token costs are priced from the genai-prices catalog instead of a table kept in token_pricing.py, at the rates in effect when the agent registered. A new model or a price change now arrives by upgrading that package. gpt-5.2 was the one model whose old entry disagreed: it is now $1.75 / $14 per million input / output tokens, double the old figure. The models the pickers offer moved to model_catalog.py, without prices. - Four groups of root modules moved into packages. Each module keeps its name, so an import gains the package segment: - glossogen.telemetry: telemetry_settings, telemetry_bootstrap, telemetry_round_processor - glossogen.prod_sync: prod_push, prod_metadata_sync, remote_run_listing - glossogen.run_derivation: replace_agent, cross_run_replace_agent, replace_manifest, cross_run_replace_manifest, run_launching - glossogen.scenario_authoring: scenario_scaffold, scenario_conformance, scenario_package_checks, scenario_target, and the scaffold_templates directory

The scenario contract (scenario_protocol, scenario_base, scenario_loader and the rest) stays where it was. - glossogen.cli is a package. Each subcommand's flags and handler sit in one module (run_command, export_commands, derivation_commands, ...), and main lives in glossogen.cli.cli_entry. glossogen.cli re-exports it, so the glossogen script's entry point and from glossogen.cli import main are unchanged, as are the commands and their flags.

Fixed

• protocol_probe runs on spot_the_difference. Its question bank writes difference_count as a JSON number and ProbeQuestion.inputs accepted only strings, so loading the bank raised before any agent was probed. Quoting the number instead would have broken the template's == 1 branch.
• An exported PDF numbers each turn, from 1 within each round, as the run viewer does. The number slot beside each avatar was blank. The PDF header's message count now counts the channel messages the PDF holds, the run viewer's message count: a channel PDF of a veyru run said "1309 messages", the whole run's messages, text reasoning and tool calls, for its 185 messages.
• A fork after the source's final round (replace-agent, fork-at-round or cross-run-replace-agent with --after-round <final round>) ends its clone before the agents were told the game was over. It ended just before simulation_ended, after every agent's read_notifications had returned the done notice, so the rebuilt agents remembered the game ending: in signal_game the remaining player answered each new round with "the game has concluded" and never moved.
• replace-agent --history-from-round windows a seat that belongs to no channel. The floor was set per channel only, so in a scenario without channels, such as signal_game, the flag did nothing and the replacement kept its predecessor's whole record of tool calls. The manifest records the flag as history_floor_round, and protocol_explanation applies it to the replaced seat's history too.
• A scenario judge under a provider outside JUDGE_PROVIDERS (judge_provider, or any model a scenario's get_judge_models returns) is refused at launch. judge_provider accepts any Provider, and such a judge passed the credential check, then failed at its first call after the run had started.
• The evaluation picker offers only models under a judge provider, read from the new judge_models field of GET /scenarios. It offered every model, and the evaluate endpoint refused the ones under any other provider.
• Each agent cycle's tokens are priced at the rates in effect when the cycle ended: by the runner, which records the run's total, and by the run list, the run detail and the ATIF export, which price the usage logged on a cycle's last response at that response's timestamp. They were priced at the rates in effect when the agent registered or its runner started, which misprices tokens when a rate changes during a run, as genai-prices' time-of-day rates for deepseek-chat do.
• An openai agent is sent reasoning.effort and reasoning.summary only when pydantic-ai marks its model as a reasoning model; a fine-tune is looked up by its base model, the second field of its ft: name. A fine-tune of a model without reasoning, such as gpt-4.1-mini, was sent both.
• A knob the scenario does not declare is refused when a run is configured (--config, a key=value override, a request body) instead of being dropped: BaseKnobs forbids extra keys, so max_round_duration_secs=5 no longer launches a run on the preset's duration. A config a run recorded is read through create_from_recorded_config, which drops a knob the scenario has since removed and logs it, so old runs still evaluate and fork.
• Provider names come from one Provider enum in model_catalog.py, with SIMULATION_PROVIDERS for what an agent, a swap or a replacement can run under and JUDGE_PROVIDERS for what glossogen evaluate and a scenario's judge can. judge_provider, model_overrides[].provider and scheduled_events[].provider are typed with it, so a misspelled provider in a knobs file fails validation before the run starts rather than at the first judged action or at the swap's round. Every --provider flag's choices derive from the same enum; glossogen run --provider google-gla with an agents.* override, and evaluate --provider huggingface, were refused by one check and accepted by another. Ollama models are priced at zero like self-hosted ones instead of logging a pricing lookup failure per call.
• compaction.enabled under a provider the runner has no compaction capability for (anything but Anthropic and OpenAI) is refused at launch; it used to run uncompacted with nothing logged.
• The round-end triggers the game clock writes are a RoundEndTrigger enum (runtime/round_end_trigger.py) that the round-ended metrics and the scenarios compare against, instead of each retyping the strings. A scenario's get_early_round_end_trigger returning one of the platform's values ends the run in error, since the metrics would have counted the round as one the clock ended.
• SendMessageResult.status is a SendStatus enum (sent, conflict, rejected), LLMResponseReceived.stop_reason a StopReason enum (end_turn, tool_use), and the base tool names are constants in communication_tools.py that the runner, the history builder, the history cleanup and the pristine-text index import. The history builder matches read_channel by name rather than by suffix.
• veyru's resume reconstruction decides each team's stabilized from the judged stages; it read it off the run-wide trigger, so a two-team round ended by veyru_mixed_outcome recorded the stabilized team as failed after a fork.
• Warehouse fleet modes are a FleetMode enum; an unhandled mode raises instead of taking the normal-traffic branch.
• textcraft's DeliveryCarrier and its four round triggers are declared once in round_vocabulary.py; LifecycleEntry.type is a NotificationType. The frontend plug-in classifies all four triggers, team_tokens_exhausted included, and the container-yard plug-in classifies its own round_completed / round_failed instead of the platform modal doing it.
• An analysis query naming a group-by, filter or measure key the selection does not carry is refused by name (422 on the API, non-zero exit on the CLI) instead of bucketing every row under an empty string or answering blanks; a gte / lte bound that is not a number is refused by the filter model. A knob filter value that cannot be read as the knob's recorded type (postmortem_enabled=ture, round_count>=abc) is refused on the run listing, the exports and the CLI instead of matching nothing. A CSV export naming a column or metric the selection lacks is a 422 instead of a blank column.
• Fixed-shape dicts read with .get(key, default) are typed values, so a renamed or missing field is an error or a logged skip instead of a silent default:
• Run discovery parses the agent, response, swap and round events it reads; a malformed line fails that run's scan, naming the line, instead of counting its tokens as zero or its provider as unknown.
• Manifest sources read replace and cross-run manifests through their typed readers, so a manifest that does not validate no longer turns a replace-agent run into a fork-at-round in the listing. fork_manifest.json is read through a ForkManifest model.
• prod_push and sync-metadata-to-prod read the remote run listing through RemoteRunPage; a page missing evaluation_content_hash is refused instead of re-uploading every report on every sync.
• A scenario's send_message executor returns a SendReceipt (status, message_id), which SendMessageResult and textcraft's send result extend; the pristine-text index and the history builder read the recorded result through it. Both receipts now serialize status and message_id first.
• The delivered read_notifications payload is a DeliveredNotification model, and the history cleanup reads it and read_channel returns through their models. The JSON is byte-identical.
• The language_repetition sidecar row is a MessageRepetitionRow shared by the metric and the export; the communication and probe sidecar readers validate each row through the model its writer uses and skip a bad row with a log line. The cutoff-trajectory reader looked for similarity where the writer writes mean_similarity, so it read nothing from a real sidecar.
• Normalized agent overrides are ResolvedAgentModel named tuples; the {"model", "provider"} dict and AgentOverridePayload are gone. A replace-agent or cross-run --knobs override with a null provider is refused; it used to be written as the string "None" and fail only in the detached run.
• The supervisor reads scheduled_events, and the CLI and the derived-run listing read replace_agent_default_channel_visibility and round_count, from the validated knobs. replace_agent_default_channel_visibility passed in --knobs applies to the fork.
• textcraft's get_agent_roles validates its knobs, and WorkspaceTaskStarted.manifest is typed as the task model.
• The test harness parses events into SimulationEvent and of_type(event_type=...) takes the event class.
• The run viewer draws the wire between a read_notifications call and its result for a payload the scenario rendered itself. The result chip knew the platform's notification types only, so a scenario's own type left the result entry empty and the wire without an end. A ScenarioPlugin can now render that chip through renderNotification; textcraft's plug-in renders its WorkspaceWake, and a scenario without one gets the payload field by field.
• A failed model request is retried after a pause that starts at 1 s, doubles, and is capped at 30 s. The attempts ran back to back, so a rate limit or an outage exhausted them within milliseconds.
• Text, thinking and tool calls a failed attempt had accumulated and not yet logged are discarded before the retry, so the retried response is logged once rather than appended to the failed one's partial content.
• The event-logging tasks of a cycle cancelled by the supervisor are awaited before the runner returns; they were awaited only for cycles that completed or failed.
• A send_message conflict result lists only the unseen messages the sender may see; it listed messages from before the sender's join point on a channel with restricted history.
• Round injections are pushed to every agent before any delivery is logged, so the agents receive the round's briefing at one point of the event loop.
• Resume matches a delivered round briefing to the same agent's read_notifications result within that round regardless of which event was logged first. The agent can return the read before InjectionDelivered is persisted; treating only later reads as complete sent that briefing again.
• A model request that fails and is retried resumes at that request. The retry restarted the agent's cycle from its first prompt, so the model was asked again from a history missing the tool calls it had already made in that cycle, and could make them a second time. A cycle that fails every retry still restarts from the last completed cycle, so a request that keeps failing is not re-sent.
• A retried cycle's token usage counts every attempt. Only the last attempt's usage reached the cost tracker.
• A round's injection in the run viewer keeps its line breaks when expanded. It rendered as Markdown, which joins consecutive lines, so a line-per-item briefing such as a recipe list ran together on one line. A briefing the two-line preview cuts short now offers "Show full briefing" beneath it.
• An evaluation judged by an OpenAI model no longer bills cached input twice. The Responses API counts cached tokens inside input_tokens, and the judge recorded that count as the non-cached input.
• A resumed or forked agent gets its earlier reasoning back the way a live run sends it. Each llm_response_received event records its thinking part by part with the provider's identifiers (thinking_parts: OpenAI's reasoning item id and encrypted content, Anthropic's block signature), and the rebuilt history carries them, so OpenAI receives reasoning items rather than the summaries as <think>-tagged assistant text. Reasoning a run recorded as text only, before this, is left out of a rebuilt history instead of being sent in that form.

Added

• spot_the_intersection scenario. Each viewer holds a private board, exactly one object is on every board of a team, and every member must submit it; two teams play the same boards and the correct team that said least wins. The team size is the team_size knob. simulated_opponent_ratio plays Team A alone against a simulated Team B, always right in that share of Team A's characters, with Team A reading exactly what it would against a real team (knobs_simulated_opponent). Every round is certified from the lines the viewers read before it is played, and scripts/certify_boards.py and scripts/audit_recorded_boards.py in the scenario package check boards before a cohort and recorded runs after it. The run page's round timeline draws every seat's board with the answer and the near-misses marked, and each team's submissions. See its README.
• glossogen.engine.fewest_characters_winner, the fewest-characters comparison competing teams are ranked by, now shared by spot_the_difference and spot_the_intersection.
• Agents can run under xai (XAI_API_KEY), groq (GROQ_API_KEY), mistral (MISTRAL_API_KEY), deepseek (DEEPSEEK_API_KEY) and openrouter (OPENROUTER_API_KEY), and the run picker offers one priced model for each. pydantic-ai's groq, mistral, openrouter and xai extras are required dependencies, which adds groq, mistralai, xai-sdk and grpcio among the installed packages. PROVIDER_SPECS in model_catalog.py has one row per provider an agent can run under, with its credential variables, pydantic-ai prefix, whether it is billed and its weights; the launch check, the model factory, pricing and the model_class export column read it. groq and deepseek count as open weights, xai as closed, and mistral and openrouter are not classified.
• textcraft_shared_workspace: a team crafts every target of a synthetic layered recipe grid from one finite shared depot, with no target assigned and no coordinator. Team size is a knob separate from the grid, so one agent on the same task is the baseline; messaging, a recipe split across agents, team action and token budgets, and undoing crafts are knobs too. Agents message with send_message(text, to), wait on read_notifications, and read messages inside tool results; a virtual_clock knob orders them by simulated API latency. Judge-free. See its README.
• A scenario can resume a parked agent itself: runtime.release_wait(agent_id, detail) ends the agent's read_notifications with released among its wake reasons and detail as the text, and runtime.parked_waits() and runtime.running_agent_ids() say who is waiting on what. on_agent_parked runs inside the parking call, so a scenario that treats a team parked on each other's messages as a deadlock can release one of them there.
• read_notifications takes wait_for (any, message, next_round) and timeout_s. message parks an agent until a teammate writes or the next briefing arrives, next_round until the next briefing, and no model request is made while it is parked. A scenario renders the result with its own read_notifications(agent_id, wake), for example to deliver message bodies and a world observation in one call, and can end a round's main phase the moment every agent is parked with no deadline, with ends_round_when_all_agents_waiting. Waits log wait_registered and agent_resumed.
• A scenario can replace send_message with send_message_executor, whose parameters become the tool's schema, and post through publish_message. direct_channel_for addresses named teammates on the primary channel whose members they are, or else on a direct channel created on first use and logged as channel_created. A scenario opts into direct channels with validate_direct_channel, which also refuses the pairings it forbids. Direct channels are restored on fork and resume, listed in the run viewer, and scored with a primary channel that sets includes_direct_channels.
• A scenario can replace the runner's system-prompt suffix and its opening and continuation prompts with runner_prompts. The registration records a replacement, so fork, resume, probes and exports render what the run used.
• Scenario hooks for simulated time: on_model_request_started, gate_model_response (awaited before a response's tool calls run), schedule_wait_timeout, clock_now_s, on_agent_parked, on_agent_resumed, on_agent_retired, on_agent_enlisted and on_simulation_stopping.
• hidden_base_tools lets a scenario withhold base tools such as read_channel from an agent, and default_any_wait_timeout_s sets how long a plain read_notifications() waits.
• Filter runs by the values in their scenario_config. Picking a scenario on the runs page offers its knobs as conditions (round_time_budget_seconds >= 200, postmortem_enabled true or false, an enum knob from its own values), each row shows what it recorded for the knobs being asked about, and the footer says what the narrowing cost. The same conditions apply to the CSV and raw exports, to glossogen export --knob and glossogen analyze --knob, and to the export endpoints as the selection's knob array. Which knobs are offered comes from the scenario's own knobs schema, so a scenario installed from another distribution is filterable with no change to the platform. See Filtering by knob.
• In-product analysis: pick a cohort of runs, group and filter it, chart metrics from the evaluation reports, and save the result as a dashboard the rest of the group can open. The same queries run from glossogen analyze with no server and no database, so a chart's numbers are checkable from a terminal. Dashboards live in Postgres when DATABASE_URL is set and in the runs directory when it is not. See docs/analysis.md.

Removed

• The simulation MCP server, its transports, the per-agent tools/list filter and AgentRunner.start's mcp_server_url / mcp_server_object arguments. The run browser's MCP server at /mcp is unchanged.

v0.5.0

Added

• Exporting many runs at once, as raw run folders or as CSV tables, over REST and from glossogen export. See docs/exporting-runs.md.

v0.4.0

Changed

• Authentication is a plug-in, and this repository ships none. Multi-tenancy stays: the groups table, /api/g/{group_slug}/ routing, and the Identity attached to every request. What left is any knowledge of how a caller proves who they are. A deployment supplies that as a distribution declaring one entry point under glossogen.identity_provider.v1, implementing IdentityProvider.

With no provider installed the server is single-tenant: every request resolves to the synthetic local group. That is why there is no built-in implementation of the contract and why nothing in it is optional, and it is also why the server performs no authentication in that mode and should not be exposed to a network.

The middleware keeps the parts a provider must not get wrong. It extracts the bearer credential, parses the URL's group slug, and resolves that slug to a groups row before calling resolve_identity, so a provider never queries that table and cannot get tenancy isolation wrong. It answers one question: does this credential grant access to this group, and as whom. The other direction of the seam is provider_services.py, which exists because that direction was previously undeclared: the one call a provider must make to finish a deferred MCP authorization was reachable only by reaching into request.app.state.

Ambiguity refuses to boot. Two declared providers, or one declared under a contract version this platform does not read, raises. The scenario and metric loaders warn and carry on in the same situation, which is right for them, because a missing scenario is a missing feature. A missing auth provider is a server that authenticates nothing while an operator believes it is protected, and that is indistinguishable from a deployment that never configured one.

Deferred OAuth consent stays platform code: parking a request is about having more than one group to choose from, not about any one vendor, and approve_pending_consent already took a resolved group_id. Only the approval endpoint is pluggable. GET /mcp/whoami stays too, since glossogen whoami calls it in either mode. - The frontend reads identity through an adapter contract, frontend/src/features/auth/auth-adapter.ts, implemented under frontend/src/features/auth/adapter/. The copy here answers "no provider configured" to every slot. Four modules rather than one object, because React's module graph forbids one: readSession needs a server-only import, the provider wrapper is a client component, getSessionToken runs in the browser with no React so api-client.ts stays importable from either side, and the proxy delegate runs in the edge runtime. /sign-in, /sign-up, /select-org and /mcp-consent stay here as shells, because the App Router resolves pages by file path. - CLERK_PUBLISHABLE_KEY became AUTH_PUBLIC_PUBLISHABLE_KEY, and RuntimeConfig gained an auth map collected from every AUTH_PUBLIC_* variable rather than a named field, since the platform cannot know what values a provider needs. The request-time read is unchanged, so one compiled image still serves any environment. Those values reach the browser and are visible in page source; an adapter's secrets stay on the server.

Removed

• clerk-backend-api and svix dependencies, and every Clerk-specific module. A deployment that used Clerk installs a provider distribution supplying it.
• CLERK_* variables from .env.example, docker-compose.yml and the documentation. A provider reads whatever it needs from the environment; the platform reads none of them.

Migration

• groups.clerk_org_id is renamed external_org_id by migration 0005_rename_external_org_id, along with its index and the auto-generated UNIQUE constraint, which ALTER TABLE ... RENAME COLUMN leaves behind. The column holds the id a group carries in whichever provider a deployment configures, so naming it after one made the schema describe a deployment choice.
• A deployment currently relying on CLERK_SECRET_KEY must install an identity provider before upgrading. Without one the server starts in single-tenant mode and authenticates nobody, which is a silent change in posture rather than an error.

v0.3.0

Added

• validate now checks a scenario's events and the hooks its metrics read. Events were covered by nothing at all, and each failure there is silent when it happens: an event_type repeating one the platform or another of the scenario's own answers to shadows one side of the parser, so the run writes fine and reads back afterwards as the other thing; an events module that raises is logged and skipped by discovery, deliberately, so a third-party plug-in cannot stop the platform reading unrelated logs, which also means its author is never told; and an events.py importing glossogen.models.event closes the cycle that module builds its union inside, which is read from the source rather than by importing, since by then the platform has failed to start. On the metric side, a probe or explanation config naming a file that is not there makes every metric in that family report having nothing to measure, which is what a run with nothing to measure reports too. get_judge_models is checked for readability and never compared against the knobs: that comparison was written and removed while the launch check was being added, because a scenario scoring its rounds without an LLM declared the knobs anyway and comparing the two refused runs for a credential it would never spend. What the hook reports is the scenario's to decide.
• A documentation site, built with mkdocs-material and published per release with mike, so a reader who pinned a tag gets the contract that tag was written against rather than whatever main says. make docs-serve previews it and make docs-build runs mkdocs build --strict, which a CI job runs on every PR. The obstacle was that these files are written to be read in the repository: dozens of links point into src/, into tests/, or at a root file, and every one of them resolves on GitHub and 404s on a site. scripts/docs_hooks.py rewrites those to permalinks at build time and adds the repository-root pages to the site from where they are, so nothing had to move and no link had to become an absolute URL in the markdown. Heading anchors now follow GitHub's slug rules for the same reason: a heading holding & or an arrow used to get one anchor in the repository and a different one on the site, so a link written against either was broken on the other.
• Three runnable notebooks under notebooks/, executed in CI. They generate their own simulation through glossogen.testing rather than reading a committed run or asking for one first, so they need no API key and reach no network: the real MCP server, game clock, world and event logger, with only the model scripted. A committed run would go stale against the event schema, and a reader without credentials would be stuck at the first cell. make install-notebooks adds jupyter, pandas and matplotlib as a notebooks dependency group, kept out of dev so the default install and the test suite carry none of it. make test-notebooks executes every cell and fails on the first that raises, and linter/check_notebook_outputs.py in make lint refuses a notebook committed with its output, which otherwise buries the next real diff under regenerated cells.
• docs/quickstart.md, a sequenced path through the platform: run a simulation, read its event log, score it with the metrics that spend nothing, then generate a scenario of your own and validate it. The reference documentation covered all of this and sequenced none of it, so the shortest route in was a 720-line guide read top to bottom. Costs are measured from three real runs rather than estimated: warehouse_robot_recovery is $0.16 to $0.34 at three rounds on haiku, container_yard_stacking $37 to $50 at fifteen on opus, and three identical haiku runs scored 0/3, 1/3 and 0/3.
• linter/check_prompt_templates.py, in make lint, checks the Jinja prompt templates: that each one parses under the environment that renders it, that every {% include %} names a partial in the directory the renderer searches, that nothing is rendered or included in vain, and that every template name in shipped code answers to a file. Nothing read the templates until a run did, and each of those failures otherwise waits until the run directory has been claimed and the agents have connected. Undeclared variables are deliberately left to StrictUndefined at render: scenarios assemble their template variables in helpers, so the set a template renders with cannot be decided from the call site, and a rule that guessed would report the templates that are fine. Prompt-sized string literals in scenario Python are advisory.
• glossogen validate <name-or-directory> replaces check-scenario, and takes either. Given a directory it reads the scenario's declaration out of that tree's own pyproject.toml, so an author's loop is edit then check rather than edit, reinstall, check: every other way into a scenario resolves a name through installed metadata. Given a name it resolves the installed scenario. The two forms cannot be confused, a scenario name being a bare lowercase identifier, so anything holding a dot or a separator is a path. A directory additionally gets four checks that stop meaning anything after installation, because installation is what hides them: package-data not covering the prompts and presets, which leaves an editable install working and the wheel rendering nothing; an entry-point group naming a contract version this platform does not read, which makes a scenario absent rather than refused; a non-empty package __init__, which closes the cycle event discovery runs inside; and a name something else already answers to, which is the one thing the name form cannot report, since it resolves to the scenario holding the name and reports that one as healthy. Needs no API key, and checks no model's reachability: describing a scenario must not require a credential.
• glossogen new-scenario <name> --target-dir <dir> writes a scenario package of your own that already runs: validate passes, pytest passes, and glossogen run completes before anything is edited. Assembling one by hand from the guide had two steps that fail long after the mistake, and both are now written for you: package-data, without which the wheel carries no prompt or preset and only an editable install works, and the entry-point key, which has to equal what name() returns or runs land where nothing looks for them. The generated package pins the glossogen release it was generated against, and carries tests written against glossogen.testing.
• Scenarios and metrics can ship in their own distributions. A package declares a scenario under the glossogen.scenarios.v1 entry-point group and a metric under glossogen.metrics, and both are discovered, listed, runnable and scorable with no change to this repo. Presets and prompts are read from the contributing package. See docs/creating-a-scenario.md and docs/creating-a-metric.md.
• docs/creating-a-metric.md, covering the Metric contract, the empty-list convention, both registration paths, and how to run one.
• glossogen validate <name> builds a scenario from every preset it ships and checks the contract the ABC cannot enforce: agents claiming channels nobody created, tool_names no tool answers to, get_agent_roles disagreeing with the agents that get built, templates that do not render, a config that does not round-trip through its own dump. The checks moved out of the test suite and into the package, because a scenario can ship from any distribution and the tests do not; the repository's conformance suite now runs the same ones over the built-ins. Reports every failure rather than the first, exits non-zero, and needs no API key.
• glossogen.testing, behind a testing extra: the harness that runs a scenario with the LLM replaced by a script. validate proves a scenario builds, but never starts the game clock, so nothing there notices if the world's state machine, the postmortem phase or the round verdict breaks. run_rounds drives the real loop, and the assert_* helpers state what a finished run must contain. metric_harness scores a finished run the way evaluate does, and assert_scenario_is_registered catches the case validating by name cannot, a name that resolves to somebody else's class. All of this was reachable only from this repository's own tests/ directory, so a scenario in another package had no way to test itself without racing the clock. See docs/testing-a-scenario.md.
• glossogen serve --ui-port PORT starts the web UI alongside the API, from the published frontend image, so someone whose scenario or metric lives in their own package reaches it without cloning this repository. The flag wires API_URL, adds the UI's origin to ALLOWED_ORIGINS, waits until the page answers, and removes the container when the server stops. --ui-image pins a version tag, which an older server needs.
• A glossogen console script, so the commands the docs spell as glossogen ... work on an installed package rather than only as python -m glossogen.
• Task-shaped documentation pages under docs/: installation, running simulations, evaluation, scenarios, agent swaps and resume, the web UI, MCP integration, and deployment.

Fixed

• A run whose environment cannot reach a model it would call is refused at the command line, instead of starting and failing once the game clock has run its course. Agent runner tasks are awaited only after the clock finishes, so a missing credential used to surface after round_count rounds of max_round_duration_seconds, from a claimed run directory holding every agent's registration and no model call, with a zero exit status. The check covers the agents' providers, a self-hosted model that SELF_HOSTED_BASE_URLS does not serve, and the model a scenario judges its own rounds with, whose provider its knobs name independently of the run's. replace-agent, cross-run-replace-agent, resume-at-round and the MCP start_run tool check before claiming a directory too. A scheduled swap_agent is checked with the rest: it names its own model and is built at a round boundary, so an unreachable one used to cost every round before the swap, at full price, and then kill the agent it was meant to bring in. Only the boundaries a run will actually cross are checked, so resuming past a swap does not ask for the credential that swap needed: a resumed run inherits its source's whole schedule, and the clock never visits what is below where it opens.
• hospital_bed_assignment_privacy no longer declares judge_model / judge_provider. It scores its rounds by comparing what was transferred against the ground truth and reads neither knob; they were kept "for parity with other scenarios", which is how a scenario with no LLM anywhere came to refuse a run for want of an Anthropic key.
• SimulationScenario.get_judge_models(knobs) is where a scenario states the models it calls itself, and its answer is what the launch check believes. The default reports the judge_model / judge_provider pair; a scenario whose judge is conditional, or which names those knobs differently, overrides it and is not second-guessed, so a configuration that calls no judge is never asked for a credential it will not spend.

Changed

• glossogen check-scenario is now glossogen validate, which takes a scenario name exactly as check-scenario did, and also takes a directory. Two commands differing only in how they found the class read as two different checks, and the second one existed to avoid renaming the first. Update any CI step that calls check-scenario <name> to validate <name>; nothing else about it changed.
• validate renders every round's injection rather than only round one. Round one is not representative: scenarios swap templates per round and bring an agent in partway through, so a template first reached at round 12 used to cost the eleven rounds before it to discover. The failure names the round and the agent. What this still cannot reach is the branch reading a previous round's outcome, since nothing has been played; that belongs to the round loop, and run_rounds(round_count=2) covers it.
• The published images are manifest lists covering linux/amd64 and linux/arm64, each architecture built on a runner of its own and merged afterwards. An amd64-only image made every docker run on an Apple Silicon machine an explicit --platform and an emulated one. serve --ui-port still runs a release published before this, by retrying under emulation when the registry has no image for the host.
• The README is a short hub that links the pages above. It had grown to cover every subject at full depth, which meant the answer to "what is this and how do I run it" was buried. Its project-structure listing is gone rather than moved: it drifted from the tree every time a module was added.
• The metric catalogue in docs/evaluation.md covers every registered metric. The README's list was missing seven of them, including all of the deterministic language metrics.
• Architecture.md's run-directory listing includes the per-metric sidecar files and the live-run stream.json.
• scenario_loader.py is the only way anything resolves a scenario name; nothing reads SCENARIO_REGISTRY directly any more.
• Knobs presets are served by SimulationScenario.knobs_preset_names / load_knobs_preset instead of by globbing a path under glossogen/scenarios, which removed a duplicated helper from the REST and MCP layers.
• SimulationScenario.name() is a classmethod, derived from the same package directory scenario_package_files() resolves.

Fixed

• --config on run, and --knobs on the swap and resume flows, take the name of a preset the scenario ships. Both only accepted a file path, and presets live inside the scenario's own package: a checkout could write src/glossogen/scenarios/veyru/knobs_default.json, while an installed package left no sane thing to type at all. A path still wins when the argument is one, and an argument that is neither is refused by naming the presets that exist. --config is now required rather than silently accepting an empty configuration that every scenario's knobs model rejected a moment later.
• A .env in the working directory is read by an installed package, not only by a checkout. python-dotenv locates the file relative to the module that calls it, which lands in site-packages once glossogen is a dependency, so a project's own .env was ignored and its ANTHROPIC_API_KEY with it. Nothing reported this: the key was simply absent and the run failed later against the provider.

v0.1.3

Added

• A cost model in the README: which knobs drive spend, and the advice to price one run before launching a sweep.
• SECURITY.md, CONTRIBUTING.md, Dependabot, and issue and pull-request templates.
• Releases are cut from a pull-request label. release:patch / release:minor / release:major bump the version, tag it, and publish the images; norelease merges without shipping. New pull requests default to release:patch, and a required check enforces that exactly one label is set.
• docker-compose.yml and a self-hosting section in the README.

Changed

• Deployment and the experimental record moved to separate repositories. The server image no longer carries a plotting stack or a multi-gigabyte ML stack to serve JSON; torch and friends are behind the metrics-ml extra and inspect-ai behind evals.
• Evaluation fails loudly. A metric that cannot run raises instead of returning an empty result, and the runner writes its report before re-raising so the exit status reflects the failure. Metrics that genuinely do not apply to a run still return nothing, as before.

Fixed

• Prompt templates, knob defaults, probe question banks, and judge golden labels now ship inside the wheel. Only .py files were packaged, so installing glossogen as a dependency produced a package that could not render a prompt.
• Concurrent launches no longer truncate each other's logs. Launch output goes to one file per launch under runs/_launch_logs/ instead of a single shared path opened in write mode.
• project.version now tracks the release tag. It read 0.1.0 for the v0.1.1 release, so importlib.metadata.version("glossogen") reported the wrong version. v0.1.1 itself cannot be corrected without moving a published tag.
• Scenarios that ship no frontend plug-in render their timeline against the right primary channel instead of an empty one.
• Image publishing triggers only on vMAJOR.MINOR.PATCH. The previous v* pattern also matched scenario tags such as veyru-stellar-v1.

v0.1.1

Fixed

• Built wheels shipped no runtime data files. Only .py files were packaged, so 139 Jinja prompt templates, 27 JSON knob defaults and probe question banks, and the judge golden labels were missing. Installing glossogen as a dependency produced a package that could not render a prompt. Source checkouts and the Docker image were unaffected, which is why it went unnoticed.

Removed

• AuthGate, a pass-through component that rendered its children unchanged.
• aiosqlite and watchdog, declared dependencies with no imports.

v0.1.0

First tagged release.

Changed

• Repository scoped to the platform. Experiment configurations and analysis tooling moved out, along with 35 of 37 entries in scripts/.
• torch, transformers, minicons, and datasets moved to an optional metrics-ml extra; inspect-ai to evals. Plotting and dataframe libraries dropped entirely. The backend image went from 5.82 GB to 480 MB.
• Frontend configuration is read at request time rather than compiled into the bundle, so one image serves any environment. NEXT_PUBLIC_API_URL and NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY became API_URL and CLERK_PUBLISHABLE_KEY, read at runtime.
• Evaluation fails loudly when a requested metric cannot run. Previously a crashing metric was logged and the process still exited zero, so a broken environment was indistinguishable from a clean run.