Changelog
All notable changes to windupjs are documented here. From 1.0 the project
follows semantic versioning: the public CLI and programmatic API are stable, and
breaking changes wait for a major bump. Format loosely follows Keep a Changelog.
1.9.0
Closing the loop on account profiles: you can now retire one, and the status can no longer imply an identity it doesn’t have.
windup claude logout [--profile <name>] [--remove]. Signing out was the one step still delegated to the raw CLI, and doing it for a profile meant remembering to setCLAUDE_CONFIG_DIRfirst. Now it’s one command: it names the account before dropping it, clears that profile’s stored credential (each config dir has its own — other profiles are untouched), and--removealso deletes the profile’s config dir so a retired client leaves nothing behind.--removerequires--profile: the default~/.claudeis never removed.- The status no longer implies an account it can’t name. A config dir can hold a valid token with no account metadata —
windup claude statusrendered that asready — logged in (pro plan), which looks like an identity for a feature whose entire purpose is telling accounts apart. It now readsready — account email not reported (pro plan)and prints the two commands that repopulate it. NewisAnonymousSession.
1.8.1
Two rough edges the 1.8.0 profiles hit on first real use.
--forcenow rebinds the project too. Pointing an already-bound project at another account (--profile ferawhere.envrcbindsmovepark) stopped with “edit it by hand” — the exact friction the command exists to remove.--forcenow rebinds that one line (every other export preserved) and prints what it replaced; without it the guard still refuses, but it now names--forceas the way out. NewensureEnvrc(…, { replace }).--profile X --forceno longer signs a working account out.--forcemeant “log out, then log in”, so rebinding a project to a profile that was already connected would drop a perfectly good token. With--profile,--forcenow means rebind this project and leaves the profile’s session alone (it reports the account and how to re-authenticate); without--profileit keeps the old meaning — switch the active account.windup claude statuslists the profiles on the machine and marks the one this directory resolves to, so “which accounts do I have, and which is active here?” is one command. NewlistProfiles.
1.8.0
One account per project, for anyone holding a personal Claude plan plus one per client — from a real workflow report.
windup claude login --profile <name>— hold several Claude accounts side by side. The CLI’s login is global (one token, in one config dir), so every project planned on whichever account signed in last: work for a client burned the personal plan. The fix is one command per project:--profile acmegives that account its own config dir (~/.claude-acme, an independent session), binds the project to it by exportingCLAUDE_CONFIG_DIRin.envrc, runsdirenv allow, and only then opens the sign-in. From there,cd-ing into the project makes that account the one that plans — including theclaudeWindup spawns, which inherits the environment. The.envrcis never clobbered: an existing file is appended to (other exports intact), a binding to the same profile is a no-op, and a binding to a different profile stops with the offending line printed and nothing changed. Without direnv, the command prints theexportto run.claude loginno longer claims success when it changed nothing. Already connected, it used to print “already connected” and return — so someone running it to switch accounts was told it worked while the previous account stayed active (the silent version of the exact bug above). It now names the active account and how to switch (--force), and points at--profilefor holding several.--forcesigns out of the current account (saying whose) before signing in again.claude status --profile <name>checks a named account without switching to it, and the plainstatusnow prints the activeCLAUDE_CONFIG_DIRwhen one is set — so “which account will this project bill?” is one command.- Fix:
claude loginwith no TTY failed by hanging. The sign-in is a browser flow driven from the terminal; with no TTY it waited forever, and a killed flow leaves you logged out (found the hard way). It now fails fast with the command to run yourself, matching the guard the install path already had. - New exported
profileSlug/profileConfigDir/ensureEnvrc. A profile name is slugified before it becomes a directory (../../etc→.claude-etc), so it can never traverse paths.
1.7.0
The two rough edges left by the 1.6.0 re-test. The first one inverts the tool’s most important signal, so it led.
- A caught regression is no longer drowned out by the self-heal that follows it (feedback). When a cached plan failed its postcondition, Windup invalidated it and re-planned — and if that re-plan then failed (a truncated response, a bad key), the re-plan’s problem overwrote the original failure. The run read
failure [plan_invalid]: degenerate/truncated response … transient API failure, i.e. “Windup is having an API problem”, when the truth was the app is broken and the tool caught it. The postcondition failure is now kept as the primary result — with its expected/actual intact (postcondition failed: text_contains: body expected to contain "…", got "…") and the correctverificationkind — and the re-plan’s trouble is appended as a secondarynote:line. The tool’s own hiccup can no longer speak louder than the bug it just found. - A truncated generation is retried differently, not identically. Output truncation (not task difficulty) was driving the expensive 3-5 call path on element-heavy pages, and the retry re-sent the same request with the same ceiling — the retry least likely to work. After a truncation, every later call in that planning round raises the output budget (8k → 16k), and the message says what happened and what to do about it (very large prompt → narrow the task or switch model with
--llm) instead of just “transient API failure”. The retry reason in the report distinguishes the first truncation from a repeat. - The planning-cost note in the README now states the retry path explicitly: it exists, and it can cost 10-30× the best case — so a
$0.07line inwindup costsis expected, not a surprise.
1.6.0
Follow-up to the 1.5.0 re-test. The 1.5.0 guard was right but half-done: it turned a false green into no test at all, and it blocklisted tag names instead of looking at the page. Both fixed — and the fix is deterministic, not “hope the model improves”.
- Windup now writes the assertion itself when the task quotes the text (no extra LLM call). The reported failure mode was that the guard rejected the plan, the model produced
bodyagain, and the user ended up with nothing after 3-5 calls and up to $0.09. Now, when the task names a literal in quotes (verify the text 'Estacionamentos Populares' appears) and the plan’s final check is bare visibility, Windup rewrites it intotext_containson that literal — deterministically, before validation, and only after confirming the live page actually contains the text, so the repair can never manufacture a failing test. A landmark selector is dropped; a specific one is kept and the text check is ANDed onto it (strictly stronger). This is the reliable path to a strong assertion: it does not depend on the planner choosing well. New exportedtaskLiteral/assertTaskLiteral. - The guard now counts matches on the live page instead of only blocklisting tags (the
h2hole).expect: { selector: "h2" }sailed past the landmark list, yet on a page with five headings it survives deleting the entire section — andwindup new --validatewas stamping exactly that plan as “validated”. A bare visibility assertion whose selector matches more than one element on the real page is now rejected, with the count in the error ("h2" matches 5 elements on this page — asserting that one of them is visible proves nothing). This uses evidence from the page under test rather than guessing from the tag name, so it also coversdiv,section,li,a,button,p; the landmark list stays as the cheap offline half. The re-plan prompt additionally forbids reusing the action’s owntarget.selector(in all four reported runs the finalexpect.selectorwas identical to it). New exportedbareSelectorAssertion. - Planning-cost docs corrected. “~3s, ~$0.002” is now stated as the best case (simple scenario, first try), alongside the measured hard case (3-5 calls, 30-90s, $0.02-$0.09) and the cheapest way out: quote the text in the task and Windup asserts it for free.
- Validated live on the reported page (five
h2s, one showcase section): the scenario plans in 1 call, passes at $0 on replay, and fails when the showcase is deleted — with fourh2elements still on the page.
1.5.0
Six items from a first-contact report on a real production app. The headline one makes a green run mean something again.
- A trivial postcondition is now rejected at plan validation (feedback — the important one). The planner could end a plan with
expect: { selector: "body" }: it PASSES as long as the page loads, so the test cannot fail — a false green that looks exactly like evidence. (Reported with receipts: the reporter deleted the entire product section and the test still passed; hints spelling out the answer, and a frontier model, did not help.) Validation now rejects a final postcondition that only asserts the visibility — or absence — of a landmark (body,html,:root,main,div,section,#root,#app, …), which routes it through the existing re-plan path with an error that tells the model exactly what to do instead. Any content/value/count/attribute/URL assertion still passes, even on a landmark selector (text_contains: { selector: "main", … }is fine — what it asserts is the text). The planner prompt also gained an explicit “the final expect MUST be able to fail” rule with BAD/GOOD examples. Validated live end-to-end: the reported scenario now planstext_contains: { selector: "main", text: "R$" }on the first call, passes at $0 on replay, and fails when the section is deleted. Note this only affects plans being (re)generated — already-cached plans keep replaying, so nobody’s suite breaks on upgrade;windup explainnow flags a cached weak verification (⚠ weak verification: …) so you can find the ones you already have. New exportedtrivialExpect. - Fix:
windup.config.tsfailed to load when the project hoists jiti v1.c12declaresjitias an optional peer and resolves whatever sits hoisted at the root ofnode_modules— in any Tailwind 3 project that isjiti@1(a direct tailwindcss dependency), which has nocreateJitiexport, so config loading died with a barecreateJiti is not a functionright afterwindup init. Windup now hands c12 its own jiti explicitly, so the hoisted version is irrelevant; if config loading fails for any other reason the error is actionable instead of a rawTypeError. Validated live against a project withjiti@1.21.7hoisted. - Fix: editing
hintsdid not invalidate the cache. Onlytaskwas compared, so hint edits silently replayed the old plan — and hints are exactly how you steer a bad plan, so it looked like they had no effect (the planner was never called). The cache key now carries a hash of the plan-shaping scenario fields (task,hints,atomic_steps,depends_on,like); runtime-only fields (seed,network,clock,failOn,tags,on_dialog, …) deliberately do not invalidate, so editing those still costs nothing. Tolerant by design: entries written before 1.5.0 have no hash and still hit, so upgrading never forces a paid re-plan of a committed cache. New exportedscenarioSig. - A bad model name (or any misconfiguration) is now a
configfailure, not a fake test failure.--llm google:gemini-3.1-proused to reportfailure [plan_invalid]with the provider’s raw 404 JSON — and, becauseplan_invalidis retryable,--retriesrepeated it as if it were a flake. Provider 404s are translated into an actionable message (naming known models for that provider), and anyWindupErrorfrom planning (missing API key, planner CLI absent, bad model) is nowkind: "config"— never retried.windup doctoradditionally warns when the configured model isn’t in the known-model table, catching the typo before a run. NewFailureKind"config". - The planner now says WHY it retried. A re-plan could take 4 calls and a minute with no explanation beyond
llm_calls=4. Each extra call now records a reason (invalid plan: …/truncated response/invalid JSON/network/quota), printed under the run line (planner retried 2× — …) and carried inRunMetrics.plan_retry_reasons. The docs’ “~3s, ~$0.002” figure describes a first plan; a re-plan can cost several calls — now visible instead of mysterious. llm.apiKeyEnvis accepted where the generated config’s comment put it.apiKeyEnvonly worked nested underllm.providers.<name>, butwindup init’s comment sits directly underllm:— so the natural reading was silently ignored, anddoctorreported the key as missing with no hint. It’s now a supported shorthand at thellmlevel (provider-specific still wins), resolved through one shared helper sodoctorand the client can never disagree; the generated comment shows both forms. Lets a project reuse an existing key (e.g.GEMINI_API_KEY) instead of duplicating the secret.windup explainalso renders the postcondition kinds it was silently dropping (text_contains,count,not_visible,attribute).
1.4.0
- The plan cache is versionable by default — replay in CI / on any machine with no LLM and no Claude CLI (feedback). Windup’s whole premise is “plan once, replay $0” — but
windup initused to gitignore the entire.windup/, so the plan cache (the very artifact that makes a replay need no LLM) was never committed, every run was acache=miss, and the suite only ran where the planner’s CLI/key was available (never CI, never a fresh machine). Nowinitwrites an internal.windup/.gitignorethat ignores only the ephemeral/sensitive dirs —state/(auth cookies),runs/(ledger),reports/— leaving.windup/cache/(plans) and.windup/map/(site map) committed. Commit.windup/and CI replays every scenario at $0 with no LLM and no CLI. Plans are portable and secret-free (selectors +value_ref, never resolved values), so committing them is safe. Existing projects: if your root.gitignorehas a blanket.windup/, narrow it to.windup/state/,.windup/runs/,.windup/reports/, then commit.windup/cache/. - Actionable message when the planner can’t run on a cache miss. The “claude CLI not found” error now spells out the two ways forward — commit the plan cache from a machine that has the CLI (replays then need neither), or use a non-interactive planner (
--llm google/--llm openaiwith an API key, or theclaude-code-openai-wrapperviaWINDUP_CLAUDE_CODE_URL) — instead of just “install the CLI”. - Validated live: a scenario whose plan is committed replays
cache=hit,llm_calls=0,$0under aclaude-codeprovider without the CLI on PATH (the planner is never invoked);initwrites the granular ignore and never blanket-ignores.windup/.
1.3.0
- Fix: a per-scenario
networkstub is now excluded from thefailOngate (feedback). The docs promised “requests answered byconfig.networkare always excluded — a deliberate stub is not a real failure”, but that only held for the 5xx response listener — the console “Failed to load resource: … 500” error a stub produces was never checked against the stubs, so--fail-on-resourcefailed the very error the scenario asked for (erro-lista-500FAILed with1 resource error(s)even though its assertion passed). The stub exclusion now also covers console errors, matched by URL (a console error carries no HTTP method) against the run’s effective rules (global + per-scenario merged) — so error-state scenarios and the gate finally compose. Fixes both the per-scenario stub from 1.0 and the same gap for global stubs. NewstubMatchesUrl. - Per-scenario
failOn(feedback). A scenario can carry its ownfailOn, merged over the global ({ "failOn": { "resourceErrors": false } }or a scenario-only"ignore"): the boolean gates take the scenario value when set, andignorelists concatenate (global noise + scenario-specific). So an exception you need for one scenario stops being a blindness across all of them — the threeignoreentries that each existed for a single scenario move onto those scenarios. NewScenario.failOn;effectiveFailOn. - HTTP status in diagnostics (feedback). A resource error now records its numeric
status(console_errors[].status: 404) parsed from the Chromium message, so you can split client 4xx from server 5xx without regexing the string. NewresourceStatus. - Validated live: a per-scenario
{ url: "v1/passports", status: 500 }stub no longer tripsresourceErrors(its console error is excluded by URL) while an unrelated Gravatar 404 still surfaces withstatus: 404, and a scenario-onlyignore: ["gravatar"]silences just that run.
1.2.0
windup record: durable output — stable selectors + a readable task (feedback). Recording a long flow worked, but the artifact didn’t survive a cache invalidation, for two reasons — both fixed:- Selectors are now unique-checked at capture time, with an accessible-anchor ladder. Before, a target with no
id/data-testid/namedegraded to a barediv/button/a(matches the wrong element), and the text fallback grabbed the concatenatedtextContent—"1Seu carrinhoR$ 35,00Ver carrinho"— which breaks the moment a count or price changes. Now each candidate (#id → [data-testid] → [name] → [aria-label] → [placeholder] → clean unique text) is accepted only if it uniquely identifies the element on the page;aria-labelis a new anchor; text is used only when it’s short, carries no dynamic value (no counts/prices), and is unique; and text is read from the element’s own direct text, not its descendants’ concatenation. When nothing stable is unique, it falls back to a short structural path (…:nth-of-type(n)) and flags the interactionunstable. - Unstable interactions are reported. After recording, Windup prints the interactions with no stable anchor (
⚠ N interaction(s) have no stable anchor …) — the same spots a screen reader would struggle with — so you know exactly where to add adata-testidor hand-edit the selector before the scenario enters a suite. - The
--no-llmtask is synthesized from the visible labels, not an interaction count. Was"Recorded flow: 14 interaction(s) ending at /checkout"— semantically empty, so a self-heal after a cache invalidation re-planned blind. Now it readsRecorded flow: click "Ver ingressos" → click "27 R$ 20" → fill "Quantidade" → click "Ver carrinho" → click "Continuar", verifying "Continuar" (ends at /checkout/identificacao)— a description the LLM can actually re-plan from (consecutive repeats collapsed, capped for long flows). New exportedsynthTask. recordunder a PTY is documented/accepted. The no-TTY error now points at the PTY workaround (script -q /dev/null npx windup record) so agents/wrappers can drive it.
- Selectors are now unique-checked at capture time, with an accessible-anchor ladder. Before, a target with no
1.1.0
- Runtime diagnostics: match
failOn.ignoreby URL, split resource errors from JS errors (feedback). Two real gaps surfaced trying to run--fail-on-consoleas a permanent CI gate against an app with noisy Gravatard=404avatars:ignorenow matches the request URL, not just the console text. A failed sub-resource logs the generic Chromium string —"Failed to load resource: the server responded with a status of 404 ()"— with no URL in the message, soignore: ["gravatar.com"]never matched and the noise was un-silenceable. Windup now reads the error’s originating URL (console.location().url) and matchesignoreagainst the message OR the URL — soignore: ["gravatar.com"]finally works. (Bonus fix:ignorewas never applied to console errors at all before — only to 5xx responses.)- Each console error now carries its
urland akind.RunMetrics.diagnostics.console_errorschanged shape fromstring[]toArray<{ message, url?, kind: "js" | "resource" }>— the URL is right there in the report/JSON (no more grepping the app for the<img onError>), and errors are classified:resource= a sub-resource that failed to load (the noisy 4xx kind),js= an uncaught exception, aconsole.error, or a CSP violation. - New
failOn.resourceErrors/--fail-on-resource, andconsoleErrorsnow gates JS only. The two noise classes have separate gates:--fail-on-console(orconfig.failOn.consoleErrors) fails on JS errors only — so it no longer drowns in broken images, and still catches the CSP/JS problems worth catching;--fail-on-resource(orconfig.failOn.resourceErrors) gates the resource 4xx loads when you want them. Behavior change:failOn.consoleErrorspreviously also counted resource-load errors; it now counts JS errors only. - Validated live: a page with a 404 image + a JS
throw— the image error is captured with its URL andkind: "resource", the throw askind: "js";ignore: ["/avatar"]silences the resource error (by URL) without touching the JS one. Newdiagnostics.ts(classifyConsoleError/matchesIgnore),Browser.ConsoleError.
1.0.0
First stable release. The API and CLI surface built across the 0.x line are now committed to under semver. No feature changes over 0.60 — this is the stability milestone plus one security hardening found in the pre-1.0 review:
- Security — scenario-id path-traversal fixed. A scenario’s
scenario_idbecomes a file path (its.json, its trajectory-cache entry).loadScenarionow rejects an id with..path segments or an absolute path (subfolder ids likeauth/loginstay valid) — both on the lookup id and on thescenario_idfield read from disk. Without this, running an untrusted windup project (windup run --allon a cloned repo) whose scenario declaredscenario_id: "../../…"could write a cache file outside.windup/. NewassertSafeScenarioId. - The 0.x line delivered: deterministic
$0replay (LLM plans once, Playwright replays), self-healing re-plan,depends_on+ session snapshots, cross-browser + device emulation,config.network/clock(global and per-scenario), richerexpectasserts, runtime-health gates (console/5xx, web-vitals budgets), CI guard-rails (--retries/--max-wall/--bail/quarantine), the read-only diagnostics family (why/explain/diff/badge/trends), coverage →suggest-scenarios, and authoring bywindup neworwindup record(by demonstration). Secrets never enter scenarios, plans, the cache or git.
0.60.0
- Per-scenario
network&clock(feedback). The determinism knobs added in 0.54 lived only inwindup.config.ts(global), so a request stub applied to every scenario that hit the endpoint — you couldn’t force a500on one listing to test its error UI without breaking the scenario that reads the same list normally. Now a scenario can carry its ownnetworkandclockin its JSON, merged over the global config with the scenario winning:{ "scenario_id": "erro-lista-500", "network": [{ "url": "v1/passports", "status": 500 }] }stubs that endpoint only for that run — global stubs still fall through, and the next scenario’s context is untouched.clockmerges field-wise (scenarionow/timezoneoverride global). Both are applied at context creation (never cached — each error scenario has its ownscenario_id, so its plan is planned against the stubbed page). A scenario carrying an override opts out of browser prewarming (the prewarmed context only knows the global config); everything else is unchanged. Validated live: the same page renders data under a global200stub, the error banner under a scenario500override merged over it, and data again in the next context (no leak). NewScenario.network/Scenario.clock;effectiveNetwork/effectiveClockmerge helpers;launchBrowser({ network, clock }).
0.59.0
windup record— author a scenario by demonstration. Opens a headful browser at your app; you click through the flow, mark the verification with a floating toolbar (”◉ marcar verificação” then click the element to assert), and hit ”■ finalizar” (or Ctrl-C). Windup writes the scenario file and caches the recorded plan, sowindup run <id>replays it $0, no LLM — and if the cache ever invalidates, it re-plans from the task like any scenario. The inverse ofwindup new: show it instead of describing it. Captured selectors mirror the engine’s own priority (#id → [data-testid] → [name] → type → role/text) with an accessibledescriptionfallback; a typed password never lands in the plan — it’s registered to.env.local(gitignored) and emitted as avalue_ref. Verification is either the element you mark (visible / text-contains) or, if you mark nothing, the final page’s URL. It’s a local dev tool (interactive, headful) — needs a TTY, not CI. Validated end-to-end: a real login recording replays at $0 (cache hit) with the password as avalue_ref. New modulerecord.ts;Browser.startRecording;writeScenarioFileextracted from authoring for reuse.
0.58.0
Runtime realism — test at real device viewports and gate on real page performance. Both tested and validated live (real-browser).
- Device emulation —
run --device "<preset>"/config.device. Run a scenario at a Playwright device preset ("iPhone 14","Pixel 7","iPad Pro 11", …) — viewport, user-agent, scale, mobile/touch. Cached plans are keyed per device, so mobile and desktop keep separate trajectories and never overwrite each other’s plan (with no device the cache path is unchanged). Mobile emulation needs chromium. Validated live: the same page reports a 390 px touch viewport underiPhone 14and 1280 px non-touch without it. - Web vitals + performance budgets —
run --web-vitals/config.budgets. Capture the final page’s TTFB / FCP / LCP / DCL / load / CLS (via buffered PerformanceObservers injected before first paint) and report them (console, HTML, JSON) — informational under--web-vitals. Setconfig.budgets({ lcp_ms, cls, load_ms, … }) and a breach fails the scenario with a newbudgetfailure kind. Timing varies run-to-run, so set budgets with headroom. Validated live:load_ms: 1fails with “budget exceeded”, a generous budget passes, and capture-only records the metrics.
New modules device.ts / vitals.ts; config.device / config.budgets; RunMetrics.web_vitals; FailureKind "budget"; Browser.webVitals(); per-device cache keying.
0.57.0
Two more from the backlog — a ledger-history view and a flaky-quarantine gate — each tested and validated end-to-end.
windup trends [scenario]— historical pass-rate, cost and duration per scenario, straight from the run ledger (no LLM). No argument → a per-scenario table sorted worst pass-rate first (problems at the top) with a pass/fail sparkline of recent runs; a scenario id → its runs over time (chronological),--last Nto trim. Joins the read-only diagnostic family (why/explain/diff/badge), all built on the sharedledger.tsreader.- Flaky quarantine —
scenario.quarantine: true. Mark a known-flaky scenario and it still runs and reports, but its failure does not fail the suite (non-zero exit) — so one persistent flake stops blocking the build while you fix it, instead of the usual bad choices (delete the test, or let it redden every run). Surfaced, never hidden: a🔶 N quarantined scenario(s) failed but did NOT fail the buildline on the console, aQUARANTINEDbadge in the HTML report, andquarantined: truein the JSON. NewScenario.quarantine,RunMetrics.quarantined.
New modules trends.ts; Scenario.quarantine / RunMetrics.quarantined; quarantinedScenarioIds().
0.56.0
Verificação mais forte + inteligência de cobertura — três recursos, cada um testado e validado ao vivo (real-browser).
Asserts ricos no expect. A verificação de uma ação vai além de selector/url/selector_value: text_contains ({ selector, text } — o texto do elemento contém a string), count ({ selector, equals?/min?/max? } — quantos elementos batem), not_visible (um selector sumiu/oculto — o negativo de selector) e attribute ({ selector, name, value }). Combinam entre si (AND); o planner conhece os novos tipos e o schema (Ajv + Gemini) os valida. Validado ao vivo: um expect com os quatro juntos passa numa página real; mutações falham com a failed_condition exata.
Console/rede como falha — run --fail-on-console / --fail-on-5xx. O Windup passa a observar passivamente o runtime: um cenário pode “passar” enquanto a página logou um erro de JS ou recebeu um 5xx silencioso. Erros de console + exceções não capturadas e respostas 5xx são sempre registrados (RunMetrics.diagnostics, mostrados no console/HTML/JSON, como o --a11y); e falham o run (kind diagnostics) quando --fail-on-console/--fail-on-5xx (ou config.failOn.{consoleErrors,http5xx}) estão ativos. Stubs deliberados do config.network (ex.: um {status:500}) e URLs em config.failOn.ignore não contam. Uma falha diagnostics nunca invalida o cache nem re-planeja, e não é retentável. Validado ao vivo em todos os caminhos, incluindo a exclusão de stubs.
windup suggest-scenarios. Das rotas do site map sem cobertura (via windup coverage) + LLM, propõe (escreve) os cenários faltantes — uma chamada por rota, reusando a maquinaria do windup new (id único, start_url validado, segurança de credenciais, gasto no ledger como authoring). --limit N, --force, --dry-run (lista sem chamar LLM/escrever), --llm, --json. Fecha o ciclo scan → coverage → suggest: sai de “o que falta” para “aqui está o rascunho pra revisar”.
Novos módulos suggest-scenarios.ts; config.failOn; RunMetrics.diagnostics; FailureKind "diagnostics"; Browser.{textContent,count,getAttribute,waitForHidden,consoleErrors,failedResponses}; campos de Expect text_contains/count/not_visible/attribute.
0.55.0
- Browser prewarming — the next scenario’s context launches off the critical path (on by default;
--no-prewarmto disable). In a sequentialrun --all, while a scenario runs (navigation + actions), Windup pre-creates the freshBrowserContext+ page the next scenario will use, so that scenario no longer waits ~200 ms onnewContext/newPage. Isolation is identical to a per-scenario launch — every scenario still gets its own clean context; only the launch moves off the wait (a scenario’ssetupsegment drops to ~0 in the breakdown, verified live). Safe by construction: the prewarmed session is a one-shot — a--retriesre-attempt and the session-snapshot (storageState) fast path each launch fresh, and an unused warmed session (a--bail/--max-wallearly stop) is closed. Only sequential runs prewarm (--concurrency > 1already overlaps launches across workers). This is the deliberate, zero-risk answer to “context pool”: measurement showed reusing a live context saves only ~18 ms and would leak state between tests, so Windup prewarms a fresh one instead. NewRunOptions.prewarmed.
0.54.0
Diagnostics & determinism — a batch of tools that read what Windup already knows and two config knobs that make hard-to-reproduce states testable. Every command is LLM-free; each feature is unit-tested and validated (real-browser for the two that touch the page).
Four read-only commands (zero LLM, straight from the ledger/cache):
windup why <scenario>— one place for a scenario’s whole story: is a plan cached and ready to replay ($0) or will the next run plan; re-plan churn (a stability signal); thedepends_onchain; run history (pass rate, avg cost/time); and the last run with its failure kind/message and whether a snapshot is stored. Turns “why is this red/slow/re-planning?” from a ledger grep into one line.windup explain <scenario>— the cached plan as readable steps (go to /login · fill one-time code with {otp_code} · click Place order ↳ verify #confirmation is visible). Review a plan without opening the JSON. A fill’s value is never shown (avalue_refrenders as its name) — secrets/OTP stay out.windup diff <scenario>— compares the two most recent runs: result flip, cache, and Δ time / Δ cost / Δ actions. Catches “this scenario got 2× slower” or a plan that quietly grew.windup badge [--json] [--out <path>]— a suite-status badge from each scenario’s latest run: a self-contained SVG (271/271 passing · $0, no external fetch — safe to commit) or a shields.io endpoint JSON.
Determinism (config, applied every run, never cached, author-declared):
config.network— request stubbing: match by URL (substring or glob) + optional method, then respond with astatus/body/jsonorabortthe request — test a 500, an empty list, a slow or failing endpoint without touching the backend. First match wins. Validated live: the same page renders 3 (real) → 0 (stub[]) → error (abort).config.clock— pin the page’s time:nowfreezesDate/Date.now()to a fixed instant (injected before any page script, transform-proof) andtimezonesets the browser’s IANA zone (native Playwright) — so “orders from today” or a countdown stops drifting at midnight. Validated live:now: "1999-12-31…"→ the page’snew Date()reads 1999-12-31.
CI guard-rail:
run --all --bail— stop starting new scenarios after the first failure (fast PR-check feedback). Completes the trio with--retries/--max-wall; works sequentially and under--concurrency(shared halt), prints⏹ --bail: stopped after the first failure — X/Y ran, Z not started.
New modules why.ts/explain.ts/diff.ts/badge.ts/ledger.ts/network.ts/clock.ts; config.network/config.clock; a windup doctor config check.
0.53.0
Resilient-CI pair — turn a flaky suite green without hiding the flake, and cap how long a suite may run. Both tested (unit + real-browser live) and LLM-free on the happy path:
- Retry a flake —
run --retries N. Re-run a scenario that failed a transient way (network reset, a hydration-race verification miss, a wobblysetup/dependency) up to N extra times; the first pass wins. Aforbiddenblock is never retried — aconfig.forbidguard is a deliberate stop, not a flake. Crucially the flake is surfaced, not swallowed: a scenario that only passes on a retry is flaggedflaky(↻ N passed only on retryin the console, aFLAKY N×badge in the HTML report,flaky/attemptson the JSON record and therun:endstream event) so you fix the root cause instead of laundering it green. Validated live: a page whose first connection is dropped fails attempt 1 (network) and passes attempt 2 — recovered, marked flaky, zero LLM calls. NewRunMetrics.attempts/RunMetrics.flaky. - Time budget —
run --all --max-wall <seconds>. A CI guard-rail: once the suite’s wall-clock crosses the cap, Windup stops starting new scenarios (in-flight ones finish — no work is cancelled mid-run) and exits non-zero so a runaway suite fails the build instead of hanging the runner. Works in both sequential and--concurrencymodes (the pool stops pulling new jobs). The console reports⏱ --max-wall Ns exceeded — X/Y ran, Z not started. Validated live: a 3-scenario suite under a 0.7 s cap ran 1, skipped 2, exit 1. - New
runWithRetriesand ashouldStoppredicate onrunPool.
0.52.0
Three items from the beta round (feedback #8), in the requested priority order — each tested and validated live:
- Accessibility label fallback + a11y gap report (#2.2). When the plan’s CSS selector misses (the model guessed one for a control with no stable anchor), the executor now retries the target by its accessible name — matching
descriptionagainst the page’sgetByLabel/getByPlaceholder/getByRole("textbox"), and acting only when exactly one visible field matches (never a guess). The recovered step is marked in the report with a≈ found "<label>" by label (plan selector "<sel>" missed)note, so a run that leaned on the fallback is visible rather than silent. When neither the selector nor the label resolves, the failure message now names the likely cause — “the control likely has no accessible label (a11y gap) — anchor it with a hint” — turning a dead end into an actionable a11y finding. Validated live: a fill whose selector was#wrong-guessed-selectorrecovered via the “Measurement ID” label and passed, with the note surfaced. - Mandatory step granularity —
atomic_steps(#2.1). Set"atomic_steps": trueon a scenario and the planner is instructed to emit one interaction per action — never merging a reveal/expand click with the action it uncovers (e.g. “open the menu, then click Delete” becomes two steps, not one). Keeps the replay debuggable and the report readable when a UI hides controls behind disclosure. NewScenario.atomic_steps. - Per-scenario dialog default —
on_dialog(#1b). Set"on_dialog": "accept"(or"dismiss") on a scenario and a persistent dialog handler is installed for the whole run — every nativeconfirm()/alert()/beforeunloadis answered automatically, no per-actiondialogfield needed. Complements the existing plan-leveldialogon a single click; whenon_dialogis set it wins (the per-actionarmDialogno-ops). Validated live: a page with two separate delete buttons, each firing its ownconfirm(), cleared both rows under a singleon_dialog: accept. NewScenario.on_dialog. - New
Scenario.atomic_steps/Scenario.on_dialog,ActionMetrics.note, andBrowser.{clickByDescription,fillByDescription,isVisibleByDescription,setDialogHandler}.
0.51.0
Three CI features (from the roadmap brainstorm), each tested and validated live:
- Trace + screenshot on failure —
run --trace. When a scenario fails, save a Playwright trace (.windup/reports/traces/<id>.zip, openable in the trace viewer — DOM snapshots, network, console per step) plus a full-page screenshot next to the report; the HTML report links both from the failed row. You can finally see what happened in CI instead of reading ms numbers. (Trace is captured only on failure; a passing run discards it, no overhead kept.) - Scenario tags —
run --all --tag <names>. Tag scenarios ("tags": ["smoke", "checkout"]) and run a subset:--tag smoke,checkoutruns any scenario carrying one of those tags. Run smoke on every push, the full suite nightly — composes with--shardand--changed. NewScenario.tags. - GitHub Actions output —
run --github(auto-on whenGITHUB_ACTIONS=true). Emits a::error::workflow annotation for each failed scenario (shown inline on the PR) and appends a Markdown suite summary + per-scenario table to$GITHUB_STEP_SUMMARY(shown on the job page). Newgithub.ts. - New
ActionMetrics→browser.saveTrace()/screenshot(),RunMetrics.artifacts.
0.50.0
- Readable action table — see WHAT each step did (feedback). The per-scenario action list showed
a1 · 76 ms · 2 mswith no way to tell whata1was. Each action now carries itstype(goto/click/fill/wait_for/use) and alabel— the target’s description or selector, the goto URL, or= {ref}for a resolved fill — so the HTML report readsa4 · fill · otpinstead of an opaque id. A fill’s VALUE is never shown (secrets/OTP stay out — the label is the field description, and avalue_refrenders as its name, not its value). NewActionMetrics.type/ActionMetrics.label.
0.49.0
- Make
resolvereliable — deterministic field binding (config.resolveFields). The 0.47resolvemechanism worked, but it depended on the planner emittingvalue_reffor the right field — which the LLM did unreliably (filling a literal, so the resolver never ran; or a mis-cased name, so the plan failed validation). Now bind the field yourself:resolveFields: { "[name=otp]": "otp_code" }(a selector substring → resolver name). The executor fills any matching field from that resolver, overriding whatever the plan put there — so an OTP/token flow is deterministic regardless of what the model planned. Validated live: a cached plan that fills a literal000000into the OTP field fails without the binding and passes with it (the executor substitutes the real code). Plus two robustness fixes:value_ref/url_refnames are normalized (OTP_CODE/otp-code→ a declaredotp_code) and the schema tolerates that casing (no moreplan_invalidon a stray form); and agotowithurl_ref(a resolved URL, no literalurl) now passes validation. Newconfig.resolveFields.
0.48.0
- Report honesty: separate active work from concurrency contention (feedback #2). Under
--concurrency N, a scenario’s per-case breakdown was dominated by an opaque “other” bucket that is really the scenario waiting for a CPU/browser slot while siblings run — idle time, not cost (a “19.7 s” archive was ~0.4 s of work). That leftover is now labeledcontention(not “other”) when concurrency > 1, and each scenario shows anactivems figure — its own work (setup + deps + plan + nav + actions), roughly stable across--concurrencyso scenarios stay comparable. JSON’s per-caseduration_breakdowngainsactiveandcontention. - Data preconditions — scenario
requires(feedback #3).depends_oncaptures a scenario dependency;requires: ["1 active attraction", "a paid order"]documents a data one — the seed data a scenario assumes. Declarative: it renders in the report (terminal on failure, HTML, JSON) so a break caused by missing data is legible and the create→use→archive cycle is visible; it is never verified (usesetup/suite.setupto seed). NewScenario.requires,RunMetrics.requires.
0.47.0
- Dynamic values —
config.resolve(unblocks OTP, magic-links, passwordless login). Windup’s steps were UI-only (goto/click/fill/wait_for) — there was no way to grab a value generated at run time (an OTP code, a magic-link URL) and use it, so no OTP/magic-link flow was testable end-to-end. Now you declare a resolver inwindup.config.ts—resolve: { otp_code: { source: { kind: "cmd"|"http"|"fn", … }, extract: { regex | json }, poll } }— and a plan references it:{ "type": "fill", "value_ref": "otp_code" }or{ "type": "goto", "url_ref": "magic_link" }. The value is fetched (with polling — the code/email arrives late) at the point of use. Sources are author-declared, never LLM-generated (no code-exec-from-model vector), and the resolved value is ephemeral — never cached, reported or logged (the plan carries the reference name, likeENV:credentials). The planner is told the available names so it emitsvalue_ref/url_refinstead of a literal. Validated live: a cached replay of an OTP login fetches the run-time code from an external source and completes the flow at$0. Newresolvers.ts,config.resolve,Action.url_ref.
0.46.0
Five roadmap features in one release (from the “ideas for later” brainstorm), each tested and validated live:
windup doctor— preflight checks. Before a run, statically verify the LLM key for the active provider, the browser binary, that every scenario parses, that no cached plan references a missing fragment, and that the site map is scanned. No browser/LLM/network; non-zero exit only on a hard problem (invalid scenario, orphaned fragment).- Sharding —
run --all --shard i/n. Round-robin-split the suite across parallel CI runners (--shard 1/4,--shard 2/4, …), each a separate job. - Accessibility audit —
run --a11y. After each scenario, run axe-core on the final page and report violations — a free a11y check on infra Windup already has. Informational (never fails the run). Opt-in: axe-core is an optional dependency loaded via dynamic import and kept out of the base install (npm i -D axe-coreto enable). - Flake root-cause hints. Each flaky scenario (from
--repeat) now carries a hint at the likely cause, read from its runs: start-page signature drift → hydration race; a network failure; always-fails-the-same-action → unstable selector; cache churn → non-deterministic replay. Shown in the terminal summary and the HTML report. - Authoring
--watch.run <id> --watchre-runs a single scenario whenever its file changes — a tight authoring loop. - New
doctor.ts,browser.runAxe(),RunMetrics.a11y,FlakyScenario.hint;--shard/--a11y/--watchonrun.
0.45.0
- Smarter readiness — stop burning the timeout on display pages (speed). The initial page-signature wait now proceeds as soon as either the app renders interactive elements or the network settles (
networkidle), whichever comes first (still capped at 5 s). Previously it polled only for interactive elements, so a display-only page — no buttons, no pending requests — waited the full 5 s on every run (it showed up as the dominantnavchunk in the 0.42 breakdown). Measured ~9× faster on such a page (exec5088 ms → 556 ms). Can only be faster, never slower — both branches share the same deadline, and pages with interactive elements already bailed early. Newbrowser.waitForIdle().
0.44.0
- Safety denylist —
config.forbid(CI guardrail against irreversible side effects). Declare selectors and URLs a plan must never touch:forbid: { selectors: ["#change-password"], urls: ["**/account/password"] }. Before each action (and on the landing page) the executor aborts with aforbiddenfailure if the action’s CSS selector CONTAINS a forbidden substring or the current/goto URL matches a forbidden path glob — so even if a re-plan wanders toward “Change password”, it’s stopped before the click. You declare the danger list; the engine never infers it (zero site knowledge). The machine-enforced backstop to the non-destructive authoring discipline. Newconfig.forbid,forbiddenfailure kind,executor.forbiddenViolation().
0.43.0
windup coverage— find coverage gaps automatically. Cross-references the routeswindup scanindexed with your scenarios: it reports how many indexed routes have at least one scenario and lists the routes that have none — the “what am I missing” audit, generated from data Windup already has (the site map + scenarios + cached plans), with no LLM and no network. A scenario covers a route when itsstart_url(or any URL in its cached plan) matches the route’s url_pattern.--jsonfor pipelines (a CI gate can fail when critical routes are uncovered). Newcoverage.ts,SiteMapStore.allRoutes().
0.42.0
- Report time transparency — reconcile where the wall-clock goes (feedback). Two report fixes, no performance change:
- Per-scenario duration breakdown. A cached run that reads as “a 113 ms action took 3.6 s” now shows why: the HTML report splits each scenario’s total into a reconciling bar —
setup(context launch) ·deps(thedepends_onchain) ·plan(LLM) ·nav·actions·other— wherenavis the goto + page load/hydration BEFORE the first action, now isolated fromexecution(it’s usually the real time sink in an SPA). JSON carries a per-caseduration_breakdown; a newduration_ms.navigationmetric backs it. - Suite time is wall-clock, not the sum. The suite header led with the sum of per-scenario totals, which inflates ~N× under
--concurrency N(e.g. “511.9s” for a 130s run at concurrency 4). It now leads with wall-clock (real elapsed) and labels the sum:wall 130s (sum 512s · concurrency 4). Terminal, HTML and JSON (wall_ms,concurrency) all reflect it.
- Per-scenario duration breakdown. A cached run that reads as “a 113 ms action took 3.6 s” now shows why: the HTML report splits each scenario’s total into a reconciling bar —
0.41.0
- Client-side fixtures — scenario
seed(feedback #5, coverage at scale). A scenario can declare"seed": { "localStorage": {…}, "sessionStorage": {…}, "origin"? }to inject browser storage before the plan runs — reaching a client-side state (a cart inlocalStorage, a POS device insessionStorage) directly and deterministically, with no server call, instead of building it through the UI. Applied via a Playwright init script that sets each key only if absent (so the app’s own mutations are never clobbered on later navigations), per origin (default: thestart_urlorigin). Not part of the cached plan — it runs every time, so seeded scenarios stay$0and deterministic on replay. Validated live via--llm claude-code: an empty cart page renders the seeded items with no add-to-cart steps, and replays at$0. NewScenario.seed,browser.seedStorage(). - Docs: non-destructive CI testing. README gains a “stay at the side-effect boundary” guide — the discipline that keeps a per-push suite from charging cards, sending OTPs, creating accounts or mutating persistent state (client-side validation, read screens,
seeded state, bogus-token error pages, and open-then-cancel confirmation dialogs are all safe; real payment, messaging, identity creation, persisting config, voucher-consuming check-in and changing the test account’s password are not).
0.40.0
- Session snapshots:
depends_onrestores auth instead of re-running the login flow (feedback #4 — the big replay-speed win). Re-executing a UI login for every scenario that depends on it was the dominant wall-clock cost of a cached suite. Windup now captures each dependency’s exit state — PlaywrightstorageState(cookies + localStorage) + final URL — after it runs, and on a later cached replay it restores that state into a fresh context and skips re-running the wholedepends_onchain (deps≈0ms, reported asreused_session_from). Still verified: if the restored session is stale or incomplete (verification fails), the snapshot is dropped and the run falls back to a full-chain replay in a fresh context — no false pass, no wasted LLM call (adeferReplanguard keeps the snapshot attempt from invalidating a good cached plan). Snapshots live in.windup/state/(gitignored — they hold auth cookies/tokens; never commit them). Validated live via--llm claude-code: with a snapshot the dependency chain is skipped and the run passes atdeps≈0; a corrupted/empty snapshot fails verification and transparently re-runs the chain to green. Newsession-cache.ts,browser.launchBrowser({ storageState })/browser.storageState(),RunMetrics.reused_session_from.
0.39.0
- Wall-clock breakdown in the run report (feedback #4).
duration_msnow splitstotalintoplanning(LLM),execution(this scenario’s Playwright actions),dependencies(the re-rundepends_onchain) andsetup(browser context launch), printed astotal=… (plan=… deps=… exec=… setup=…)and surfaced in therun:end--streamevent (exec_ms/deps_ms/setup_ms). Makes it clear where a cached run’s wall-clock goes — the cache’s promise is$0(no LLM calls), not “instant”: the plan’s real-browser actions and any dependency chain still run. README’s “How it works” now states this explicitly. (Sets up thedepends_onsession snapshot that removes the dependency-replay cost.)
0.38.0
- Isomorphic plan reuse — scenario
like(feedback #3, the last item). At scale many scenarios are the same flow on a different route/entity. A scenario can now declare"like": { "scenario": "<source_id>", "set": { "<source value>": "<new value>" } }to reuse another scenario’s already-proven cached plan instead of an LLM planning call: Windup instantiates the source plan for this scenario’sstart_urland swaps the differing fill values (deterministic, no LLM). The reused plan is still executed and verified before it’s trusted/cached — if the pages aren’t actually isomorphic it falls back to normal LLM planning, so it can never produce a silent false green. On success the run isllm_calls=0(reused_fromset) and the scenario gets its own cached plan for ordinary$0replays after. Validated live via--llm claude-code: reuse passes on an isomorphic route ($0), and a non-isomorphic route fails verification and re-plans with the LLM. Newisomorph.ts(instantiatePlan),Scenario.like,RunMetrics.reused_from. Fragments reuse action blocks;likereuses whole plans.
0.37.0
- Suite-level fixtures —
config.suite.setup/config.suite.teardown(feedback #3). Shell command(s) run ONCE around arun --all: setup before the first scenario, teardown after the last (always, even on failure) — thebeforeAll/afterAllanalogue for a shared fixture database or an external stub. Per-scenariosetup/teardown(in the scenario JSON) still handle per-test state. A failingsuite.setupaborts the suite before any scenario runs (exit 2); a failingsuite.teardownis a warning. Only fires with--all, not for a single-scenario run. Validated live: setup → scenarios → teardown ordering, abort-on-setup-failure, and the--allgate. Newconfig.suitefield (reuseshooks.ts).
0.36.0
- Reusable readiness signals per route glob —
config.readySignals(feedback #3). Map a route glob (e.g."**/workspace/**") to the CSS selector(s) that must be visible before the executor runs the first action on a matching page. Applied deterministically at run time (no LLM, $0, not part of the cached plan) whenever a run enters a matching route — so a hydration/loading wait is defined once per route instead of repeated as a hint in every scenario. Closes the load-time race where an element is present but its handlers aren’t attached yet (Playwright’s per-element wait can’t see it). Best-effort: a signal that never shows within the timeout warns and continues. Validated live via--llm claude-code: the same generated plan fails without the signal (click races hydration, form never appears) and passes with it. Newexecutor.tsreadiness gate +readySignalsconfig field.
0.35.0
- Incremental runs —
run --all --changed/--since <ref>(feedback #3, 145-scenario suites). Run only the scenarios a change affects instead of the whole suite.--changeddiffs the working tree againstHEAD;--since main(or any git ref) diffs against that ref. A scenario is selected when its own file changed, when it has no cached plan, or when its cached plan visits a route whose indexed source changed (the site map’s file→route attribution + picomatch). Sound-but-coarse and never a silent false green: if the diff touches files the map can’t attribute to a route (shared code, config), or there’s no git / site map with source info, it runs the full suite and prints why. An empty affected set exits 0. New modulechanged.ts;SiteMapStore.affectedPatternsByFiles/indexedSourceFiles;scenarioFileById.
0.34.0
- Suite report: module grouping + suite stats (feedback #3 — 145 scenarios, 17 modules).
run --allprints a suite summary — pass rate, cache-hit rate, re-plans, LLM calls, cost, total time — with a per-module (folder) breakdown. HTML groups by module with cache-hit / re-plan tiles; JUnit emits one<testsuite>per module; JSON carries the full summary (by_module,flaky) and amoduleper case; under--streamit’s asuiteevent. - Flake score.
--repeat <n>is aggregated per scenario — one passing some-but-not-all of its runs is flagged flaky (passed X/N) in the summary and reports.
0.33.0
- Native dialogs —
window.confirm/alert/prompt(fixes a beta-report blocker, #12). Playwright auto-dismisses dialogs unless a handler is registered, so a click behind aconfirm()(archive/delete/cancel) silently did nothing. Actions now take"dialog": "accept"(or"dismiss"), and the executor arms a one-time handler before the triggering action; the planner emits it for confirm/alert/prompt steps instead of inventing a fragment. Deterministically verified (accept runs the mutation, dismiss cancels) and live-verified that the planner emits it (via--llm claude-code). - Verify persistent signals, not toasts (#12). The planner now prefers a lasting postcondition (a row that appears/disappears, a changed label, a URL) over transient toast/snackbar messages that vanish in seconds and make verification a race.
0.32.0
windup newsteers the verification toward the instruction (#5, from a beta report). The authoring prompt now derives the final verification from what the instruction actually asks — preferring a visible element/text over a plausible-but-unasked destination route from the site map (a common LLM mistake when the map lists many routes).windup newalso flags that the task/verification is the LLM’s best guess and recommends confirming with--validate(generate → run → self-refine) or a first run. Revalidated live via--llm claude-code.
0.31.0
run --stream— NDJSON event stream (the machine-readable half of the beta report’s #9). Emits one JSON line per milestone to stdout (run:start,planning,plan,action,replan,run:end, each with the scenario, elapsed time and relevant data), so CI or a dashboard can follow a run in real time. Human progress (--verbose) stays on stderr, keeping stdout pure NDJSON.
0.30.0
- Guided self-heal (#10, from a beta report). When a cached plan fails verification and Windup re-plans, the re-plan context now names the exact selector that failed with a “do not reuse it” instruction, re-emphasizes the scenario hints, and — under
--suggest— feeds the same expert diagnosis you’d read straight back into the planner, so it corrects instead of re-proposing a refuted semantic selector. A loop-breaker warns when a scenario keeps re-planning without stabilizing (the app likely lacks a stable selector — an accessibility gap — or has a race), instead of churning LLM calls silently.
0.29.0
- Per-scenario
setup/teardownhooks (from a beta report on non-idempotent CREATE). Shell commands that run outside the cached plan — so they run on every replay — for fixtures or cleanup (hard-delete what a test created, reset via SQL/HTTP).setupruns before the scenario and its dependencies (a failure fails the run, kindsetup);teardownruns always, even on failure (a failure is a warning). They never enter the plan or cache. - Docs: idempotency principle — prefer idempotent scenarios (edit-to-fixed-value, toggle-and-check); a pure CREATE with a non-reusable unique key needs a teardown hook. Plus a “flakiness becomes signal” note: a plan that stops replaying deterministically is exposing an app race, not a flaky test.
0.28.0
run --verbose— a heartbeat during planning (from a beta report: planning with--llm claude-codetakes 1–3 min with no output, so a run looks frozen). Verbose mode emits milestones to stderr as planning and execution advance —planning… (llm: …),calling <provider> (attempt N)…,plan received: N actions, per-action✓/✗, and→ self-heal re-planning— each prefixed with the scenario id and elapsed time. Off by default; never affects results.
0.27.0
- Scenarios can be organized in subfolders (from a beta report).
run --all, the vitest suite anddepends_onnow discover scenarios recursively under the scenarios directory — group them by module (e2e/scenarios/contacts/…,…/auth/…). Thescenario_idfield stays the identity (resolution is by id, not file path; duplicate ids are reported).loadScenariokeeps the<dir>/<id>.jsonfast path and falls back to a recursive search byscenario_id. - Note:
windup inithas detected TanStack Router since 0.25.0 (writesframework: "tanstack-router") — re-runwindup initon a project scaffolded by an older version to pick it up.
0.26.0
- TanStack Router / TanStack Start indexer (biggest ask from the beta report).
windup scannow statically indexes TanStack’s file-based routes undersrc/routes/(orapp/routes/): dot-notation as path separators (workspace.loja.aparencia.tsx→/workspace/loja/aparencia), directory params (loja/$companySlug/checkout/pagamento.tsx→/loja/:companySlug/checkout/pagamento), pathless layout segments (_authenticated/_company/manager.companies.tsx→/manager/companies), splats,index/opt-out markers, and__rootskipped. It trusts each route’screateFileRoute('/id')string (TanStack’s resolved id) and falls back to file-name conventions.initsetsframework: "tanstack-router"automatically. On a real 118-route app this takes the map from ~7 routes to full coverage with real selectors.
0.25.0
- Self-heal reuses the provider that planned the scenario (fixes a real bug report). When a cached plan fails verification and is re-planned, Windup now re-plans with the same LLM provider that originally made the plan (recorded in the plan), before falling back to the config default — so self-healing works even when the re-run didn’t pass
--llm. Precedence:--llm/WINDUP_LLM> the plan’s recorded provider >llm.providerin config. - Actionable “no key” errors — a planning failure for a missing key now names the provider and lists the fixes (pass
--llm <provider>, set the variable, or changellm.provider), instead of a bare “VAR is not set”. - TanStack Router detected —
initrecognizes@tanstack/react-router/@tanstack/react-start(frameworktanstack-router), andscanexplains that TanStack file-based routing isn’t statically indexed yet (the map is built from executions) and how to opt into the react-router indexer. The generic “no indexer” message now points atframework: "react-router".
0.24.0
windup secret remove <account>(aliasrm) — completes credential management: drops the account fromwindup.credentials.jsonand its values from.env.local(other variables untouched), and clears it from the manifest.- Credentials docs overhauled — the package README and the docs site (en/es/pt/zh) now fully cover where values are stored, creating/listing/removing accounts, and referencing them by name in a scenario.
0.23.0
windup claude login/windup claude status— one-command onboarding for--llm claude-code:statusreports whether theclaudeCLI is installed and signed into your plan (machine-readable probe, no quota spent; non-zero exit when not ready);logininstalls the CLI if missing (interactive confirm — never a silent global install, and never in CI) and runsclaude auth login.windup statusalso shows the readiness line when claude-code is the active provider.
0.22.0
--llm claude-codeneeds no wrapper anymore — it now drives the nativeclaudeCLI you already have (claude -p … --output-format json), spawned from an isolated temp dir. Zero setup: no Python, no Poetry, no local server — just the Claude Code CLI installed and logged into your plan. The claude-code-openai-wrapper becomes the opt-in path, used only when abaseUrl/WINDUP_CLAUDE_CODE_URLis configured. Same $0 subscription cost, same mechanical un-fencing (Ajv still validates every plan). A missing/logged-out CLI fails fast with an actionable install//loginmessage. Verified end-to-end (plan → execute → verify) on the native path. Still opt-in, never a default.
0.21.0
- Plan with your Claude subscription — new opt-in provider
--llm claude-code, targeting the third-party claude-code-openai-wrapper (a local proxy over your own Claude Code session). No API key required; cost is reported as $0 inwindup costs(tokens are real and stay in the ledger, but they’re covered by your subscription — never priced at the fallback rate). Opt-in, never a default, and unsupported by us or Anthropic. The wrapper implements onlymodel/messages/stream, so the schema rides in the prompt and the reply is un-fenced mechanically (Ajv still validates every plan); a down wrapper fails fast with an actionable error instead of retrying tofetch failed. Default modelclaude-sonnet-4-6; endpoint configurable viabaseUrl/WINDUP_CLAUDE_CODE_URL.
0.20.0
- Cross-browser — run scenarios on
chromium(default, auto-provisioned),firefoxorwebkitvia--browser/WINDUP_BROWSER/config.browser. Firefox/WebKit are opt-in (npx playwright install <name>); a single plan replays across all three (CSS selectors are engine-agnostic).
0.19.0
- Parallel runs —
run --concurrency <n>runs scenarios in parallel over one shared warm browser with isolated contexts (one shared site map, order-preserving results). Measured ~2× faster on an 11-scenario suite at concurrency 4. Default 1 (behavior unchanged).
0.18.x
run --suggest— on a failed run, an LLM analyzes the executed plan, the failing step, the real final page and the site map, and proposes a concrete fix to the scenario. Closes the authoring learning loop.windup new --validate— generate → run → refine from the failure until the scenario passes (≤3 attempts); you get a scenario that already passed once.- Graceful CLI errors — expected failures print a clean, actionable line;
WINDUP_DEBUG=1shows the full stack. No more raw Node stack traces. - Security — page content is delimited as untrusted in all LLM prompts
(planner,
--summary,--suggest) to mitigate prompt injection;SECURITY.mdthreat model added. - Robustness measured: 60/60 cached replays passed with zero flakes and
llm_calls=0across four scenarios (login, multi-step checkout, add/remove, a second site), 15 replays each. - Docs: demo GIF in the README; full English translation of the repository.
0.15.0
- Scenario dependencies (
depends_on) — prerequisites run in the same browser session, each with its own cache and self-healing; a dependent scenario withoutstart_urlcontinues from the dependency’s final page (the planner sees the real post-login screen). Editing a task now invalidates its cached plan.
0.13.0 – 0.14.x
run --summary— post-run AI debrief quoting concrete observed values (prices, messages), off by default; collapsed block in the HTML report.- Secure test credentials —
windup secret set/list; values live in.env.local/CI secrets, the account→ENV mapping in committedwindup.credentials.json;windup newauto-registers and scrubs credentials.
0.12.0
- HTML reporter —
run --reporter html, a self-contained page (no JS/deps).
0.11.0
windup new— LLM-assisted scenario authoring grounded in the site map and project manifest; suggestsdepends_onfrom existing scenarios.
0.10.0
- Multi-provider LLM — Google Gemini and OpenAI (plain REST), selectable per
run with
--llm provider[:model]; per-provider cost breakdown inwindup costs.
0.9.0
- CI/CD reporters (JUnit/JSON),
run --all, environment-portable start URLs (--base-url/WINDUP_BASE_URL, path-keyed cache).
0.6.0 – 0.8.x
- Engine migrated to Playwright (trusted input events, warm browser pool);
windup scan(Next.js + react-router indexers with LLM-assist); trajectory fragments; auto-provisioned Chromium.
0.1.0 – 0.5.0
- First installable package: natural-language scenarios → LLM plan → deterministic
execution → cheap verification → trajectory cache → zero-LLM replays. Page
signatures, site map, project manifest,
windup costs, vitest adapter.