Windup — test report

generated 2026-07-13 15:10 UTC · deterministic replay, LLM only on cache miss

10scenarios

9passed

1failed

9zero-LLM runs

1llm calls

$0.0012est. cost

24.3sduration

Scenario

Cache

LLM calls

Model

Duration

Cost

PASS

duelo-compra-tripla

14 action(s)

id	status	action	verify
a1	passed	37 ms	0 ms
a2	passed	24 ms	0 ms
a3	passed	78 ms	0 ms
a4	passed	33 ms	0 ms
a5	passed	31 ms	0 ms
a6	passed	31 ms	0 ms
a7	passed	33 ms	0 ms
a8	passed	30 ms	0 ms
a9	passed	4 ms	0 ms
a10	passed	3 ms	0 ms
a11	passed	3 ms	0 ms
a12	passed	38 ms	0 ms
a13	passed	30 ms	0 ms
a14	passed	1 ms	1 ms

hit

—

1234 ms

PASS

duelo-login-bloqueado

5 action(s)

id	status	action
a1	passed	12 ms
a2	passed	24 ms
a3	passed	30 ms
a4	passed	1 ms
a5	passed	1 ms

hit

—

316 ms

PASS

duelo-remover-carrinho

7 action(s)

id	status	action
a1	passed	12 ms
a2	passed	21 ms
a3	passed	44 ms
a4	passed	36 ms
a5	passed	30 ms
a6	passed	29 ms
a7	passed	1 ms

hit

—

425 ms

FAIL

orangehrm-login

[verification] action=a4: element .oxd-dashboard-grid did not become visible within 10000ms

4 action(s)

id	status	action
a1	passed	20 ms
a2	passed	35 ms
a3	passed	39 ms
a4	failed	10016 ms

miss

google/gemini-3.1-flash-lite

17799 ms

$0.001225

PASS

saucedemo-carrinho-vazio

10 action(s)

id	status	action	verify
a1	passed	14 ms	0 ms
a2	passed	30 ms	0 ms
a3	passed	40 ms	0 ms
a4	passed	34 ms	0 ms
a5	passed	1 ms	0 ms
a6	passed	0 ms	0 ms
a7	passed	1 ms	0 ms
a8	passed	1 ms	0 ms
a9	passed	0 ms	0 ms
a10	passed	0 ms	1 ms

hit

—

475 ms

PASS

saucedemo-checkout

12 action(s)

id	status	action	verify
a1	passed	13 ms	0 ms
a2	passed	25 ms	0 ms
a3	passed	38 ms	0 ms
a4	passed	37 ms	0 ms
a5	passed	30 ms	0 ms
a6	passed	32 ms	0 ms
a7	passed	2 ms	0 ms
a8	passed	3 ms	0 ms
a9	passed	3 ms	0 ms
a10	passed	22 ms	0 ms
a11	passed	31 ms	0 ms
a12	passed	0 ms	1 ms

hit

—

489 ms

PASS

saucedemo-compra-dupla

14 action(s)

id	status	action
a1	passed	12 ms
a2	passed	21 ms
a3	passed	41 ms
a4	passed	34 ms
a5	passed	30 ms
a6	passed	32 ms
a7	passed	30 ms
a8	passed	33 ms
a9	passed	4 ms
a10	passed	4 ms
a11	passed	3 ms
a12	passed	36 ms
a13	passed	31 ms
a14	passed	1 ms

hit

—

581 ms

PASS

saucedemo-conta-qa

4 action(s)

id	status	action	verify
a1	passed	12 ms	0 ms
a2	passed	24 ms	0 ms
a3	passed	48 ms	0 ms
a4	passed	2 ms	1 ms

hit

—

331 ms

PASS

saucedemo-login

3 action(s)

id	status	action	verify
a1	passed	12 ms	0 ms
a2	passed	31 ms	0 ms
a3	passed	74 ms	15 ms

hit

—

397 ms

PASS

the-internet-login

7 action(s)

id	status	action	verify
a1	passed	20 ms	0 ms
a2	passed	44 ms	0 ms
a3	passed	315 ms	0 ms
a4	passed	8 ms	0 ms
a5	passed	1 ms	0 ms
a6	passed	1 ms	0 ms
a7	passed	0 ms	1 ms

hit

—

2260 ms