Windup — test report

generated 2026-07-13 15:09 UTC · deterministic replay, LLM only on cache miss

10scenarios

9passed

1failed

1zero-LLM runs

9llm calls

$0.0169est. cost

47.1sduration

Scenario

Cache

LLM calls

Model

Duration

Cost

PASS

duelo-compra-tripla

14 action(s)

id	status	action	verify
a1	passed	70 ms	0 ms
a2	passed	38 ms	0 ms
a3	passed	89 ms	0 ms
a4	passed	31 ms	0 ms
a5	passed	28 ms	0 ms
a6	passed	30 ms	0 ms
a7	passed	35 ms	0 ms
a8	passed	26 ms	0 ms
a9	passed	3 ms	0 ms
a10	passed	3 ms	0 ms
a11	passed	3 ms	0 ms
a12	passed	36 ms	0 ms
a13	passed	31 ms	0 ms
a14	passed	0 ms	1 ms

miss

google/gemini-3.1-flash-lite

4735 ms

$0.002428

PASS

duelo-login-bloqueado

5 action(s)

id	status	action	verify
a1	passed	23 ms	0 ms
a2	passed	35 ms	0 ms
a3	passed	39 ms	0 ms
a4	passed	0 ms	0 ms
a5	passed	1 ms	1 ms

miss

google/gemini-3.1-flash-lite

2302 ms

$0.001759

PASS

duelo-remover-carrinho

7 action(s)

id	status	action	verify
a1	passed	12 ms	0 ms
a2	passed	35 ms	0 ms
a3	passed	52 ms	0 ms
a4	passed	32 ms	0 ms
a5	passed	34 ms	0 ms
a6	passed	32 ms	0 ms
a7	passed	1 ms	1 ms

miss

google/gemini-3.1-flash-lite

2453 ms

$0.001814

FAIL

orangehrm-login

[verification] action=a4: element .oxd-dashboard-grid did not become visible within 10000ms

4 action(s)

id	status	action
a1	passed	12 ms
a2	passed	33 ms
a3	passed	43 ms
a4	failed	10012 ms

miss

google/gemini-3.1-flash-lite

20807 ms

$0.001225

PASS

saucedemo-carrinho-vazio

10 action(s)

id	status	action	verify
a1	passed	13 ms	0 ms
a2	passed	31 ms	0 ms
a3	passed	40 ms	0 ms
a4	passed	33 ms	0 ms
a5	passed	0 ms	0 ms
a6	passed	1 ms	0 ms
a7	passed	1 ms	0 ms
a8	passed	1 ms	0 ms
a9	passed	1 ms	0 ms
a10	passed	0 ms	1 ms

miss

google/gemini-3.1-flash-lite

2907 ms

$0.002116

PASS

saucedemo-checkout

12 action(s)

id	status	action	verify
a1	passed	10 ms	0 ms
a2	passed	24 ms	0 ms
a3	passed	45 ms	0 ms
a4	passed	31 ms	0 ms
a5	passed	32 ms	0 ms
a6	passed	31 ms	0 ms
a7	passed	3 ms	0 ms
a8	passed	4 ms	0 ms
a9	passed	6 ms	0 ms
a10	passed	32 ms	0 ms
a11	passed	31 ms	0 ms
a12	passed	0 ms	1 ms

miss

google/gemini-3.1-flash-lite

3007 ms

$0.002194

PASS

saucedemo-compra-dupla

14 action(s)

id	status	action	verify
a1	passed	14 ms	0 ms
a2	passed	31 ms	0 ms
a3	passed	59 ms	0 ms
a4	passed	38 ms	0 ms
a5	passed	30 ms	0 ms
a6	passed	32 ms	0 ms
a7	passed	30 ms	0 ms
a8	passed	34 ms	0 ms
a9	passed	4 ms	0 ms
a10	passed	4 ms	0 ms
a11	passed	5 ms	0 ms
a12	passed	33 ms	0 ms
a13	passed	30 ms	0 ms
a14	passed	1 ms	1 ms

miss

google/gemini-3.1-flash-lite

3447 ms

$0.002427

PASS

saucedemo-conta-qa

4 action(s)

id	status	action	verify
a1	passed	14 ms	0 ms
a2	passed	32 ms	0 ms
a3	passed	48 ms	0 ms
a4	passed	1 ms	1 ms

miss

google/gemini-3.1-flash-lite

2001 ms

$0.001571

PASS

saucedemo-login

3 action(s)

id	status	action	verify
a1	passed	12 ms	0 ms
a2	passed	29 ms	0 ms
a3	passed	43 ms	3 ms

hit

—

356 ms

PASS

the-internet-login

7 action(s)

id	status	action	verify
a1	passed	12 ms	0 ms
a2	passed	32 ms	0 ms
a3	passed	336 ms	0 ms
a4	passed	8 ms	0 ms
a5	passed	1 ms	0 ms
a6	passed	1 ms	0 ms
a7	passed	1 ms	1 ms

miss

google/gemini-3.1-flash-lite

5083 ms

$0.001409