Documentation / Guides / Deterministic testing

Test with `RuntimeTestKit`¶

Use the real public runtime and replace only external time, IDs, transport, and providers.

The three test primitives¶

Primitive	Purpose
`ScriptedAdapter`	immutable credential-free call/provider event tape
`FakeMonotonicClock`	explicit monotonic time and deadline control
`DeterministicIds`	repeatable call/evidence IDs

RuntimeTestKit assembles them and provides a VoiceRuntime, invocation, and post-call observation.

adapter = ScriptedAdapter(
    "offline",
    (
        {"kind": "connected"},
        {"kind": "participant_ready", "participant_id": "caller"},
    ),
)
clock = FakeMonotonicClock()
kit = RuntimeTestKit(
    clock=clock,
    identifiers=DeterministicIds(("call-1",)),
    adapters=(adapter,),
)
result = await kit.runtime(price_book=price_book).run(
    kit.invocation(telephony=telephony, consumer_state={}),
    setup,
)

Use the complete event pattern from examples/minimal_offline.py in the source repository, not this abbreviated construction snippet, for a runnable turn.

Advance deadlines without sleeping¶

Start the runtime in a task, yield control until the phase has installed its waiter, then call:

await clock.advance(Decimal("5"))

Assert the normalized timeout, terminal outcome, one CLOSED transition, zero adapter.open_resources, and zero clock.pending_waiters.

What every end-to-end test should prove¶

expected prompt/profile/tool composition;
transcript and provider path;
business state change only after admitted tool work;
exactly one terminal result;
expected failure or no failure;
canonical result digest when stable evidence matters;
all transport/provider/clock resources drained.

Keep live checks separate¶

An offline passing test proves runtime composition, not provider availability or telephony. Live preflight should validate required environment names and route configuration without connecting. Credentialed smokes should be explicit, bounded, secret-free, and skipped/blocked when infrastructure is absent.

Repository checks¶

python -m contracts.verify
python scripts/check_docs.py
python -m unittest tests.test_runtime_call_path tests.test_runtime_tools

Each command is offline. Run only the focused suite needed for a change before the final relevant suite.

Next: production readiness or troubleshooting.

Test with RuntimeTestKit¶