Scenario explorer

Every eval scenario, replayed through the real runtime with a fake clock and the scripted model: what came in, what the agent decided and why, what it sent, and the state of every open ask after each step. The data is produced at build time by uv run load-agent export; 30 of 30 scenarios match their expectations.

Search

Tags

Cross-channel Escalation Idempotent Model call Do nothing Timers

30 scenarios

• Delay risk high; ETA asked twice; driver asks for the PO; escalate to dispatcher and Hero. client_a_load_481207_full_timeline Client A EscalationModel callTimers 5 steps
• Address answers follow the stop named in the question; an ambiguous one goes to a Hero. address_question_names_the_stop_it_is_about Client A EscalationModel call 4 steps
• Broker stand-down becomes a Hero task; a Hero override then cancels the chain. broker_stand_down_cancels_the_eta_chain Client A EscalationModel callDo nothing 6 steps
• Literal assignment load JSON; the PO is parsed from the instructions and answered. client_a_po_from_instructions Client A Model call 2 steps
• Client B delay risk high; one SMS, then a broker Slack alert, never the dispatcher. client_b_delay_risk_sms_then_slack_never_dispatcher Client B EscalationTimers 3 steps
• Client B dispatcher asks for the PO by email; a Hero answers it, with and without an open ask. client_b_dispatcher_po_email_goes_to_a_hero_with_or_without_an_open_ask Client B EscalationModel call 3 steps
• Reefer driver departs the pickup; one SMS asks for trailer temperature and seal number. client_b_reefer_departed_pickup_asks_temperature_and_seal Client B reefer 2 steps
• Pickup ETA ask open, delivery stop goes high; a separate delivery ETA ask is opened. delivery_risk_high_while_pickup_eta_ask_is_open_asks_for_the_delivery_eta Client A Timers 3 steps
• DEPARTED without ARRIVED closes the pickup ask; a later ETA resolves the delivery ask. departing_without_arriving_closes_the_pickup_ask Client A Model call 4 steps
• Dispatcher emails the ETA; the driver ask resolves and the 07:31 timer is a noop. dispatcher_email_eta_resolves_driver_ask Client A Cross-channelModel call 4 steps
• The driver reaches the pickup before the follow-up; the ETA ask is cancelled, nobody is chased. driver_arrives_before_follow_up_cancels_eta_ask Client A Do nothingTimers 5 steps
• The same timer delivered twice produces one follow-up SMS; the copy is dropped. duplicate_timer_delivery_is_noop Client A IdempotentDo nothingTimers 3 steps
• A dispatcher question in a side email thread becomes a Hero task; in the main thread it is answered. email_outside_the_main_thread_is_escalated_not_answered Client A EscalationModel call 2 steps
• One ETA answer closes the ask of the nearest stop only. eta_reply_resolves_only_the_nearest_stop_ask Client A Model call 3 steps
• A second high ping while already high sends nothing and opens no second ask. high_ping_while_already_high_is_not_a_trigger Client A Do nothing 2 steps
• A reply that names the delivery resolves the delivery ask, not the nearest-stop pickup ask. named_stop_eta_resolves_only_that_stops_ask Client A Model call 3 steps
• Driver sends "ok" with no open ask; the agent does nothing. ok_with_no_open_ask_is_noop Client A Do nothing 2 steps
• Driver answers the ETA request with "ok"; no reply, the ask stays open. ok_with_open_eta_ask_keeps_ask_open Client A Do nothing 2 steps
• Driver asks for the PO but the metadata has none; Hero task, no invented number. po_question_without_po_number_becomes_hero_task Client A EscalationModel call 1 step
• The same answer to the same contact inside 10 minutes is not repeated. repeated_answer_within_ten_minutes_is_sent_once Client A Model callDo nothing 3 steps
• Risk returns to low while the ETA ask is open; the ask is cancelled and nobody is chased. risk_falls_back_cancels_the_eta_chain Client A Do nothingTimers 4 steps
• High, low, high within 10 minutes; one text per 30 minutes and the escalation still happens. risk_flapping_through_low_keeps_the_chain_on_schedule Client A EscalationDo nothingTimers 5 steps
• High, medium, high within 10 minutes; one text per 30 minutes and the escalation still happens. risk_flapping_through_medium_keeps_the_chain_on_schedule Client A EscalationDo nothingTimers 5 steps
• A late low ping recorded before the high ping changes nothing. stale_ping_is_ignored Client A Do nothing 2 steps
• Driver sends "👍" with no open ask; the agent does nothing. thumbs_up_with_no_open_ask_is_noop Client A Do nothing 2 steps
• Driver answers the ETA request with a thumbs up; no reply, the ask stays open. thumbs_up_with_open_eta_ask_keeps_ask_open Client A Do nothing 2 steps
• A timer delivered after the ETA arrived is a filtered noop with no LLM call. timer_after_eta_received_is_noop Client A Cross-channelModel callDo nothingTimers 3 steps
• A repeated unknown-sender message gets one Hero task per 30 minutes; new content still gets one. unknown_sender_gets_one_hero_task_per_half_hour Client A EscalationDo nothing 4 steps
• An unclassifiable broker Slack message becomes a Hero task, nothing else. unknown_situation_escalates_to_hero_task Client A EscalationModel call 1 step
• Driver asks for the pickup address; it is answered and the ETA ask is unchanged. unrelated_driver_question_is_answered_and_ask_stays_open Client A Model call 2 steps