Method and downloads

The project asks whether repairing stale or inaccurate travel-time inputs can improve route efficiency and robustness. It does not try to imitate the historical driver sequence.

Source data and simulated conditions

The starting route, stop and package data come from the 2021 Amazon Last Mile Routing Research Challenge Dataset. The dataset contains historical deliveries from 2018. Its location data are obfuscated; the map shows approximate schematic locations.

Amazon’s directed travel-time matrices are historical averages, not measurements of current traffic. Current traffic conditions, report ages, recent observations and input faults in this project are synthetic. An input’s age alone cannot reveal an arbitrary traffic shock.

The source dataset is attributed to Amazon and the authors listed in the AWS dataset registry, which describes its Creative Commons Attribution-NonCommercial 4.0 license. The downloads below retain this project’s export provenance.

How a recorded comparison is produced

  1. Separate complete routes chronologically. Training, validation and evaluation receive different routes before synthetic variants are created. Variants of a route stay together.
  2. Create a simulated decision snapshot. Historical inputs seed synthetic current conditions and the reports available at the modeled decision time. The hidden reference stays fixed across paired method comparisons.
  3. Prepare the planner’s travel-time input. Compare raw reports, a simple repair rule and CatBoost repair. Repair receives observable evidence, not hidden current values, defect flags, future measurements or evaluation outcomes.
  4. Run the same route planner with the same budget. The equal-budget comparison isolates changes in the travel-time input. A recorded guarded policy may choose a fallback; its decision and reasons belong to that saved policy.
  5. Evaluate the saved routes. Measure directed travel, service, waiting, lateness and validity against the fixed simulated reference. Report unfavorable cases and estimate uncertainty at the route level.
  6. Review the recorded decision. The demo supports accepting a plan, keeping the recorded raw plan or deferring. A saved review is a decision record; it does not dispatch a vehicle.

What the replay means

Each time point is a separate counterfactual restart of a complete route from its depot under that decision snapshot. Moving the time control does not show a vehicle progressing through a continuous trip. This is recorded simulation, not live tracking.

The website displays frozen plans and outcomes. It does not train or run a model in the browser. Map paths connect stops through an illustrative city; they do not represent real street geometry, actual building heights or turn-by-turn navigation.

Estimated metrics describe the information used by the planner. Revealed outcomes use the retrospective simulated reference. Review effort and computational runtime are different quantities and should be reported separately. Viewing results or downloading evaluations can inform a reviewer’s judgment.

Limits of the current evidence

  • Round 8 contains 16 previously inspected routes and 1,024 correlated planning snapshots. It is not a fresh final holdout.
  • Synthetic traffic and reporting faults cannot cover all operational conditions. A repair may help one condition and harm a clean or unfamiliar case.
  • Historical raw runs with an older planner budget are separate baselines. Differences against them do not measure the effect of repair alone.
  • A heuristic search does not establish an optimal route. Duration improvements alone do not establish acceptable lateness or route validity.
  • Missing evidence remains missing. No recorded aggregate effect guarantees operational savings, safety or deployment readiness.
  • Human review and deferral must be counted, including the effort and outcomes of decisions left unresolved.

Downloads and recorded fallback

The manifest identifies the saved export and its source artifacts. Preserve it with any downloaded evidence so the results remain traceable.

Inspect the provenance manifest
Provenance manifest
FieldExported value
schemaVersion1.0
datasetIdround8-replay-v1
provenancerecorded_simulation
sourceCommit52991922e9d0fb53768b5f799285ce95677fe18b
sourceLocalIdenticalTreeCommit0272ddbab5c60800f161f8d076db50f818fc7b4c
sourceTree01cb88949184c6dd46bc32acfdaa6e831c540290
files
Provenance manifest: files / 17 records
Recordpathsha256bytes
1README.mde0d0e3782add0d11229167330d51901e06bf37e98df600c6954a00660aba9d992889
2catalog.jsona947e5ac82ee89f7d5324a59166604e374dcd657e0fb366423de0dff4033ffb54557
3evidence-export-receipt.jsona5231dbfadd4d392936c8197d89d61b38741e5a53ed5a80b5a762c539c8169fa3312
4evidence.json0b21133536c3e36d6a9277f17202318d0a5cf52604e51d80d4ac9feaccc3d6cd240307
5export_evidence.py64bebdface32bc5b5e3844fca4526ba2ecc0e5b6ed60b49611a80f1f7173845622210
6export_manifest.py0b0c021c69a922dd86a6e0164b317fefae204e803867e74b034665905608d0684058
7export_receipt.json61f1848910d9ce6ace0cef235add100a72d50450008d301a9365412e41d7101147617
8export_replay.py2df9a4cc372da91e7ed62bea4266e70399326c22cc6ab7ef73baef206a9956bf18272
9guard-fallback-evaluation.json683e894c2292b817ae5630b80d0857fe378a29b51b14840df13b4cb4214de58f12445
10guard-fallback.json2a5374d4b3f43193ee0ff2db31c552be2d91270c0533bc79ba40010ee7ead9f4313222
11harmful-repair-evaluation.jsonbbce609c657ce779358db82c8884e7c01538a53c365c62872cd4fa733a9ef14b12378
12harmful-repair.json392432cd2e8a03d885ef0773b95a8b4ad567a8da87f2b5e885d95a56661fd3f8308638
13ordinary-evaluation.json1d24195c714818ff423501b7432d51274671211a1fca59ca9e35c22963d98ff312534
14ordinary.jsonb030386709686421ad92bc8904d60b0221854a600f78cea0b279ba4dc9c1d12a322058
15repair-benefit-evaluation.jsonfe37edc50529a7f5bce1bee4c504b5a60b02ae8efdf86de500de5c2c0eededec12462
16repair-benefit.json4716e257945f210e9c02ba78de59ebd4f8fb888cdadde276b8427692930762d7316454
17verify_replay.py5391d850d06783d153124e0c2d1776bd7e67ab7c5518f2fc2e937717a5fd618522215
manifestExclusions
Provenance manifest: manifestExclusions / 2 records
RecordValue
1manifest.json (self)
2verification.json (independent receipt pins manifest)
sourceIdentities
Provenance manifest: sourceIdentities / 10 records
Recordpathsha256
1src/caseathon/v8_storage.py75599ac24b11a62b2b8e18eb46001aae309f4f2e0dde8c8c46e08f8e12845a1a
2src/caseathon/v8_routing.pyc865aba7f78262fb0118d36361414bf354cd5e2b12b243aaa41dfc0f77f64e77
3src/caseathon/v8_policy.py862ccb975ab3b5fe76ebda99bba52ac33d6cbe266515269e242d7840fbb40900
4src/caseathon/v3_simulation.pyd655dda078b6df1a8a246a59fa7094d2a7d9f35ca4bce2711095a657bea31483
5src/caseathon/data.pye339471d7904d3ff82ffc68b0e13acfe960c5ec8e8b940016d813abaae7b53de
6docs/ROUND8_PROTOCOL.md29d69487263a26d81f92d93654a2671b352d95a7118cd21acf38e487a4575b7d
7configs/round8.json8c7bcc406bc5b73d1d937760f54497d0e26090abe1adf7cefb8e2fb167bd11cf
8configs/round8_splits.jsone2151fcaef653ad772c41f2f4cc2623583ebfae07306d928dd15997c1871d109
9outputs/round8/protocol_lock.json4ba33aee477979d00a4baa84889ef896472a884b3ba8793d8efca8ee6e876adc
10outputs/round7/fit_manifest.jsondf5fbe2834cd092f761f5a47dcee954fdffb21ac259526181fa60d4aba9a22bf
model
Provenance manifest: model
FieldExported value
idS79_RMSE
modelSha256670651e76b40c5e97e53c753441f1fb5a0fbecb3a8351a3459fb6af1d55de146
featureColumns79
frozenRound7
loadedForExportFalse
uncertaintyNull — not available in this export
simulation
Provenance manifest: simulation
FieldExported value
idfactorial-v3.0.0
seed1207
syntheticCurrentTrafficTrue
historicalMatricesAreCurrentTruthFalse
cohort
Provenance manifest: cohort
FieldExported value
splitpreviously inspected round8 calibration
sourceIndependentRoutes16
sourceCorrelatedDecisionsPerPolicy1024
illustrativeRoutes4
illustrativeSnapshots32
selectionRetrospective illustrative selection, not random or representative
sourceRouteDate2018-08-13
conditions
Provenance manifest: cohort: conditions / 4 records
Provenance manifest: cohort: conditions / 4 records
RecordValue
1normal
2stale
3inaccurate
4combined
timeMinutes
Provenance manifest: cohort: timeMinutes / 2 records
Provenance manifest: cohort: timeMinutes / 2 records
RecordValue
10
2120
freshConfirmationInspectedFalse
finalInspectedFalse
units
Provenance manifest: units
FieldExported value
travelAndServiceseconds
latenesssummed package-seconds at service start
latePackagespackage count, not rate
reportAndFeedAgeminutes
timeMinutessaved snapshot label; route restarts at zero
coordinatesdegrees, rounded to three decimals
missingFraction0..1 of saved report slots
attribution
Provenance manifest: attribution
FieldExported value
dataset2021 Amazon Last Mile Routing Research Challenge
publisherAmazon
licenseCreative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)
licenseUrlhttps://creativecommons.org/licenses/by-nc/4.0/
registryUrlhttps://registry.opendata.aws/amazon-last-mile-challenges/
citationAmazon Last Mile Routing Research Challenge dataset (2021), Amazon. Distributed under CC BY-NC 4.0. Dataset description: 2021 Amazon Last Mile Routing Research Challenge: Data Set, Transportation Science, doi:10.1287/trsc.2022.1173.
retrievalDateNull — not available in this export
retrievalDateNoteExact original retrieval timestamp was not recorded. Saved source metadata records checksum verification on 2026-10-17; no download occurred for this export.
exportDate2026-10-19
sourceMetadataSha25664cdde9416bc70215ad628f907a514c18ddd7b3418d9eca9755a15dd21afeaf0
effort
Provenance manifest: effort
FieldExported value
exportNewFits0
exportModelLoads0
exportPredictions0
exportPlannerCalls0
exportSimulatorCalls0
exportEvaluatorCalls0
exportNewOutcomeCalculations0
recordedScientificPlannerCallsRound825152
actualHumanReviews0
actualDeferrals0
humanSecondsNull — not available in this export
humanCostNull — not available in this export
computeCostNull — not available in this export
notes
Provenance manifest: effort: notes / 2 records
Provenance manifest: effort: notes / 2 records
RecordValue
1Recorded scientific compute is historical; exporter only reads, verifies and serializes saved artifacts.
2Human effort and monetary costs have not been measured.
aiUseDisclosureAI coding/research assistants developed the simulation and export tooling and helped interpret saved results. The displayed values are copied from hash-verified saved artifacts; narrative labels and example selection are illustrative.
limitations
Provenance manifest: limitations / 13 records
RecordValue
1Four examples were selected retrospectively from inspected development results; they are illustrative, not representative or fresh validation.
2Amazon travel matrices are historical averages. Current traffic, ages, report faults and supporting feeds are simulated.
3All snapshots restart a complete route at clock zero. timeMinutes selects the saved report snapshot, not an already driven route.
4Lateness is measured at service start against sparse package deadlines. Stop windows summarize cached bounds; they do not replace the original package deadline arrays.
5Coordinates are rounded to three decimal places. Orders are schematic directed stop sequences, not street geometry.
6Per-stop package counts are unavailable in the cached route object and remain null; no per-package data is exported.
7Estimated metrics only copy saved observable schedules. Missing estimated validity remains null; retrospective evaluation never fills estimated fields.
8At most 24 selected edges per plan are shown, prioritized by observable repair/report disagreement. The full order is always retained.
9Shared supporting observations are not independent per-edge reports; supportCount at snapshot level copies the saved unique_feed_reports counter.
10Original raw is an older-budget historical plan and is the exact guard fallback; matched raw uses common starts and the new budget.
11No new model loads, training, prediction, planning, simulation, reference-matrix decoding or outcome evaluation occurs in this export.
12No policy qualified in round eight. No real Amazon savings, optimality, calibrated safety or unseen-test claim for S79.
13Actual human review/deferral count is zero; human effort, cost and inference uncertainty are unmeasured/null.

Team and responsibilities

Team: Not yet provided
Members: Not yet provided

Responsibility assignments
ResponsibilityOwner
Data provenance and simulation assumptionsNot yet provided
Repair models and route planningNot yet provided
Evaluation and independent validationNot yet provided
Human review and escalationNot yet provided
Website, recording and presentationNot yet provided