# Week 7 — Simulation-to-real comparison

DIS OpenLab · Foundations of Autonomous Systems · v0.1.0-prepilot · 2026-10-15
Track: matched computational/simulation experiments; physical comparison optional only after approval
License proposal: CC BY 4.0 for original text, pending rights review

## Learning objectives

Learners will design matched experiments, identify parameters rather than hide them, calculate quantitative deviation, separate model discrepancy from measurement uncertainty, and propose the next experiment that most reduces uncertainty.

## Truth boundary

The supplied deterministic runner is not a physics simulator. Webots is a physics-based simulator but still a model. A physical run is evidence only for the exact assembled configuration and procedure. Do not fill physical columns with expectations, demonstration video estimates, or simulator values.

## Matched protocol

Define before running:

- start pose and coordinate frame;
- command profile and controller revision;
- duration and stop condition;
- wheel radius, wheelbase, mass/configuration where available;
- obstacle geometry/position and sensor mapping;
- time step and sample rate;
- output metrics and acceptance bands;
- number of repeat trials;
- environmental and safety conditions.

Suggested metrics: end-position error, heading error, minimum clearance, mission completion, stop reason, elapsed time, and command saturation count.

## Session A — parameter sensitivity

Run the nominal deterministic scenario. Then change exactly one model factor for each trial:

1. right wheel effective radius -5%;
2. obstacle signal shifted by +0.10 and clamped to 1;
3. controller step delayed beyond the watchdog once.

Record which output changed, direction of change, and whether the safe state activated.

## Session B — deviation report

Use this table. Rows without a source stay blank and are labeled `NOT RUN`.

| Metric | Nominal computational | Disturbed computational | Native Webots | Physical | Difference definition |
|---|---:|---:|---:|---:|---|
| End x m | | | | | |
| End y m | | | | | |
| Heading rad | | | | | |
| Minimum clearance | | | | | |
| Completion/stop reason | | | | | exact match / category |

For numeric pairs use a declared difference such as `observed - predicted`, absolute error, or percentage error. Percentage error is inappropriate when the reference is zero or near zero.

## Cause matrix

| Observed discrepancy | Plausible causes | Evidence that would distinguish them | Next experiment |
|---|---|---|---|
| Curved path on straight command | wheel radius mismatch, speed response, slip, alignment | encoder rates, wheel measurement, surface change | repeat with raised wheels then floor |
| Late obstacle turn | sensor mapping, update delay, threshold, approach speed | raw timestamped trace | fixed-distance sensor sweep |
| Unstable heading | gain, sample jitter, backlash, noise | command/error timing plot | fixed input with one gain change |

## Evidence to submit

- frozen matched protocol before results;
- nominal plus three one-factor disturbance runs;
- deviation table with evidence source per column;
- configuration hashes/versions and raw telemetry references;
- at least two competing explanations for one discrepancy;
- next experiment selected for information value, not appearance.

## Instructor notes

Do not allow teams to tune the model after every result without recording the prior configuration; that destroys traceability. A closer match is not automatically a more truthful model if multiple parameters were changed together.

Physical trial results may only be introduced by an approved adult-supervised host. If none exist, assessment uses the computational/simulator comparison and a correctly incomplete physical column.

## Low-bandwidth and accessibility option

Distribute printed nominal and disturbed telemetry extracts. Learners may compare tables using selected rows. Provide a plain-text description of arena geometry and permit a written experimental protocol instead of a drawn one.

## Exit ticket

Name one discrepancy that cannot be resolved by changing controller gains and identify the direct physical measurement required.
