How accurate is it? We measured.
eBike Multnomah Falls · one shared 50-bike fleet on FareHarbor · 2026-08-01 to 2026-08-16 · 16 days
The founder owns this business, so its real reservation system could be read and compared, day by day, with what bookingcomp reconstructed from the public calendar alone. Below is every date in the audit window — nothing omitted, nothing adjusted after the fact.
Seats per service date. Value labels on every bar; the raw numbers are in the table below. On a phone, swipe the chart sideways.
Raw table
| Service date | Actual seats | Estimated seats | Error | Est. bookings |
|---|---|---|---|---|
| 2026-08-01 | 20 | ~26 | +6 | ~10 |
| 2026-08-02 | 38 | ~25 | -13 | ~9 |
| 2026-08-03 | 7 | ~5 | -2 | ~2 |
| 2026-08-04 | 12 | ~12 | 0 | ~5 |
| 2026-08-05 | 29 | ~29 | 0 | ~6 |
| 2026-08-06 | 10 | ~10 | 0 | ~3 |
| 2026-08-07 | 9 | ~7 | -2 | ~1 |
| 2026-08-08 | 7 | ~6 | -1 | ~2 |
| 2026-08-09 | 21 | ~21 | 0 | ~7 |
| 2026-08-10 | 9 | ~9 | 0 | ~3 |
| 2026-08-11 | 5 | ~5 | 0 | ~2 |
| 2026-08-12 | 6 | ~6 | 0 | ~2 |
| 2026-08-13 | 12 | ~12 | 0 | ~4 |
| 2026-08-14 | 10 | ~9 | -1 | ~3 |
| 2026-08-15 | 6 | ~6 | 0 | ~2 |
| 2026-08-16 | 4 | ~4 | 0 | ~1 |
| Total | 205 | ~192 | -13 (-6.3%) |
Estimated bookings are shown for completeness but are not scored — the fixture has no per-booking ground truth. Every estimate is labelled ~ because it is one.
Two scores, both shown
Production score when this model shipped (2026-08-05): MAE 1.81 seats/day,
11 of 16 dates exact, 200 vs 205 seats
(-2.4%) — recorded from
npm run backtest against the live database, and the figures quoted on the front page.
The per-day table above is the only archived per-day series: the same model re-run a day later
on a local copy of the observations, which scores MAE 1.56, 10 of 16 exact,
192 vs 205. The two runs differ by a few seats on the first two dates; we show
both rather than pick the flattering one. Both are inside the ~10% we claim.
How the audit was done
- Ground truth: per-departure seat usage read from the operator's own FareHarbor dashboard on 2026-08-05 (seats = sum of per-departure 'All Bikes' resource usage on listed departures; remaining = the number FareHarbor displays per departure). Checked into the repository as a fixture.
- Estimate: the same estimator customers get, run on the public availability readings for
the same dates, with no knowledge of the actuals. Scored by
scripts/backtest.mjs(node scripts/backtest.mjs --local (2026-09-03) over daily_totals recomputed 2026-08-06T23:35Z). - Error is estimated minus actual per date; MAE is the mean of the absolute errors.
Caveats — read these
- One operator, one platform, 16 days. A rental fleet on FareHarbor in peak season. A guided-tour company on another platform in shoulder season will score differently; possibly better (independent departures are easier), possibly worse.
- The two worst days are the first two. Aug 1–2 predate the hourly cadence, so several walk-up bookings landed inside one long gap and could not be separated from each other. Everything after the cadence change is within two seats.
- The operator's own calendar was filtered. Hourly rentals and staff holds on unlisted departures are not in the actuals but do draw the same bike pool, which is a source of both over- and under-count.
- No revenue ground truth. This audit scores seats only. Revenue inherits the seat error plus blindness to discounts and comps, and should be read as a range.
- The founder owns the benchmark business. That is what made the comparison possible; it is also why we would rather you run your own.
Audit us on your own books. Add your own company free, verify it with a one-line code on your website, and compare our numbers with your reservation system for as long as you like. Start here — no card needed for your own company.