Prepared for Halcyon Labs (fictitious) · TargetSpace Pilot engagement · report TSR-SAMPLE-001
45-day evaluation window
Report version 1.0
Across 1,284 sealed forecasts on 12 consenting targets over 45 days, Meridian beat the population prior (R1) and each target's own routine (R2), stayed calibrated, and lost its skill when scored against the wrong target. Evidence ablation shows the passive-audio stream earns its cost on only one of three task families — the other two reach equivalent gated skill from calendar, messaging, and app-usage evidence alone.
All numbers on this page are invented for illustration. A real TargetSpace Report carries target-clustered confidence intervals on every estimate.
A memory-augmented personal assistant that claims to anticipate user needs from calendar, messages, app usage, and opt-in passive audio. Evaluated as a black box: the system emitted sealed probability distributions through the TargetSpace forecast adapter; no architecture details were required.
12 consenting targets · 45 days · forecasts sealed daily (SHA-256, externally witnessed) before outcomes existed · deterministic resolution rules pre-registered · organizer-issued queries · strict walk-forward evidence access · R1/R2 fit per target, admitted only where R2 beat R1.
| Task family | Question shape | Answer space | Forecasts | Resolved |
|---|---|---|---|---|
| Commitment resolution | Will commitment c resolve by r? | complete / defer / cancel / replace | 468 | 462 |
| Response behavior | Substantive reply to message m by r? | yes / no | 540 | 536 |
| Task continuation | Continue vs switch after interruption? | continue / switch | 292 | 286 |
Leave-one-out population base rates per question family and horizon. Meridian exceeded R1 on all three families (+0.31 to +0.44 bits). Beating R1 alone demonstrates only generic skill.
An organizer-fit, frozen model of each target's own habits (recency-weighted frequencies, persistence, time-of-day/week effects, calendar recurrences). R2 was admitted on all families. Skill over R2 is the headline: +0.14 bits/forecast.
Skill concentrates in transition-heavy strata: on routine-continuation instances the lift over R2 is near zero (+0.02), as expected — the signal appears where routine breaks. Synthetic illustration.
Reliability diagram, all families pooled. Slope 0.94, intercept +0.03, ECE 0.041 — within the pass band. Synthetic.
Classwise (one-vs-rest) diagnostics passed on all admissible classes; two rare classes below the minimum event count are reported non-assessable, not passed.
Re-scored against matched wrong targets, skill over R2 collapses to −0.52 bits — exceeding the pre-registered collapse margin. With the correct target but shuffled evidence order, skill drops from +0.14 to +0.05: temporal order carries most of the dynamic signal. Synthetic.
Each arm passed all gates before its skill counted. Passive audio contributes +0.05 bits overall — but the family split below shows the contribution is concentrated in one task family. Synthetic.
Gated skill saturates while observation cost keeps rising. Ties within the pre-registered equivalence margin rank the less invasive configuration first. Synthetic.
| Task family | Minimum sufficient tier | Audio earns its cost? |
|---|---|---|
| Commitment resolution | calendar + messages + app usage | No |
| Response behavior | calendar + messages | No |
| Task continuation | + passive audio | Yes |
Recommendation: audio capture can be scoped to the task-continuation surface (or disabled with a −0.02-bit cost on the other families) — a concrete data-minimization decision the evaluation licenses.
Skill is demonstrated on three task families over 45 days with 12 targets. It does not establish cross-task transfer, longer-horizon stability, or performance on other populations. Transition-stratum estimates carry wide intervals at this cohort size.
A passing report certifies calibrated, target-specific predictive skill on the evaluated families — not understanding in a richer sense, not consciousness, and not permission to act on forecasts. Deployment decisions require separate consideration.
| Criterion | Threshold | Result | Status |
|---|---|---|---|
| Target-specific lift (R2) | > 0 with 95% CI excluding 0 | +0.14 [0.06, 0.22] | PASS |
| Calibration | slope ≥ 0.85 · ECE ≤ 0.10 | 0.94 · 0.041 | PASS |
| Wrong-target collapse | exceeds pre-registered margin | −0.52 | PASS |
| Evidence minimization | MSO reported · non-paying streams identified | reported | PASS |
| Cohort power | powered replication (≥ pre-registered n) | 12 targets | PENDING |
Readiness verdict (synthetic): Ready pending powered replication — the system meets certification thresholds at pilot scale; a larger pre-registered cohort is required before any certification mark would issue. TargetSpace Certify is a readiness program; no certification marks have been issued to date.