EXPERIMENT #001 · PUBLISHED
Arc H1 hESC CRISPRi
Negative primary result: target-ridge underperformed both declared baselines on panel MAE and RMSE.
GaiaLab Research Infrastructure
GaiaWorld
A machine willing to publish when it loses.
GaiaWorld is a scientific flight recorder for biological prediction experiments. It records evaluation boundaries, freezes models and predictions before scoring, compares them against declared baselines, and preserves the result whether positive, negative, or closed before reveal. Current evidence consists of three self-published retrospective benchmarks — one negative primary result and two narrow positive primary results — plus one pre-reveal eligibility closure. These records are independently inspectable but have not yet been independently reproduced.
Experiment #001 conclusion: the frozen target-ridge model did not beat the no-change or empirical-mean baselines on the predeclared primary panel-MAE metric. A modest secondary discrimination signal is preserved but is not relabeled as success.
Experiment #002 conclusion: the frozen baseline-residual-ridge model beat both declared baselines on a precommitted process-hidden 200-case K562 partition drawn from a public dataset — by a narrow ~1% relative margin over the stronger baseline, published with the full decomposition.
Experiment #003 conclusion: on a separate precommitted process-hidden RPE1 200-case partition of a public dataset, the frozen baseline-residual-ridge lane scored panel MAE 0.23221 versus 0.23818 for empirical mean and 0.29546 for no-change — a 2.5% relative improvement over the stronger baseline. This remains a retrospective benchmark result, not clinical or causal evidence.
Engineering proof · canonical self-published record
The machinery passed its engineering checks. The first biological benchmark was allowed to fail.
The engineering record covers deterministic execution controls, temporal cutoffs, gating, hashes, replay behavior and abstention. It is separate from biological performance and is re-verified in the browser against the version-controlled JSON record.
ENGINEERING PROOF · 9/9 DECLARED CONTROLS PASSED
Engineering controls
9/9 PASSEDEvery declared engineering assertion in the self-published zero-cost proof passed.
Arc H1 Experiment #001
SCORED · NEGATIVE PRIMARYExperiment #001 is a published retrospective result. The frozen target-ridge model underperformed both declared baselines on panel MAE and RMSE, with only a modest secondary discrimination signal.
Scientific posture
FAILURE PRESERVEDThe primary metric was not moved after reveal. The raw score and lock hashes remain auditable.
Retrospective benchmark · Experiment #001
Arc H1 scientific flight recorder
Retrospective held-out evaluation on a pre-frozen 100-gene H1 hESC CRISPRi panel. These are GaiaWorld panel metrics, not official Arc leaderboard metrics.
Public benchmark state
Arc H1 hESC CRISPRi
Benchmark state
PUBLISHED
Retrospective 100-gene panel result · not official Arc leaderboard
01 · PROTOCOL READYMethods, leakage rules, baselines and evaluator declared.
02 · SEALEDPrediction identities and scoring lock frozen before reveal.
03 · SCOREDHeld-out outcomes introduced once under the locked scorer.
04 · PUBLISHEDPositive or negative, the result remains inspectable.
Verification record
Benchmarkgaiaworld-arc-h1-2025-v1
Scoring lock SHA-25607a540e1e60e130547e434ff6300b030768a265ac02edca7535b724ce5a07108
Score semantic hashd896bc70b7555a867304b90a6237ade5025176fe556abbfef318e8d966fe00da
Lock recorded at2026-08-30T04:20:52Z
Target-ridge performance
Case coverage100.0%
Abstention0.0%
Panel MAE0.1326
Panel RMSE0.2064
Direction—
Replay integrity100.0%
Experiment #001 is scored. Primary result: negative. Target-ridge panel MAE was worse than both declared baselines; the secondary panel-discrimination signal does not override the primary result.
Public experiment registry
Every experiment gets a number. Losing does not erase the record.
One negative primary result, two narrow positive retrospective results, and one eligibility closure before reveal. A closed experiment is not scored and receives no scientific verdict.
EXPERIMENT #002 · PUBLISHED
Replogle K562 CRISPRi
Narrow positive primary result on a precommitted process-hidden partition of a public dataset: ~1.02% relative panel-MAE improvement over empirical mean.
EXPERIMENT #003 · PUBLISHED
Replogle RPE1 CRISPRi
Narrow positive primary result on a separate process-hidden public-dataset partition: 2.51% relative panel-MAE improvement over empirical mean, with 33 abstentions.
EXPERIMENT #004 · CLOSED BEFORE REVEAL
Norman K562 CRISPRa candidate
The predeclared gate required exactly 105 eligible single-gene conditions; identity-only inspection found 81. The experiment closed before partition creation, model fitting, prediction freeze, scoring, or outcome reveal.
What the result says
Negative on the primary metric. Mixed only in a secondary diagnostic.
The baseline beat the model on magnitude accuracy.
Lower panel MAE is better. The target-ridge model scored 0.13257 versus 0.07616 for no-change and 0.08195 for empirical mean. It won only 11% of cases against no-change and 8% against empirical mean.
No-change panel MAE0.07616
Empirical-mean panel MAE0.08195
Target-ridge panel MAE0.13257
The target-ridge lane did show a panel discrimination diagnostic of 0.5332 versus about 0.505 for both baselines, a +0.0282 difference. That measure was secondary, adapted to the frozen 100-gene panel, and is not official Arc PDS.
Direct-evidence lane
Abstention was preserved instead of converted into fake precision.
The separately audited direct-evidence route admitted zero Arc-compatible transition effects. It therefore remained at 100/100 abstentions and zero coverage rather than fabricating expression-delta magnitudes from generic literature confidence or association signals.
Next experiment
Experiment #001 is closed to tuning.
The revealed holdout can now inform diagnosis of failure, but it cannot be reused as untouched evidence for a revised model. Experiment #002 must use a genuinely new evaluation boundary and freeze its architecture, hyperparameters, metrics, baselines, prediction hashes and scorer before reveal.
Retrospective public-dataset benchmark · Experiment #002 · published
Positive on the primary metric. By a narrow, fully-disclosed margin.
The redesigned model beat both baselines — narrowly.
On a distinct evaluation context (Replogle K562 CRISPRi, precommitted process-hidden 200-case partition of a public dataset), the frozen baseline-residual-ridge model scored panel MAE 0.02498 versus 0.02523 for empirical-mean and 0.04705 for no-change — strictly better than both, meeting the pre-written success criterion. Against the stronger baseline it won 35 cases, lost 9, and tied 156; the top 5 winning cases contribute about 47% of the gain.
No-change panel MAE0.04705
Empirical-mean panel MAE0.02523
Baseline-residual-ridge panel MAE0.02498
The margin is ~1% relative over the stronger baseline, concentrated in the cases whose perturbation target is present in the measurement matrix. The panel discrimination diagnostic was 0.5123 versus 0.5025 for both baselines (+0.0098) — secondary, non-overriding, and not official Arc PDS. The evaluation outcomes were already public; the claim is process separation under a precommitted hash rule, not secrecy of the source outcomes.
Retrospective public-dataset benchmark · Experiment #003 · published
Positive on the primary metric. A separate RPE1 holdout.
The frozen ridge lane beat both declared baselines.
On the RPE1 CRISPRi precommitted process-hidden 200-case partition of a public dataset, the frozen baseline-residual-ridge lane scored panel MAE 0.23221 versus 0.23818 for empirical mean and 0.29546 for no-change. The relative margin over the stronger baseline is 2.5%.
No-change panel MAE0.29546
Empirical-mean panel MAE0.23818
Baseline-residual-ridge panel MAE0.23221
The scorer reports 33 abstentions. This is a one-time, retrospective, held-out 100-gene panel evaluation with a pre-committed split and frozen predictions. It does not establish general biological prediction ability, causal validity, patient benefit, safety, efficacy, or clinical utility.
Scientific receipts
Scientific claims should increasingly arrive with receipts.
A result is more useful when the path to it can be inspected. GaiaWorld records the parts of a benchmark that can otherwise be quietly rewritten after outcomes are known:
BEFORE REVEAL
What was frozen, when it was frozen, which code version ran, which data were available, and which baselines and primary metric were declared.
BOUNDARY
What changed, why it changed, what was excluded, and—where applicable—who reviewed the eligibility boundary.
AFTERWARD
What the system predicted before outcome reveal, what happened afterward, and the raw artifacts needed to check the account.
These receipts do not make a scientific result true, clinically useful, or independently validated. They make the process more inspectable and make it harder to quietly rewrite history after seeing the outcome.
Independent review · requested, not completed
Inspect the record. Challenge the method.
Current status: no independent reproduction has been completed or claimed. GaiaWorld's published experiments are self-published retrospective records, not externally validated findings. GitHub issue #130 remains the public request for independent reproduction, hash verification, and critique—including findings that identify limitations or discrepancies.
Scientific boundary
Experiments #001–#003 are self-published retrospective held-out GaiaWorld benchmarks on pre-frozen 100-gene panels. For #002 and #003, the underlying source outcomes were public; “process-hidden” means the evaluation partition was isolated under a precommitted rule, not that the outcomes were secret from the world. Experiment #004 closed before reveal and has no performance verdict. These records do not establish general biological prediction ability, causal validity, patient benefit, safety, efficacy, clinical utility, prospective predictive validity, or independent validation. DES and official Arc full-transcriptome leaderboard metrics were not computed.