Beyond Prediction Accuracy: Testing What Biomedical Models Rely On
1st-place Owkin hackathon project: test whether pathology foundation-model predictions depend on meaningful tissue representations — with matched-random controls and structured evidence cards.
Team: Ishika Nahar · Ben Moskowitz · Sophia Zhou · Eddie Bae
Models
Phikon-v2
+ H-optimus-0 substrate
Tiles
5,400
NCT-CRC-HE · 9 classes
Tests
Probe→Ablate
Matched-random · specificity
Output
certify()
JSON evidence card
In one breath
THE PROBLEM
Problem
Pathology foundation models can score well and still rely on the wrong internal signal. Accuracy alone does not say what a prediction depended on.
MY ROLE & ACTION
What we built
A short evaluation battery — probe, ablate, matched-random, specificity — packaged as certify(prediction) evidence cards.
THE RESULT
Outcome
1st place at Owkin Hacking Biology. Working prototype on Phikon-v2 / H-optimus-0 + NCT-CRC-HE tiles — not clinical validation.
My contribution
- Built the local evidence-card dashboard UI (Case / Proof / Verdict cockpit) so the team could demo certify(prediction) without a GPU live.
- Hardened the UI for the zero-claims edge case so the demo fails gracefully.
- Collaborated on framing the certification idea as a structured evaluation workflow rather than only a visualization.
Team-built (not claimed as mine alone)
- · Causal evaluation battery: probe, ablation, matched-random null, specificity, confound hooks
- · MCP tools exposing certify / probe / ablate verbs
- · Embedding extraction and SageMaker GPU infrastructure for large ViTs
Screen recording of the K-Scope certify interface from the hackathon.
Try certify(prediction)
Pick a tissue claim, run the battery, and read the evidence card. Scores come from a recorded team artifact — not a live GPU pass in this browser.
Contrast: TUM vs NORM
Evaluation steps
- 01
Probe
Can we decode the concept from the embedding?
- 02
Ablate
Does removing that axis hurt the prediction?
- 03
Matched-random
Was the targeted edit worse than a random edit?
- 04
Specificity
Is the effect specific to this concept?
Architecture
- 01
H&E tissue tile
224×224 colorectal patches
- 02
Foundation model
Phikon-v2 / H-optimus-0
- 03
Multi-layer embeddings
CLS + mean-patch at selected depths
- 04
Probe & intervene
Decode vs dependence tests
- 05
Matched-random null
Control for generic damage
- 06
certify(prediction)
Structured evidence card
Three ideas behind the battery
Expand a card if you want the definition.
Why it mattered here
This is the core mistake accuracy-only evaluation makes in biomedical AI.
How I used it
Every evidence card reports probe scores next to necessity and matched-random controls.
Technical detail
Frozen embedding → linear probe; then targeted edit vs size-matched random edits.
What can go wrong
Strong probes can coexist with weak necessity when representations are redundant.
Limitations
- · Hackathon prototype — not clinical validation.
- · Model-internal dependence ≠ biological causality.
- · Tile-level evidence ≠ whole-slide interpretation.
- · Confound testing is limited without multi-site metadata.
What I learned
A representation can encode information without the final prediction depending on it — and a prediction can change after an intervention simply because the model was perturbed. Useful evaluation needs targeted tests, matched controls, and explicit limitations packaged as structured evidence.