Models

Runtime: classifier v1.1

The local classifier uses BioClinical ModernBERT, with separate heads for scope, disease and eleven signal categories. It supports triage inside SAGA's rules and review controls. The base model's MIT model card was checked on 2 September 2026.

The artifact lives in models/classifier-v1/, outside Git. Setup installs the release after checking its manifest. All model files are local during replay.

Training

Training used 389 frozen rows and 64 source-conditioned synthetic rows: 16 each for trial status, trial data, safety and guideline. Synthetic rows receive 0.25 category/disease loss weight and zero scope loss weight. Validation has 79 rows; the 95-row test remains sealed.

The fixed run used seed 13, four epochs, 512 tokens, batch 2, accumulation 4, encoder learning rate 2e-5 and head rate 1e-3. Validation did not select a checkpoint.

Only training records supplied source-format anchors. Generated examples had to meet structure, length, provenance, overlap and duplication checks, then pass two fresh label-blind critic rounds. Accepted synthetic records are train-only, tier D and never appear as source evidence. The augmentation manifest records the selection and hashes.

Promotion required scope F1 ≥0.9248, scope recall ≥0.9456, category accuracy ≥0.8861, supported-class macro F1 ≥0.6120, safety recall ≥1/5 and no new known scope false positives. The fixed run passed; the promotion report records the comparison. See Results for current scores and limits.

Earlier experiments

The v2 weak-label candidate was rejected: category accuracy fell from 0.8861 to 0.8481 without a safety gain. The first free-form synthetic augmentation also weakened scope recall and was stopped. Their data and machine reports remain for reproducibility; neither is a release model. Synthetic or AI-reviewed labels are not a substitute for human-reviewed holdouts.

Drafting

GPT-5.6 Luna prepares structured drafts from stored source passages. Replay uses saved responses; live mode can call the configured provider. Exact and semantic checks can reject a draft or replace it with source facts and review guidance. Generated interpretation remains a key evaluation gap.

Model evaluation must preserve the frozen split and keep the test sealed until a documented release evaluation. Use the v2 review instructions for new gold data.

Broader coverage experiment (September 2026)

A fixed four-epoch experiment added 200 AI weak-labelled training records: 180 fictional scenarios and 20 records anchored to exact public-source spans. All eleven labels received training support. An additional 64-case diagnostic corpus was built from separate source records and independently authored scenario forms, with explicit subtype evidence and title-only cases.

These are AI development labels, not a new human-reviewed benchmark. Source and scenario separation is recorded; semantic independence from pretraining or related products is not established.

The candidate used 0.35 augmentation category/disease weight and 0.25 augmentation scope weight. Its preregistered promotion gates required at least 73/79 category agreement, 43/45 scope recall, 38/45 disease agreement and 3/5 safety recall on the unchanged frozen validation labels.

The candidate was rejected. The released model remains v1.1. No checkpoint search or further validation tuning followed this result. Broader AI-labelled data did not solve the safety weakness. The fixed plan, candidate metrics, promotion decision and data provenance retain the evidence.

Two of the five frozen safety labels describe prospective safety objectives under the current annotation guide rather than observed findings. They remain unchanged in the benchmark; this label disagreement is a reason for fresh adjudication, not a reason to adjust the score.

Input coverage in the application

Runtime processing now uses up to four overlapping chunks and records partial coverage. Short inputs retain their historical behavior. Scope negatives with incomplete coverage, and uncertain title-only negatives, remain reviewable. The full permitted text also receives a safety scan.

Benchmark evaluation uses the original truncation path so a changed input policy cannot silently inflate model comparisons. These controls mitigate known failure paths; they do not establish full-system safety recall.