Revisiting the ABCs of Working with AI: A Replication with Radiologists

CLAIM (Claim-Specific Citation Network audit, sometimes called CSN) is a forensic method for testing whether a scientific or medical claim's authority is supported by evidence or by citation dynamics. It detects citation bias, amplification, citation diversion, citation transmutation, dead-end citation, and back-door invention.

The ASSAY skill runs a structured, CLAIM-compatible extraction and integrity assessment on an article. Output is a verdict (sound, mixed, flagged, problematic, or cascade), a count of claims extracted, the central key claim, and an integrity note describing the structural read.

This scan restricts ASSAY to peer-reviewed publications and preprint servers. Journalism, opinion pieces, and government documents are evaluated under different frameworks (CAIHL for power and agency; editor's note for context).

ASSAY found the central claims partially supported. Some scaffolding holds; other parts of the argument lean on weaker or contested evidence. Read with the integrity note in mind.

Core findings on radiologist-AI collaboration (calibration drift, anchoring, automation bias) replicate under updated model conditions, with the magnitudes shifted but the directional results preserved — supporting the original conceptual framework while highlighting empirical recalibration needs.

Total claims extracted from the article: 7. The key claim is the single most load-bearing assertion the rest of the argument depends on.

Replication methodology is the contribution and is appropriate. Sample size and recruitment are honestly disclosed; single-specialty (radiology) scope is acknowledged. The 'directional results preserved' claim is well-supported; the 'magnitudes shifted' caveat is where readers will need to do their own translation to deployment decisions. Preprint, not peer-reviewed.

Editor's note: Replication is the methodological floor every clinical AI claim has to clear before it can move from preprint to deployment. The fact that this study had to be published as a replication five years after the original findings is the field's own admission that the move had not been made.

Summary: arXiv: Replication study testing whether prior-decade findings on radiologist-AI collaboration (calibration, anchoring, automation bias) hold up under updated model conditions and contemporary radiology workflow.

Read the original source → CAIHL read of this item →