The Gap That Real Images Leave Open
Medical imaging datasets are expensive to build and narrow in what they capture. Images sit in institutional silos, constrained by privacy rules and governance, and labeling them requires clinical expertise that does not scale. Even large collections can underrepresent rare diseases, minority populations, atypical presentations, or uncommon acquisition protocols. Those gaps do not merely slow model development — they shape what a model can learn and whom it will serve well. A classifier trained on a skewed sample inherits the skew, and the patients missing from the training set are the ones most likely to be misread.
A retrospective multimodal study published on arXiv tested whether a general-purpose image generator, Nano Banana, could close part of that gap. The researchers examined four modalities — chest radiography, fundus photography, CT, and carotid ultrasound — and asked three questions: whether synthetic images alone could train models that generalize to real scans, whether targeted synthetic augmentation could lift performance for underrepresented subgroups, and whether image editing could serve as a controlled auditing tool. The results show a concrete, bounded gain rather than a revolution.
The first finding is that synthetic-only training can work for some tasks. A classifier trained exclusively on generated chest radiographs reached an AUROC of 0.877 (95% CI 0.843-0.910) for pneumonia recognition on the RSNA dataset, against 0.908 for a model trained on the same number of real images — a difference that did not reach statistical significance. [1] For retinal detachment on fundus photographs, the synthetic-trained model achieved 0.963 (95% CI 0.948-0.975) on the public Eye Disease Image Dataset, comparable to the ophthalmic foundation model VisionFM (Qiu et al., 2024), which reached 0.937. For renal calculus detection on CT, the synthetic model reached 0.711 against 0.749 for real-data training, again without a significant gap. arXiv
Curation Is the Price of Admission
The gains did not come from the generator alone. Raw outputs carried digital watermarks, extended fields of view, and modality-specific border artifacts. When the researchers trained on uncropped chest radiographs, AUROC on the Chest X-Ray dataset fell from 0.792 to 0.643, and AUPR dropped from 0.902 to 0.814. [1] Modality-specific cropping — the lower 30 percent of each chest radiograph, 10 percent from the top and bottom of fundus images, and a tailored border trim for carotid ultrasound — restored the performance. The practical lesson is that a synthetic pipeline needs image review, artifact removal, and quality control aligned with real acquisition conventions; prompt generation by itself is not enough.
Feature-space analysis supported the same conclusion. Models fine-tuned on cropped synthetic radiographs produced more separable clusters than models trained on raw synthetic images or on ImageNet features alone, with higher accuracy and adjusted Rand index under unsupervised clustering. Grad-CAM visualizations showed attention over clinically plausible regions — consolidation in pneumonia cases, cardiothoracic structures in healthy controls — rather than over watermarks or borders.
Where Augmentation Pays Off Most
The clearest gain appears in subgroup representation. The Human Phenotype Project carotid ultrasound cohort skewed toward middle age: participants aged 44 or younger made up 18.8 percent of the data, and those 60 or older 19.7 percent. The researchers generated 240 synthetic carotid images targeting those two underrepresented groups and added them only to the training partition. Age prediction improved across every augmentation setting. For the younger subgroup, AUROC rose from 0.748 to 0.777 and AUPR from 0.400 to 0.417; for the older subgroup, AUROC climbed from 0.790 to 0.814 and AUPR from 0.467 to 0.504. The best overall result came from adding the 40-to-44 age group, lifting mean AUROC from 0.719 to 0.747 and mean AUPR from 0.524 to 0.566.
A size-matched control ruled out the simplest explanation. When 240 real images were removed and replaced with an equal number of synthetic ones, performance stayed roughly level — mean AUROC 0.729 with cropped synthetic replacements, 0.724 with raw, against 0.719 for real data alone. The improvement therefore came from added diversity, not from a larger training set.
That improvement carried into external validation. Applied to the CUBS cohort of 633 subjects, a model trained with synthetic-real mixed data produced an ordinal vascular age index that stratified participants into older- and younger-appearing subgroups matched for chronological age and sex. The vascularly older subgroup showed significantly higher BMI under both models, but significantly higher glucose only under the augmented model (p < 0.01), whereas the real-data-only model found no glucose difference (p = 0.428). Synthetic augmentation surfaced a cardiometabolic signal that the baseline missed.
Editing as a Microscope
Beyond generation, the model’s editing function turned real images into controlled test cases. Removing labels and side markers from RSNA chest radiographs left the Eva-x foundation model’s performance essentially unchanged — AUROC moved from 0.965 to 0.967 — indicating the model did not lean heavily on those shortcuts. Adding a small calcified granuloma to normal images, by contrast, dropped AUROC to 0.957 and AUPR to 0.968 (p < 0.001), with the mean predicted probability of the normal class among true-normal cases falling from 0.754 to 0.721. A subtle, clinically plausible perturbation exposed a decision boundary that marker removal did not.
Fundus editing produced a graded stress test. Under moderate tessellated patterns, the true-class probability for central serous chorioretinopathy fell from 0.996 to 0.822 and for preretinal hemorrhage from 0.983 to 0.857, while other categories held steady. Such perturbation sets give auditors a way to probe robustness without collecting new patient data.
Clinician-guided refinement closed the loop. An expert reviewing 100 generated CT images gave an overall plausibility score of 4.16 out of 5, but stone-positive cases scored only 3.40, with comments flagging overly rounded or artificial-looking stones. Feeding those comments back as editing instructions raised the mean score of the ten weakest images from 3.10 to 4.30. [1] Expert review became a refinement mechanism rather than a gate.
The Limit of Looking Real
Plausibility and fidelity are not the same thing. Across 95 edited fundus images, the mean plausibility score was 4.21 out of 5, with 77 images rated 4 or higher. But in the BMI-conditioned carotid task, experts correctly identified the higher-BMI image in only 18 of 50 pairs (36.0 percent), and the generated images did not consistently encode the intended attribute despite scoring 4.00 on plausibility. The Doppler stenosis task exposed a deeper problem: the expert reached 65.3 percent four-class accuracy, yet rated plausibility lowest at 3.47, noting inconsistencies among B-mode anatomy, color flow, spectral waveform, and expected hemodynamics. Component-level realism did not guarantee case-level physiological coherence.
Calibration told a similar story. Synthetic-trained models sometimes tracked the ideal diagonal more closely than real-trained ones, as in pneumonia and renal calculus, but the pattern did not hold across tasks and modalities, and synthetic-only training generally tended to overestimate risk. Comparable AUROC and AUPR did not translate into comparable risk estimation or clinical net benefit. The study’s interpretation is that synthetic data should be judged by calibration, decision curve analysis, and clinician review together, not by discrimination alone.
What the Institutions Will Have to Decide
The study’s own caveats point toward the consequence that is rarely stated outright. Synthetic images may reduce dependence on identifiable patient data during early experimentation, but they do not remove the need for provenance tracking, bias assessment, and external validation on representative real-world cohorts. General-purpose generators can also import their own biases: an unconstrained prompt asking for an older patient may summon large plaques or severe stenosis that were never intended, and downstream models will learn those associations. The institutional question is not whether synthetic data will enter medical AI development, but who will be accountable for the curation, the expert review, and the validation that make it safe. That work is unglamorous, labor-intensive, and hard to automate, and it is precisely the work on which the gains depend. arXiv