Mimicking clinical trials with synthetic acute myeloid leukemia patients using generative artificial intelligence

NPJ digital medicine 2024 AI 7 Explanations View Original
Original Paper (PDF)

Unable to display PDF. Download it here or view on PMC.

Plain-English Explanations
Pages 1-2
The Data Problem Holding Back Leukemia Research

Acute myeloid leukemia (AML) is an aggressive blood cancer that starts in the bone marrow, where abnormal immature blood cells multiply uncontrollably and crowd out healthy cells. AML is genetically complex - with dozens of different mutations shaping how it behaves and responds to treatment - making it a particularly difficult disease to study.

Advancing AML research requires large datasets with detailed patient information including genetics, lab values, treatment history, and outcomes. But collecting this data is enormously costly and slow. Clinical trials that enroll hundreds of patients can take years to complete and require millions of dollars in funding. Meanwhile, privacy laws and institutional policies restrict sharing of patient data across institutions - a significant barrier that prevents smaller research centers from participating in data-driven science.

Generative artificial intelligence offers a potential solution: instead of sharing real patient data, researchers could generate synthetic data that statistically mimics the real data but does not correspond to any actual patient. This synthetic data could be shared freely while preserving privacy, potentially democratizing access to large clinical datasets.

This study tested whether two state-of-the-art generative AI technologies could create realistic synthetic AML patient datasets that accurately capture the biological complexity of the disease - including genetic mutations, laboratory values, and patient survival outcomes - using data from 1,606 real AML patients enrolled in four clinical trials.

TL;DR: This study tested whether AI could generate realistic synthetic datasets of AML patients that could be shared freely without privacy concerns, using data from 1,606 real leukemia patients across four clinical trials.
Pages 1, 2, 9
Two Approaches to Generating Synthetic Patients

The researchers used two fundamentally different generative AI architectures. The first, CTAB-GAN+, is based on a Generative Adversarial Network (GAN) - a framework where two neural networks compete against each other. A generator network creates synthetic patients from random noise, while a discriminator network tries to tell real from synthetic patients. This competition drives the generator to produce increasingly realistic data until the discriminator can no longer reliably distinguish them.

The second approach, Normalizing Flows (NFlow), works differently by learning a series of mathematical transformations that map a simple probability distribution (like a Gaussian bell curve) to the complex, high-dimensional distribution of real patient data. Because these transformations are invertible, the model can generate new samples by running a simple distribution backward through the learned transformations.

The training data included 50 molecular and cytogenetic variables per patient - covering mutations in genes that regulate epigenetics (DNMT3A, IDH1, IDH2, TET2), the cohesin complex (STAG2, RAD21), transcription factors (RUNX1, WT1, CEBPA), and key genes like FLT3, NPM1, and TP53. Cytogenetic abnormalities like specific chromosomal translocations were also included.

Critically, the model also had to learn time-to-event outcomes - how long patients survived and whether they achieved complete remission after chemotherapy. Modeling survival data is technically harder than modeling static variables because it involves censoring (patients still alive when the study ended) and the complex interplay between event-free and overall survival.

TL;DR: Two distinct AI architectures - a Generative Adversarial Network and Normalizing Flows - were trained on real AML patient data including 50 genetic variables and survival outcomes.
Pages 2-4
How Accurately the Synthetic Data Captured AML Biology

Both generative models scored highly on established performance metrics measuring how faithfully synthetic data represents real data. For distribution similarity (Regularized Support Coverage), CTAB-GAN+ scored 0.95 and NFlow scored 0.97 out of 1.0. For statistical properties like mean and standard deviation of continuous variables, both scored 0.91-0.92. These scores indicate that the overall shape and spread of patient characteristics were well preserved.

For the 50 genetic and cytogenetic variables, both models accurately reproduced mutation frequencies seen in the real cohort with only minor deviations. The exceptions were a handful of rare mutations - BCORL1, DNMT3A, PHF6, ZRSR2 for NFlow, and CUX1 and GATA2 for CTAB-GAN+ - which showed statistically significant but clinically small differences. Rare mutations are inherently harder to model because there are fewer examples to learn from.

Importantly, the models also preserved co-mutation patterns - the tendency for certain mutations to occur together or be mutually exclusive. For example, certain combinations of mutations are known to define biologically distinct AML subtypes. Both models showed high congruency with real data for co-occurrences, with deviations mainly affecting rare low-frequency combinations.

For patient outcomes, complete remission rates were similar between real (70.7%) and synthetic cohorts (CTAB-GAN+: 73.7%, NFlow: 69.1%) with no statistically significant differences. Overall survival curves showed plausible shapes in both synthetic cohorts, though event-free survival was somewhat harder to replicate precisely because it was not directly optimized during model training.

TL;DR: Both AI models accurately reproduced the genetic landscape, mutation co-occurrence patterns, and patient survival outcomes seen in real AML patients, with only minor deviations for rare variables.
Pages 5-6
Risk Associations Were Preserved

For synthetic data to be scientifically useful, it must not just look like real data on average - it must also preserve the relationships between variables. In AML, well-established genetic markers predict whether patients achieve remission, how long before the disease progresses, and overall survival.

The researchers tested whether these known associations held in the synthetic data. Using the ELN2022 risk stratification guidelines (the standard clinical framework for AML), they confirmed that every risk marker showed the same directionality in synthetic data as in real data: mutations associated with favorable outcomes remained favorable, and those linked to poor outcomes remained adverse. No inversions were observed.

Statistical significance was also preserved: associations that were statistically significant in real data remained significant in synthetic data, and non-significant associations remained non-significant. Only one exception occurred (del(5q) and remission achievement in NFlow-generated data), a minor discrepancy given the large number of comparisons made.

This preservation of risk associations means the synthetic data could legitimately support exploratory research questions - such as whether a specific genetic combination affects prognosis, or whether a subgroup of patients might respond differently to treatment. Researchers without access to real patient data could use these freely available synthetic datasets to generate and test hypotheses.

TL;DR: The synthetic AML patient data faithfully preserved known cancer biology: genetic risk factors that predicted good or bad outcomes in real patients retained the same associations in the AI-generated data.
Pages 6-7
Privacy Protection and Data Security

A fundamental concern with synthetic health data is whether a determined attacker could reverse-engineer the data to identify real patients. The researchers assessed privacy protection using Hamming distance - a measure of how many variables would need to be changed in a synthetic patient to match a real one exactly.

Critically, there were zero exact matches between any synthetic patient and any real patient across both AI models. The median Hamming distance was 9 for both models - meaning on average, 9 variables would need to be altered to turn a synthetic patient into a real one. This provides substantial protection against re-identification attacks.

A privacy leakage coefficient was calculated to detect whether synthetic data was suspiciously close to the training data (a sign of memorization rather than genuine generation). Both models scored well below the 0.05 threshold that would signal concern - 0.0178 for CTAB-GAN+ and 0.0069 for NFlow - confirming that neither model simply memorized the training patients.

The researchers contrast this with conventional privacy approaches like data anonymization (removing names and birthdates) or adding noise, which have been repeatedly shown to be vulnerable to re-identification attacks. Truly synthetic data generated by AI offers a fundamentally different - and potentially more robust - approach to enabling data sharing in healthcare research.

TL;DR: Neither AI model memorized real patients or allowed re-identification, with zero exact matches found and strong mathematical evidence of genuine privacy protection in the synthetic datasets.
Pages 7-8
Transforming Clinical Research with Synthetic Data

The practical implications of this work extend well beyond leukemia research. The researchers outline several concrete use cases for synthetic AML data: exploratory analysis to generate hypotheses about genetic associations; training and validating machine learning models that predict patient outcomes; augmenting small proprietary datasets to increase statistical power; and potentially replacing control groups in future clinical trials.

The possibility of using synthetic patients as clinical trial control groups is particularly transformative. If a new treatment is being tested, the control group - patients receiving standard care - could theoretically be replaced by synthetic patients matched to the investigational cohort. This could reduce the number of real patients exposed to potentially inferior treatment and dramatically cut trial costs and duration.

The study acknowledges important limitations and regulatory considerations. The synthetic data was generated from Middle-European patients receiving intensive chemotherapy, and may not represent other populations or treatment approaches. Current regulatory frameworks (GDPR, HIPAA) were not designed with synthetic data in mind, leaving significant legal ambiguity around its use in formal clinical trial submissions.

The researchers also note the chicken-and-egg problem of synthetic data: generating high-quality synthetic data requires large real datasets, but those large datasets are exactly what synthetic data is meant to substitute for. This suggests synthetic data generation is most powerful when seeded with large multicenter collaborative datasets before being distributed more broadly.

TL;DR: High-quality synthetic AML data could democratize cancer research by enabling hypothesis testing, machine learning development, and potentially replacing expensive clinical trial control groups - though regulatory frameworks still need to catch up.
Pages 8-9
Open Access Data to Accelerate AML Research

A distinctive feature of this project is that both synthetic AML datasets - each containing 1,606 patients with comprehensive clinical, genetic, and outcome variables - were made fully publicly available through the Zenodo data repository. This means any researcher worldwide can download and analyze rich AML patient-level data without institutional agreements, ethics approvals, or data sharing negotiations.

The availability of such data is particularly impactful for rare diseases like AML, where any single institution may only treat a few hundred patients per year. By pooling synthetic data from multiple trials and making it freely available, the researchers hope to enable scientific progress that would otherwise require years of multicenter collaboration.

The synthetic data also specifically represents patients treated with intensive anthracycline-based chemotherapy in German multicenter trials. Future work should extend this approach to newer targeted therapies (such as venetoclax, FLT3 inhibitors, or IDH inhibitors) as well as to real-world patient populations that differ from clinical trial participants.

This study establishes an important proof of concept that generative AI can create realistic, privacy-preserving synthetic data for rare blood cancers. It opens a path toward a future where data scarcity no longer limits cancer research - where the genetic complexity of AML can be studied at scale without requiring access to restricted patient records.

TL;DR: The researchers made both synthetic AML datasets publicly available, demonstrating how generative AI could overcome the data scarcity that limits research into rare cancers like acute myeloid leukemia.
Citation: Open Access, 2024. Available at: PMC10954666.