Hepatocellular carcinoma (HCC) ranks as the sixth most common cancer globally and the third leading cause of cancer-related death, with particularly high incidence in East Asia and sub-Saharan Africa. Major risk factors include chronic hepatitis B and C infections, cirrhosis from excessive alcohol consumption, non-alcoholic fatty liver disease (NAFLD), and metabolic disorders. Despite advances in surgery, liver transplantation, and chemotherapy, outcomes remain poor. The targeted therapy sorafenib, a kinase inhibitor and first-line treatment for advanced HCC, provides only modest survival improvements. Late-stage diagnosis, high recurrence rates, and the complex tumor microenvironment continue to limit therapeutic success.
This review by Bhange and Telange (published in Discover Oncology, 2025) examines the convergence of two emerging technologies: nanotechnology and artificial intelligence. Nanotechnology enables the engineering of particles at the 1-100 nm scale for precise drug delivery, enhanced imaging contrast, and theranostic (combined diagnostic and therapeutic) platforms. AI contributes computational power for diagnostic imaging analysis, predictive modeling of disease progression, personalized treatment planning, and the optimization of nanoparticle design itself. The review synthesizes evidence from preclinical studies, case studies, and emerging clinical applications across these two fields.
AI-guided nanoparticle design: A central theme is how AI algorithms predict the ideal shape, size, and surface chemistry of nanoparticles. Zhang et al. demonstrated that AI-designed rod-shaped nanoparticles achieved 25% higher penetration into tumor tissues compared to spherical counterparts. A separate study using genetic algorithms optimized polymeric nanoparticle size, resulting in 18% enhanced drug accumulation in HCC tumors in preclinical models. Nanoparticles in the 10-100 nm range exploit the enhanced permeability and retention (EPR) effect for passive tumor targeting, but AI models also account for systemic clearance, kidney filtration, and macrophage uptake to fine-tune particle characteristics.
Scope of the review: The paper covers AI applications in diagnostics and imaging, personalized treatment planning, robotic surgery integration, AI-driven nanocarrier optimization, regulated cell death pathways, safety and toxicity prediction, regulatory and ethical concerns, and future directions for scaling these technologies to clinical practice.
AI has substantially improved liver cancer diagnosis across ultrasound, CT, and MRI modalities. Yasaka et al. demonstrated that convolutional neural networks (CNNs) detected liver tumors in CT scans with higher accuracy than traditional methods. Beyond simple detection, AI algorithms differentiate malignant from benign lesions, reducing the risk of unnecessary biopsies or surgeries. This is particularly important in liver cancer, where lesions frequently mimic benign growths and complicate clinical assessment.
Radiomics and predictive modeling: AI has been applied to optimize radiomic analysis, a process that extracts large volumes of quantitative features from medical images. These radiomic features correlate with the tumor's molecular and genetic profile, enabling personalized treatment decisions. When combined with machine learning (ML) algorithms, radiomics provides predictive insights into tumor behavior, prognosis, and likely treatment response. ML models analyze clinical, genetic, and imaging data simultaneously, forecasting disease progression, patient survival, and therapy response across variables including demographics, tumor characteristics, liver function, and prior treatment outcomes.
Treatment response prediction: ML models have been employed to identify biomarkers, such as gene expression profiles and specific mutations, that predict a patient's likelihood of responding to sorafenib. This extends to radiation therapy planning as well. AI algorithms leverage large datasets of patient outcomes to simulate various treatment plans, predicting how a tumor might shrink or grow under different radiation doses. This precision ensures radiation is delivered effectively while minimizing harm to healthy liver tissue.
Personalized treatment planning: AI-powered systems synthesize patient health records, genomic data, tumor biology, and drug response information to create tailored treatment approaches. ML algorithms recommend optimal systemic therapies based on gene expression patterns, suggest combination therapies (such as chemotherapy with immunotherapy), and predict graft survival, recurrence risk, and postoperative complications for liver transplant candidates. AI is also being used to select patients for clinical trials by predicting who is most likely to respond positively to experimental treatments based on genetic and clinical profiles.
The integration of AI into robotic-assisted surgery represents a significant advancement for liver cancer. Robotic systems offer greater precision, flexibility, and control compared to traditional open or laparoscopic procedures, which is especially critical when operating on complex hepatic structures. AI enhances these procedures by processing real-time imaging data and dynamically adjusting surgical tools. For example, AI can guide a robotic arm to remove a tumor while actively avoiding critical structures such as blood vessels and bile ducts, minimizing surgical complications and speeding recovery.
Real-time therapy monitoring: AI works alongside nanotechnology to monitor treatment in real time. AI algorithms analyze imaging data from nanoparticles used in drug delivery systems, tracking how well a liver tumor is responding to therapy. This provides clinicians with immediate feedback and enables rapid adjustments. The convergence of these technologies optimizes the potential of nanotechnology by utilizing AI's computational power to design more effective nanocarriers, monitor drug release, and improve therapeutic outcomes.
Theranostic nanoparticles: Nanoparticles designed for dual imaging and drug delivery generate simultaneous therapeutic and diagnostic data. A study by Zhao et al. used AI to process imaging data from gold nanoparticles functionalized with fluorescent markers and chemotherapeutic agents. The nanoparticles allowed real-time tracking of drug release and tumor uptake in liver cancer models. AI algorithms identified patterns of suboptimal drug delivery, leading to protocol adjustments that increased tumor suppression rates by 35%.
AI-powered adaptive systems also refine chemotherapy regimens using patient-specific data such as liver enzyme levels and imaging results. A study highlighted in Nature Medicine (2021) described an AI system that adjusted nanoparticle-based chemotherapy regimens for HCC, with adaptive dosing reducing treatment-related toxicity by 30% compared to standard protocols while maintaining therapeutic efficacy.
Liposomes: AI-driven models analyze drug release kinetics, lipid compositions, and surface modifications to optimize liposomal formulations. In the case of sorafenib-loaded liposomes, AI algorithms predicted the ideal lipid composition and particle size. In preclinical HCC animal models, these AI-optimized liposomes demonstrated superior drug accumulation in tumor tissues, reduced systemic toxicity, and enhanced therapeutic outcomes. Liposomes remain the most widely used nanocarriers for encapsulating chemotherapeutic agents due to their biocompatibility and tissue-targeting capability.
Polymeric nanoparticles and dendrimers: ML models predict polymer degradation rates to ensure sustained drug release over extended periods. PLGA (poly lactic-co-glycolic acid) nanoparticles enhanced using AI to encapsulate doxorubicin showed a 30% increase in therapeutic efficacy against HCC cells in vitro. Dendrimers, with their highly branched structures, provide multiple functionalization sites for drug molecules. AI-guided optimization of dendrimer surface chemistry has improved drug loading capacities and targeting efficiencies in challenging liver microenvironments. Additional nanocarrier types include solid lipid nanoparticles (SLNs), nanostructured lipid carriers (NLCs), and chitosan-based nanoparticles, each offering distinct advantages in stability, drug loading, and biocompatibility.
Gold nanoparticles for imaging: Gold nanoparticles (AuNPs) serve as excellent CT contrast agents due to their high atomic number and biocompatibility. AI algorithms optimize their functionalization with targeting ligands. In one application, AI designed ligands that bind specifically to glypican-3 (GPC3), a biomarker overexpressed on HCC cells. In animal studies, these AI-optimized AuNPs significantly enhanced CT imaging contrast, enabling precise tumor delineation and early detection. Quantum dots, nanoscale semiconductors that emit fluorescence when excited by light, have also been optimized by AI for surface coatings and emission spectra targeting alpha-fetoprotein (AFP), a common liver cancer biomarker, resulting in improved fluorescence imaging for early diagnosis and real-time surgical guidance.
Metal-based nanoparticles: Iron oxide nanoparticles (IONPs) serve dual roles in MRI-based diagnosis and hyperthermia treatment, where localized heating destroys cancer cells. Silver nanoparticles (AgNPs) demonstrate potent anticancer activity by inducing oxidative stress and apoptosis, though long-term toxicity concerns remain unresolved. AI is also advancing multimodal nanoparticles that combine CT, MRI, and fluorescence imaging capabilities for comprehensive tumor profiling, assessing size, vascularization, and metabolic activity simultaneously.
The heterogeneity of liver cancer poses significant challenges for conventional therapies. AI-driven personalized nanosystems address this by integrating patient-specific genomic, proteomic, and clinical data to design tailored therapeutic solutions. AI models analyze large datasets of genetic mutations, protein expressions, and tumor microenvironment characteristics to create nanoparticles that specifically target liver cancer subtypes.
Targeting mutant p53: Mutations in the p53 tumor suppressor gene are common in liver cancer. AI algorithms were used to develop lipid nanoparticles targeting mutant p53 proteins. These nanoparticles encapsulated siRNA specifically designed to silence mutant p53 and restore normal cell function. In preclinical models, this approach demonstrated selective tumor targeting and significant tumor regression. The study illustrates how multi-omics data integration (genomics, transcriptomics, and proteomics) enables the identification of novel biomarkers and therapeutic targets.
FGFR-targeted nanoparticles: AI-designed nanoparticles targeting liver cancer cells with overexpressed fibroblast growth factor receptors (FGFRs) achieved a 50% increase in drug uptake compared to non-targeted systems. This represents a substantial improvement in delivery specificity, driven entirely by AI analysis of receptor expression patterns and ligand-receptor interaction modeling.
Adaptive nanoplatforms: AI-powered nanoplatforms can adjust their properties in response to the tumor microenvironment. For instance, pH-sensitive nanoparticles developed using AI release their drug payload only in the acidic environment of liver tumors, minimizing damage to healthy tissues. These adaptive systems represent the frontier of precision nanomedicine, where nanoparticle behavior is dynamically matched to biological conditions. AI also integrates real-time feedback from clinical trials to continuously accelerate the optimization of nanocarriers and imaging agents.
Tumor nanomedicine increasingly focuses on inducing regulated cell death (RCD), including ferroptosis, apoptosis, and necroptosis. AI assists by analyzing complex datasets to determine the most effective nanoparticle-mediated RCD pathways for liver cancer cells. This multi-pathway approach is especially important for heterogeneous tumors like HCC, where a single cell death mechanism may be insufficient for comprehensive tumor eradication.
Ferroptosis induction: Ferroptosis is characterized by iron-dependent lipid peroxidation and accumulation of reactive oxygen species (ROS). Wang et al. (2023) used AI to design iron oxide nanoparticles coated with a hepatocyte-targeting peptide. These nanoparticles achieved a 40% higher tumor inhibition rate in liver cancer models compared to conventional treatments, with enhanced ROS production and lipid peroxidation within tumor cells effectively triggering ferroptosis. The study also demonstrated reduced systemic toxicity, highlighting the precision of AI-guided nanoparticle engineering. AI-designed ferroptosis-inducing nanoparticles offer a novel approach to circumventing chemotherapy and radiation resistance by exploiting vulnerabilities in cancer metabolism and oxidative stress pathways.
Apoptosis activation: AI models analyze high-throughput screening data to identify nanoparticle formulations that activate caspase-3 and caspase-9, the key mediators of apoptosis. Curcumin, a natural apoptosis inducer with poor bioavailability, was encapsulated in AI-optimized nanoparticles with precisely tuned size, charge, and surface functionalization. These AI-guided formulations showed higher caspase activation rates, leading to selective apoptosis in HCC cells. Preclinical studies reported a 50% increase in tumor regression compared to non-optimized formulations.
Precision oncology case studies: Chen et al. demonstrated AI-designed mesoporous silica nanoparticles (MSNs) loaded with cisplatin that achieved a 60% reduction in tumor size in HCC mouse models compared to conventional formulations. The AI fine-tuned pore sizes, surface functionalization, and drug release profiles to enhance cellular uptake and controlled drug release within the tumor microenvironment. Separately, ML algorithms optimized lipid nanoparticles targeting KRAS mutations, common drivers in liver and other cancers, achieving heightened specificity for KRAS-mutated cells with reduced off-target effects and enhanced tumor suppression in preclinical trials.
The translation of nanotechnology into clinical practice faces significant safety challenges, including cytotoxicity, immunogenicity, and long-term stability. AI mitigates these concerns through advanced predictive capabilities, optimization techniques, and real-time monitoring solutions. An ML model trained on a dataset of over 200 nanoparticle formulations, including their physicochemical properties and cytotoxicity profiles, demonstrated 92% predictive accuracy in determining cytotoxic potential. This computational screening reduces costs and risks before formulations advance to experimental or clinical testing.
Minimizing immune responses: The immune system often recognizes nanoparticles as foreign entities, leading to rapid clearance and reduced therapeutic efficacy. AI-driven optimization of PEGylation (coating nanoparticles with polyethylene glycol) determines the optimal thickness and density of PEG coatings to achieve maximum stealth properties while preserving functionality. Beyond PEG, AI has identified alternative biocompatible materials, such as zwitterionic polymers, that reduce immune recognition. For HCC-targeted nanoparticles, AI-driven designs have produced PEGylated liposomes with enhanced pharmacokinetics, prolonged systemic circulation, higher tumor accumulation, and minimal immune activation.
Real-time in vivo monitoring: Nanosensors embedded within therapeutic nanoparticles allow continuous monitoring of their behavior in the body. AI systems analyze sensor data to detect early signs of adverse reactions, such as premature degradation or accumulation in non-target organs, enabling timely interventions. Gold nanoparticles functionalized with nanosensors for liver cancer imaging and therapy have been enhanced with AI algorithms to monitor ROS levels, ensuring that nanoparticles induce therapeutic effects (such as ferroptosis) without crossing toxicity thresholds.
Adaptive treatment strategies: AI enables real-time feedback loops where treatment protocols are refined based on evolving patient data. In one example, lipid nanoparticles designed for RNA-based therapies were combined with AI algorithms that analyzed patients' genetic profiles and immune responses. The system dynamically adjusted the nanoparticle's lipid composition, resulting in a 25% increase in RNA delivery efficacy and reduced off-target effects. AI also predicts nanoparticle stability under various physiological conditions, analyzing temperature sensitivity, pH-dependent changes, and enzymatic degradation to suggest optimal construction materials with controlled degradation rates.
The intersection of AI and nanotechnology presents unique regulatory and ethical challenges. Regulatory agencies such as the FDA and EMA have established procedures for evaluating traditional drugs, but these frameworks are insufficient for AI-nanotechnology-based treatments. Nanoparticles vary in size, shape, surface charge, and composition, all of which affect safety, efficacy, and pharmacokinetics. AI-based algorithms introduce additional complexity through real-time data processing and patient-specific decision-making. Clear guidelines are needed that accommodate both the material properties of nanoparticles and the computational logic of AI algorithms, potentially through adaptive clinical trial designs where platforms are continuously monitored and optimized as new data is collected.
Algorithm transparency: Most AI systems, particularly those based on machine learning, function as "black boxes" where the decision-making process is not easily explainable. In clinical settings involving high-stakes cancer treatment decisions, physicians and patients need to understand how AI recommendations are made. Ensuring explainability and accountability of AI systems remains a key ethical challenge that current technology has not fully resolved.
Healthcare equity: AI models trained predominantly on Western population data may not generalize well to patients in East Asia or sub-Saharan Africa, where liver cancer incidence is highest. This creates a risk that AI-driven nanotechnology could exacerbate existing healthcare disparities rather than reduce them. The high cost of developing and manufacturing nanomedicines, involving complex multi-step production processes, further threatens accessibility in low- and middle-income countries.
Patient autonomy and consent: As AI takes on a more prominent role in treatment decisions, questions arise about informed consent for AI-based treatments involving novel nanomedicines. AI systems may recommend personalized treatment plans based on complex data analysis that patients, and even physicians, may not fully understand. The current consent processes may require reevaluation to accommodate this new paradigm.
Scalability: Patient-specific nanosystems optimized for unique genomic or proteomic profiles require highly customized formulations that are difficult to scale. AI offers solutions through generative adversarial networks (GANs) that predict nanoparticle configurations balancing personalization with manufacturability, and through modeling production workflows to optimize resource allocation and quality control. Integration of AI with robotic automation could enable real-time adjustments during production without compromising scalability.
Data limitations: AI algorithms rely on large, high-quality datasets, but liver cancer data is often limited, incomplete, or biased. Many training datasets are based on Western populations and may not generalize to regions where HCC is most prevalent. Medical imaging data can be noisy or inconsistent, degrading model performance. Overfitting, where models perform well on training data but fail on unseen data, remains a persistent challenge that cross-validation and regularization techniques only partially mitigate. Most evidence for AI-nanotechnology convergence comes from preclinical studies and animal models rather than prospective clinical trials.
Clinical workflow integration: Many current AI tools are not designed with clinicians in mind. They may require extensive training, and their recommendations may not be easily interpretable by healthcare professionals. Ensuring user-friendliness and actionable insights is critical for adoption. The nanoparticle field faces additional hurdles: biodistribution, cellular uptake, and clearance mechanisms of nanoparticles differ fundamentally from conventional drugs, necessitating novel clinical trial designs. Regulatory agencies may require additional preclinical data to assess the long-term effects of nanoparticle accumulation in organs like the liver.
Future prospects: In silico trials powered by AI, which simulate patient responses to nanoparticle therapies, show promise in reducing the time and cost of regulatory approvals. Advanced deep learning and multi-omics integration models can synthesize imaging, genomic, and clinical data to refine nanoparticle design further. Digital twins could simulate nanoparticle behavior in virtual patient models before clinical trials, significantly cutting development costs and timelines. Emerging areas include AI-enhanced nanoparticle-based gene therapies for HCC, where AI-driven models optimize the release kinetics of nanoparticles loaded with tumor-suppressing genes, and AI-designed nanorobots for targeted drug delivery.
The review concludes that overcoming these challenges requires interdisciplinary collaboration between AI researchers, nanotechnologists, clinicians, and regulatory bodies. The full promise of AI-enhanced nanomedicine for liver cancer will depend on developing diverse, high-quality datasets, establishing new regulatory frameworks, and validating preclinical findings in large-scale prospective clinical trials.