NeurIPS 2024 Past Large language modelsInterpretabilityHealthcare & biology

Advancements In Medical Foundation Models: Explainability, Robustness, Security, and Beyond

AIM-FM Workshop @ NeurIPS'24

Submission deadline
Sep 26, 2024, 23:59 UTC
imported from OpenReview — check the website for extensions
Submission portal
OpenReview
Notes
Auto-imported from the OpenReview venue record on 2026-06-10 — please verify and enrich (topics are keyword-guessed).

Accepted papers (81)

Fetched from OpenReview (v2) on 2026-06-10.

  1. A Benchmark for Long-Form Medical Question Answering

    Pedram Hosseini, Jessica M. Sin, Bing Ren, Bryceton G. Thomas, Elnaz Nouri, Ali Farahanchi, Saeed Hassanpour · PDF
  2. A self-supervised framework for learning whole slide representations

    Xinhai Hou, Cheng Jiang, Akhil Kondepudi, Yiwei Lyu, Asadur Zaman Chowdury, Honglak Lee, Todd C Hollon · PDF
  3. A Self-Supervised Model for Multi-modal Stroke Risk Prediction

    Camille Delgrange, Olga V. Demler, Samia Mora, bjoern menze, Ezequiel De la Rosa, Neda Davoudi · PDF
  4. A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis

    Yue Yang, Mona Gandhi, Yufei Wang, Yifan Wu, Michael S Yao, Chris Callison-Burch, James Gee, Mark Yatskar · PDF
  5. Adaptive Reasoning and Acting in Medical Language Agents

    Abhishek Dutta, Yen-Che Hsiao · PDF
  6. An Autonomous Dual-channel Entity Recognition Method for Chinese Acupuncture

    Changshuai Zhang, Yuxin Zhang, Wenjun Tan · PDF
  7. Anatomical 3D Style Transfer Enabling Efficient Federated Learning with Extremely Low Communication Costs

    Yuto Shibata, Yasunori Kudo, Yohei Sugawara · PDF
  8. Application of Contrastive Learning on ECG Data: Evaluating Performance in Japanese and Classification with Around 100 Labels

    Junichiro Takahashi, JingChuan Guan, Masataka sato, Kaito Baba, Kazuto Haruguchi, Daichi Nagashima, Satoshi Kodera, NORIHIKO TAKEDA · PDF
  9. Armadillo: Robust Secure Aggregation for Federated Learning with Input Validation

    Yiping Ma, Yue Guo, Harish Karthikeyan, Antigoni Polychroniadou · PDF
  10. Assessment of Medical Foundation Models for Survival Prediction with Whole Slide Images

    Elena Spirina Menand · PDF
  11. ASTRID - An Automated and Scalable TRIaD for the Evaluation of RAG-based Clinical Question Answering Systems

    Mohita Chowdhury, Yajie Vera He, Ernest Lim, Aisling Higham · PDF
  12. Best of Both Worlds: Harmonizing LLM Capabilities in Decision-Making and Question-Answering for Treatment Regimes

    Hongxuan Liu, Zhiyao Luo, Tingting Zhu · PDF
  13. Biomedical SAM-2: Segment Anything in Biomedical Images and Videos

    Zhiling Yan, Weixiang Sun, Rong Zhou, Zhengqing Yuan, Kai Zhang, Yiwei Li, Sekeun Kim, Sifan Song, Hui Ren, Tianming Liu, Quanzheng Li, Xiang Li, Lifang He, Lichao Sun · PDF
  14. Catching the Spikes: Heteroscedastic Uncertainty Quantification for Enhanced Malaria Prediction

    Feng Chen, Qi Qi, Jiayu Qiu, Kemeng Zhang, Xiang Li · PDF
  15. Cell Instance Segmentation with Large Vision Model for Circulating Aberrant Cells Identification through Fluorescence In Situ Hybridization Images

    Zhenyu REN, Huajia Wang, Yinglan Kuang, Xing Lu · PDF
  16. Cell ontology guided transcriptome foundation model

    Xinyu Yuan, Zhihao Zhan, Zuobai Zhang, Manqi Zhou, Jianan Zhao, Boyu Han, Yue Li, Jian Tang · PDF
  17. Classification of Melanoma Skin Cancer with Ensemble Learning and Stratified K-Fold Validation

    Shanker Ram, Yajat Nagaraj Kiran · PDF
  18. Contextual Evaluation of Large Language Models for Classifying Tropical and Infectious Diseases

    Mercy Nyamewaa Asiedu, Nenad Tomasev, Chintan Ghate, Tiya Tiyasirichokchai, Awa Dieng, Geoffrey Siwo, Steve Adudans, Oluwatosin Wuraola Akande, Katherine A Heller · PDF
  19. Controlling for Unobserved Confounding with Large Language Model Classification of Patient Smoking Status

    Samuel Lee, Zach Wood-Doughty · PDF
  20. Data Adaptive Few-shot Multi-label Segmentation with Foundation Model

    Gurunath Reddy, Dattesh D. Shanbhag, Deepa Anand · PDF
  21. DAug: Diffusion-based Channel Augmentation for Radiology Image Retrieval and Classification

    Ying Jin, Zhuoran Zhou, Haoquan Fang, Jenq-Neng Hwang · PDF
  22. Deep Generative Models Unveil Patterns in Medical Images Through Vision- “Language” Conditioning

    Xiaodan Xing, Junzhi Ning, Yang Nan, Guang Yang · PDF
  23. Demographic Bias of Expert-Level Vision-Language Foundation Models in Medical Imaging

    Yuzhe Yang, Yujia Liu, Xin Liu, Wei Wu, Shwetak Patel · PDF
  24. Development and bilingual evaluation of Japanese medical large language model within reasonably low computational resources

    Issey Sukeda · PDF
  25. DiversityMedQA: A Benchmark for Assessing Demographic Biases in Medical Diagnosis using Large Language Models

    Rajat Rawat, Hudson McBride, Rajarshi Ghosh, Dhiyaan Chakkresh Nirmal, Jong Moon, Dhruv Alamari, Kevin Zhu, Sean O'Brien · PDF
  26. DMM: Distributed Matrix Mechanism for Differentially-Private Federated Learning using Packed Secret Sharing

    Alexander Bienstock, Ujjwal Kumar, Antigoni Polychroniadou · PDF
  27. Do Histopathological Foundation Models Eliminate Batch Effects? A Comparative Study

    Jonah Kömen, Hannah Marienwald, Jonas Dippel, Julius Hense · PDF
  28. Do Large Language Models have Shared Weaknesses in Medical Question Answering?

    Andrew Michael Bean, Karolina Korgul, Felix Krones, Robert McCraith, Adam Mahdi · PDF
  29. Enhancing Interpretability and Fairness in Medical Foundation Models: A Generative Approach for Explainable and Bias-Mitigated Medical Image Analysis

    Carlos Minutti · PDF
  30. Enhancing Medical NLP Systems: Integrating Upstash Vector and BGE-M3 for Accurate and Ethical Healthcare Data Management with Reduced Bias

    Binxi Xie · PDF
  31. Enhancing Trust in AI-Driven Dermatology: CLIP for Explainable Skin Lesion Diagnosis

    Sadia Kamal, Tim Oates · PDF
  32. Explaining Chest X-ray Pathology Models using Textual Concepts

    Vijay Sadashivaiah, Pingkun Yan, James Hendler · PDF
  33. Explaining Clusters Using Minimal Weighted Edge Coverage

    NIMA SAFAEI · PDF
  34. Exploring Fairness in Long-Term Prediction of Type 2 Diabetes Microvascular Complications

    Elizabeth Remfry, Rafael Henkin, Zainab Awan, Rohini Mathur, Michael R Barnes, Aakanksha Naik · PDF
  35. Fairness Of AI Models in vector embedded Chest X-ray representations

    Gebreyowhans Hailekiros Bahre, Hassan Hamidi, Francesco Calimeri, Andrew Sellergren, Leo Anthony Celi, Laleh Seyyed-Kalantari · PDF
  36. Federated Self-Supervised Single-cell Clustering of scRNA-seq Data

    Shentong Mo · PDF
  37. Gaze-Assisted Medical Image Segmentation

    Leila Khaertdinova, Ilya Pershin, Tatyana Shmykova, Bulat Ibragimov · PDF
  38. Generative AI in the Hospital: A Participatory Assessment of Healthcare Needs and Challenges

    Giorgia Carra, Bogdan Kulynych, François Bastardot, Noémie Boillat-Blanco, Jean Louis Raisaro · PDF
  39. Going beyond H&E and Oncology: how do Histopathology Foundation Models perform for multi-stain IHC & Immunology?

    Amaya Gallagher-Syed, ELENA PONTARINI, Myles J. Lewis, Gregory Slabaugh, Michael R Barnes · PDF
  40. GPT Sonograpy: Hand Gesture Decoding from Forearm Ultrasound Images via VLM

    Keshav Bimbraw, Ye Wang, Jing Liu, Toshiaki Koike-Akino · PDF
  41. Hetero-UNet: Heterogeneous Transformer with Mamba for Medical Image Segmentation

    Zhiling Yan, Yixin Liu, Xiang Li, Lichao Sun · PDF
  42. How Does Diverse Interpretability of Textual Prompts Impact Medical Vision-Language Zero-Shot Tasks?

    Sicheng Wang, Che Liu, Rossella Arcucci · PDF
  43. Improving the Classification Effect of Clinical Images of Diseases for Multi-Source Privacy Protection

    Bowen Tian, Zhengyang Xu, Zihao Yin, JingyingWang, Yutao Yue · PDF
  44. Integrating Image Interpretation and Textual Context for Improved Breast Imaging Classification

    Halil Ibrahim Gulluk, Olivier Gevaert · PDF
  45. Intraoperative Glioma Segmentation with YOLO + SAM for Improved Accuracy in Tumor Resection

    Samir Kassam, Angelo Markham, Katie Vo, Yashas Revanakara, Kevin Zhu, Michael Lam · PDF
  46. JRadiEvo: A Japanese Radiology Report Generation Model Enhanced by Evolutionary Optimization of Model Merging

    Kaito Baba, Ryota Yagi, Junichiro Takahashi, Risa Kishikawa, Satoshi Kodera · PDF
  47. Learning biologically relevant features in a pathology foundation model using sparse autoencoders

    Nhat Minh Le, Neel Patel, Ciyue Shen, Blake Martin, Alfred Eng, Chintan Shah, Sean Grullon, Dinkar Juyal · PDF
  48. Leveraging Multimodal Machine Learning for Predictive Diagnostics of Adolescent Mental Health Disorders

    Ankit Rao · PDF
  49. Lung-DETR: Deformable Detection Transformer for Sparse Lung Nodule Anomaly Detection

    Hooman Ramezani · PDF
  50. M3H: Multimodal Multitask Machine Learning for Healthcare

    Dimitris Bertsimas, Yu Ma · PDF
  51. MambaHealth: A Lightweight Foundation Model for Efficient Drug Recommendation

    Yuda Wang, Xuxin He, Shengxin Zhu · PDF
  52. Masked Modeling for Single-cell Clustering of scRNA‐seq Data

    Shentong Mo · PDF
  53. Med-FastSAM: Improving Transfer Efficiency of SAM to Domain-Generalised Medical Image Segmentation

    Yuxiang Luo, Qing Xu, Jinglei Feng, Guangwu Qian, Wenting Duan · PDF
  54. Medical Imaging Complexity and its Effects on GAN Performance

    William Cagas, Chan Ko, Blake T. Hsiao, Shryuk Grandhi, Rishi Bhattacharya, Kevin Zhu, Michael Lam · PDF
  55. MediConfusion: Can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models

    Mohammad Shahab Sepehri, Zalan Fabian, Maryam Soltanolkotabi, Mahdi Soltanolkotabi · PDF
  56. MSA-LM: Integrating DNA-level Inductive Biases into DNA Language Models

    Vishrut Thoutam · PDF
  57. Multimodal Lego: Model Merging and Fine-Tuning Across Topologies and Modalities

    Konstantin Hemker, Nikola Simidjievski, Mateja Jamnik · PDF
  58. Necessity of Uncertainty Quantification for Audio-driven Healthcare Diagnosis

    Shubham Kulkarni, Hideaki Watanabe, Fuminori Homma · PDF
  59. Neural machine translation of clinical procedure codes for medical diagnosis and uncertainty quantification

    Pei-Hung Chung, Shuhan He, Norawit Kijpaisalratana, Abdel Badih el Ariss, Byung-Jun Yoon · PDF
  60. PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals

    Ruiyi Wang, Stephanie Milani, Jamie C. Chiu, Jiayin Zhi, Shaun M. Eack, Travis Labrum, Samuel M Murphy, Nev Jones, Kate V Hardy, Hong Shen, Fei Fang, Zhiyu Chen · PDF
  61. PICASO: Secure Aggregation for Federated Learning with Minimal Synchronization

    Harish Karthikeyan, Antigoni Polychroniadou · PDF
  62. Precise Lens Status Classification via Projection Tuning for Efficient Adaptation to Data Shifts in Small Cataract Image Datasets

    Ji Young Byun, Jordan M Shuff, Kunal S Parikh, Rama Chellappa · PDF
  63. ProMISe: Promptable Medical Image Segmentation using SAM

    Jinfeng Wang, Sifan Song, Xinkun Wang, yiyi wang, Yiyi Miao, Jionglong Su, S Kevin Zhou · PDF
  64. Promoting cross-modal representations to improve multimodal foundation models for physiological signals

    Ching Fang, Christopher Michael Sandino, Behrooz Mahasseni, Juri Minxha, Hadi Pouransari, Erdrin Azemi, Ali Moin, Ellen L. Zippi · PDF
  65. QViSTA: A Novel Quantum Vision Transformer for Early Multi-Stage Alzheimer’s Diagnosis Using Optimized Variational Quantum Circuits

    Aryan Singhal, Hursh Shah · PDF
  66. Random Token Fusion for Multi-View Medical Diagnosis

    Jingyu Guo, Christos Matsoukas, Fredrik Strand, Kevin Smith · PDF
  67. Reliability in AI-Assisted Critical Care: Assessing Large Language Model Robustness and Instruction Following for Cardiac Arrest Identification

    Ugurcan Vurgun, Sy Hwang, Danielle L. Mowery · PDF
  68. Research Journey of Generative Protein Modeling

    Xinhui Chen, Yiwen Yuan, Joseph Liu, Chak Tou Leong, Zhen Xie, Xiaoye Zhu, Ying Chen, Songyue Chen, Chenyi Wang, Kun Li, Jie Zhang, Zuchao Li, Jiaqi Chen · PDF
  69. RespLLM: Unifying Audio and Text with Multimodal LLMs for Generalized Respiratory Health Prediction

    Yuwei Zhang, Tong Xia, Aaqib Saeed, Cecilia Mascolo · PDF
  70. SAM-MPA: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-prompting

    Jie Xu, LiXiaokang, Chengyuyue, Chen Ma, Yi Guo, Yuanyuan Wang · PDF
  71. Sentiment Reasoning for Healthcare

    Khai-Nguyen Nguyen, Khai Le-Duc, Bach Phan Tat, Le Duy, Jerry Ngo, Long Vo-Dang, Anh Totti Nguyen, Hy Truong Son · PDF
  72. Surgical SAM 2: Real-time Segment Anything in Surgical Video by Efficient Frame Pruning

    Haofeng Liu, Erli Zhang, Junde Wu, Mingxuan Hong, Yueming Jin · PDF
  73. Surgical-LLaVA: Toward Surgical Scenario Understanding via Large Language and Vision Models

    Juseong Jin, Chang Wook Jeong · PDF
  74. Synthetic Multimodal Data Generation and Training Optimization For Computed Tomography Cardiac Imaging Applications

    Masaki Ikuta, Quanqi Hu, Ashok Vardhan Addala, Ravi Soni, Avinash Gopal · PDF
  75. Towards a Personal Health Large Language Model

    Justin Cosentino, Anastasiya Belyaeva, Xin Liu, Zhun Yang, Yun Liu, Shyam A. Tailor, Tim Althoff, John B Hernandez, Yossi Matias, Greg Corrado, Shwetak Patel, SHRAVYA SHETTY, Jiening Zhan, Daniel McDuff, Cory Y McLean · PDF
  76. Towards Foundation Models for Critical Care Time Series

    Manuel Burger, Fedor Sergeev, Malte Londschien, Daphné Chopard, Hugo Yèche, Eike Christian Gerdes, Polina Leshetkina, Alexander Morgenroth, Zeynep Babür, Jasmina Bogojeska, Martin Faltys, Rita Kuznetsova, Gunnar Ratsch · PDF
  77. Training Free Adaptive Text Classification through Aggregated Large Language Models

    Iman Zare Estakhraji, Allison Li, Andreas Markus Loening, Akshay S Chaudhari, R. Marc Lebel · PDF
  78. TTT-UNet: Enhancing U-Net with Test-Time Training Layers for Biomedical Image Segmentation

    Rong Zhou, Zhengqing Yuan, Zhiling Yan, Weixiang Sun, Kai Zhang, Yiwei Li, Xiang Li, Quanzheng Li, Lifang He, Lichao Sun · PDF
  79. Universal Speech Disorder Recognition: Towards a Foundation Model for Cross-Pathology Generalisation

    Giulia Sanguedolce, Dragos-Cristian Gruia, Sophie Brook, Patrick Naylor, Fatemeh Geranmayeh · PDF
  80. VidLPRO: A Video-Language Pre-training Framework for Robotic and Laparoscopic Surgery

    Mohammadmahdi Honarmand, Muhammad Abdullah Jamal, Omid Mohareri · PDF
  81. Visual_Interpretability_of_Bioimaging_Deep_Learning_Models__neurips_

    Oded Rotem · PDF