ICML 2024 Past Large language models

First Workshop on Long-Context Foundation Models @ ICML 2024

LCFM 2024

Submission deadline
Jun 7, 2024, 12:59 UTC
imported from OpenReview — check the website for extensions
Submission portal
OpenReview
Notes
Auto-imported from the OpenReview venue record on 2026-06-10 — please verify and enrich (topics are keyword-guessed).

Accepted papers (30)

Fetched from OpenReview (v2) on 2026-06-10.

  1. Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling

    Yair Schiff, Chia Hsiang Kao, Aaron Gokaslan, Tri Dao, Albert Gu, Volodymyr Kuleshov · PDF
  2. CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory

    Zexue He, Leonid Karlinsky, Donghyun Kim, Julian McAuley, Dmitry Krotov, Rogerio Feris · PDF
  3. CD-Pos: Long Context Generalization in LLMs Through Continuous and Discrete Position Synthesis

    Zhiyuan Hu, Yuliang Liu, Jinman Zhao, Suyuchen Wang, WangYan, Wei Shen, Chao Yin, Bryan Hooi · PDF
  4. Demonstrations in In-context Learning for LLMs with Large Label Space

    Zhan Li, Fanghui Liu, Volkan Cevher, Grigorios Chrysos · PDF
  5. ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models

    Thibaut Thonet, Jos Rozen, laurent besacier · PDF
  6. FastDecode: High-Throughput LLM Serving through Disaggregating Attention Computation

    Jiaao He, Kezhao Huang, Jidong Zhai · PDF
  7. Fine-Tuning Medical Language Models for Enhanced Long-Contextual Understanding and Domain Expertise

    Qimin Yang, rongshengwang, CHEN JIEXIN, Runqi Su, Tao Tan · PDF
  8. From Text to Pixel: Advancing Long-Context Understanding in MLLMs

    Yujie Lu, Xiujun Li, Tsu-Jui Fu, Miguel Eckstein, William Yang Wang · PDF
  9. Improved Algorithms for Kernel Matrix-Vector Multiplication

    Piotr Indyk, Michael Kapralov, Kshiteej Sheth, Tal Wagner · PDF
  10. In-Context Learning with Long-Context Models: An In-Depth Exploration

    Amanda Bertsch, Maor Ivgi, Uri Alon, Jonathan Berant, Matthew R. Gormley, Graham Neubig · PDF
  11. InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory

    Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, Maosong Sun · PDF
  12. Learning to Reduce: Towards Improving Performance of Large Language Models on Structured Data

    Younghun Lee, Sungchul Kim, Ryan A. Rossi, Tong Yu, Xiang Chen · PDF
  13. Long Context Understanding using Self-Generated Synthetic Data

    Jerry Li, Subhro Das, Aude Oliva, Dmitry Krotov, Leonid Karlinsky, Rogerio Feris · PDF
  14. Long-Context Vision Large Language Models: Empirical Insights and A Baseline

    Yongshuo Zong, Ismail Elezi, Yongxin Yang, Jiankang Deng, Timothy Hospedales · PDF
  15. LongAlign: A Recipe for Long Context Alignment of Large Language Models

    Yushi Bai, Xin Lv, Jiajie Zhang, Yuze He, Ji Qi, Lei Hou, Jie Tang, Yuxiao Dong, Juanzi Li · PDF
  16. Many-Shot In-Context Learning

    Rishabh Agarwal, Avi Singh, Lei M Zhang, Bernd Bohnet, Luis Rosias, Stephanie C.Y. Chan, Biao Zhang, Ankesh Anand, Zaheer Abbas, Azade Nova, John D Co-Reyes, Eric Chu, Feryal Behbahani, Aleksandra Faust, Hugo Larochelle · PDF
  17. MINI-SEQUENCE TRANSFORMER: Optimizing Intermediate Memory for Long Sequences Training

    Cheng Luo, Jiawei Zhao, Zhuoming Chen, Beidi Chen, Anima Anandkumar · PDF
  18. Mitigate Position Bias in Large Language Models via Scaling a Single Dimension

    Yijiong Yu, Huiqiang Jiang, Xufang Luo, Qianhui Wu, Chin-Yew Lin, Dongsheng Li, Yuqing Yang, Yongfeng Huang, Lili Qiu · PDF
  19. MSAMamba: Adapting Subquadratic Models To Long-Context DNA MSA Analysis

    Vishrut Thoutam, Dina Ellsworth · PDF
  20. PhaseEvo: Towards Unified Long-Context Prompt Optimization for Large Language Models

    Wendi Cui, Jiaxin Zhang, Zhuohang Li, Hao Sun, Damien Lopez, Kamalika Das, Bradley A. Malin, Sricharan Kumar · PDF
  21. Position Coupling: Leveraging Task Structure for Improved Length Generalization of Transformers

    Hanseul Cho, Jaeyoung Cha, Pranjal Awasthi, Srinadh Bhojanapalli, Anupam Gupta, Chulhee Yun · PDF
  22. Pretrained Hybrids with MAD Skills

    Nicholas Roberts, Samuel Guo, Zhiqi Gao, Satya Sai Srinath Namburi GNVV, Sonia Cromp, Chengjun Wu, Chengyu Duan, Frederic Sala · PDF
  23. Probing the Decision Boundaries of In-context Learning in Large Language Models

    Siyan Zhao, Tung Nguyen, Aditya Grover · PDF
  24. RepoQA: Evaluating Long Context Code Understanding

    Jiawei Liu, Jia Le Tian, Vijay Daita, Yuxiang Wei, Yifeng Ding, Yuhan Katherine Wang, Jun Yang, LINGMING ZHANG · PDF
  25. Spatio-Spectral Graph Neural Networks

    Simon Geisler, Arthur Kosmala, Daniel Herbst, Stephan Günnemann · PDF
  26. Spectral State Space Models

    Naman Agarwal, Daniel Suo, Xinyi Chen, Elad Hazan · PDF
  27. Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack

    Xiaoyue Xu, Qinyuan Ye, Xiang Ren · PDF
  28. Vision-LSTM: xLSTM as Generic Vision Backbone

    Benedikt Alkin, Maximilian Beck, Korbinian Pöppel, Sepp Hochreiter, Johannes Brandstetter · PDF
  29. xLSTM: Extended Long Short-Term Memory

    Korbinian Pöppel, Maximilian Beck, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael K Kopp, Günter Klambauer, Johannes Brandstetter, Sepp Hochreiter · PDF
  30. ZigMa: A DiT-style Zigzag Mamba Diffusion Model

    Vincent Tao Hu, Stefan Andreas Baumann, Ming Gui, Olga Grebenkova, Pingchuan Ma, Johannes Schusterbauer, Björn Ommer · PDF