ICML 2025 Past Large language models

ICML 2025 Workshop on Long-Context Foundation Models

LCFM 2025

Submission deadline
May 29, 2025, 11:59 UTC
imported from OpenReview — check the website for extensions
Submission portal
OpenReview
Notes
Auto-imported from the OpenReview venue record on 2026-06-10 — please verify and enrich (topics are keyword-guessed).

Accepted papers (35)

Fetched from OpenReview (v2) on 2026-06-10.

  1. Accelerated Inference with Long-Sequence Transformers on CPUs

    Yuzhen Mao, Martin Ester, Ke Li · PDF
  2. ALCo-FM: Adaptive Long-Context Foundation Model for Accident Prediction

    Pinaki Prasad Guha Neogi, Ahmad Mohammadshirazi, Rajiv Ramnath · PDF
  3. BSA: Ball Sparse Attention for Large-scale Geometries

    Cătălin-Emanuel Brița, Hieu Nguyen, Lohithsai Yadala Chanchu, Domonkos Nagy, Maksim Zhdanov · PDF
  4. CalibQuant: 1-Bit KV Cache Quantization for Multimodal LLMs

    Insu Han, Zeliang Zhang, Zhiyuan Wang, Yifan Zhu, Susan Liang, Jiani Liu, Haiting Lin, Mingjie Zhao, Chenliang Xu, Kun Wan, Wentian Zhao · PDF
  5. Dynamic Causal‐Graph Memory: Structured Retrieval for Million–Token Reasoning

    Thomas Y Chen · PDF
  6. e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMs

    Amrith Setlur, Matthew Y. R. Yang, Charlie Victor Snell, Jeremiah Greer, Ian Wu, Virginia Smith, Max Simchowitz, Aviral Kumar · PDF
  7. Enhancing Retrieval-Augmented Generation with Dehallucinating Parallel Context Extension

    Zexiong Ma, Shengnan An, Zeqi Lin, Yanzhen Zou, Jian-Guang Lou, Bing Xie · PDF
  8. Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions

    Yuanzhe Hu, Yu Wang, Julian McAuley · PDF
  9. Foreign Sparse Attention: Effective Distillation into Sparse Attention

    Vijaykaarti Sundarapandiyan, Tom Goldstein, Ashwinee Panda · PDF
  10. GSM-Infinite: How Do your LLMs Behave over Infinitely Increasing Reasoning Complexity and Context Length?

    Yang Zhou, Hongyi Liu, Zhuoming Chen, Yuandong Tian, Beidi Chen · PDF
  11. HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading

    Cheng Luo, Zefan Cai, Hanshi Sun, Jinqi Xiao, Bo Yuan, Wen Xiao, Junjie Hu, Jiawei Zhao, Beidi Chen, Anima Anandkumar · PDF
  12. How Much Context Does Natural Language Actually Require? An Analysis Using LLMs as Statistical Oracles

    Vala Vakilian, Sadegh Mahdavi, Christos Thrampoulidis · PDF
  13. Improving Context Fidelity via Native Retrieval-Augmented Reasoning

    Suyuchen Wang, Jinlin Wang, Xinyu Wang, Shiqi Li, Xiangru Tang, Sirui Hong, Xiao-Wen Chang, Chenglin Wu, Bang Liu · PDF
  14. Jailbreaking in the Haystack

    Rishi Rajesh Shah, Chen Henry Wu, Ziqian Zhong, Alexander Robey, Aditi Raghunathan · PDF
  15. Kinetics: Rethinking Test-Time Scaling Laws

    Ranajoy Sadhukhan, Zhuoming Chen, Haizhong Zheng, Yang Zhou, Emma Strubell, Beidi Chen · PDF
  16. Language Modeling with Learned Meta-Tokens

    Alok Shah, Khush Gupta, Keshav Ramji, Pratik Chaudhari · PDF
  17. Looking beyond the next token

    Abitha Thankaraj, Yiding Jiang, J Zico Kolter, Yonatan Bisk · PDF
  18. Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcement

    Xuechen Zhang, Zijian Huang, Chenshun Ni, Ziyang Xiong, Jiasi Chen, Samet Oymak · PDF
  19. MatMuls are Enough for Efficient and Performant Linear-Time Attention

    Andrew Argatkiny, Ilya Makarov · PDF
  20. MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning

    Dong Liu, Yanxuan Yu, Xuhong Wang, Ben Lengerich, Ying Nian Wu · PDF
  21. MMLongBench: Benchmarking Long-Context Vision-Language Models Effectively and Thoroughly

    Zhaowei Wang, Wenhao Yu, Xiyu REN, Jipeng Zhang, Yu Zhao, Rohit Saxena, Liang Cheng, Ginny Wong, Simon See, Pasquale Minervini, Yangqiu Song, Mark Steedman · PDF
  22. Multiscale Byte Language Models -- A Hierarchical Architecture for Causal Million-Length Sequence Modeling

    Eric Egli, Matteo Manica, Jannis Born · PDF
  23. NovelHopQA: Diagnosing Multi-Hop Reasoning Failures in Long Narrative Contexts

    Abhay Gupta, Kevin Zhu, Vasu Sharma, Sean O'Brien, Michael Lu · PDF
  24. OracleKV: Oracle Guidance for Question-Independent KV Cache Eviction

    Yuanbing Zhu, Zhenheng Tang, Xiang Liu, Ang Li, Bo Li, Xiaowen Chu, Bo Han · PDF
  25. Pause-Tuning for Long-Context Comprehension: A Lightweight Approach to LLM Attention Recalibration

    James Begin, Namit Agrawal, Eshan Singh, Yicheng Fu, Sean O'Brien, Vasu Sharma, Kevin Zhu · PDF
  26. PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluation

    Albert Gong, Chao Wan, Kamilė Stankevičiūtė, Anmol Kabra, Raphael Thesmar, Johann Lee, Julius Klenke, Carla P Gomes, Kilian Q Weinberger · PDF
  27. pLSTM: parallelizable Linear Source Transition Mark networks

    Korbinian Pöppel, Richard Freinschlag, Thomas Schmied, Wei Lin, Sepp Hochreiter · PDF
  28. Say as It Is: Verbatim Fidelity Evaluation of Long-Context Language Model

    Kyu Won Kim, Suhwan Choi, Myeongho Jeon · PDF
  29. Scalable LLM Math Reasoning Acceleration with Low-rank Distillation

    Harry Dong, Bilge Acun, Beidi Chen, Yuejie Chi · PDF
  30. Scaling Laws for Many-Shot In-Context Learning with Self-Generated Annotations

    Zhengyao Gu, Henry Peng Zou, Aiwei Liu, Yankai Chen, Weizhi Zhang, Philip S. Yu · PDF
  31. Simple, Scalable Reasoning via Iterated Summarization

    Vivek Vajipey, Aditya Tadimeti, Justin Shen, Ben Prystawski, Michael Y. Li, Noah Goodman · PDF
  32. SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling

    Krishna C Puvvada, Faisal Ladhak, Santiago Akle Serano, Cheng-Ping Hsieh, Shantanu Acharya, Somshubra Majumdar, Fei Jia, Samuel Kriman, Simeng Sun, Dima Rekesh, Boris Ginsburg · PDF
  33. Thinformer: Guaranteed Attention Approximation via Low-Rank Thinning

    Annabelle Michael Carrell, Albert Gong, Abhishek Shetty, Raaz Dwivedi, Lester Mackey · PDF
  34. Towards Understanding Self-Pretraining for Sequence Classification

    Omar Coser, Antonio Orvieto · PDF
  35. Unable to Forget: Proactive Interference Reveals Working Memory Limits in LLMs Beyond Context Length

    Chupei Wang, Jiaqiu Vince Sun · PDF