ICML 2024PastLarge language models

ICML 2024 Workshop on In-Context Learning

ICML 2024 Workshop ICL

Official website ↗OpenReview venue ↗See all ICML workshops →✎ Edit this entry

Submission deadline: May 28, 2024, 12:00 UTC
imported from OpenReview — check the website for extensions
Submission portal: OpenReview
Notes: Topics were auto-suggested and may be imprecise — edits welcome.

Accepted papers (39)

Fetched from OpenReview (v2) on 2026-06-10.

A Theoretical Understanding of Self-Correction through In-context Alignment
Yifei Wang, Yuyang Wu, Zeming Wei, Stefanie Jegelka, Yisen Wang · PDF
An In-Context Learning Theoretic Analysis of Chain-of-Thought
Chenxiao Yang, Zhiyuan Li, David Wipf · PDF
Automatic Domain Adaptation by Transformers in In-Context Learning
Ryuichiro Hataya, Kota Matsui, Masaaki Imaizumi · PDF
Can large language models explore in-context?
Akshay Krishnamurthy, Keegan Harris, Dylan J Foster, Cyril Zhang, Aleksandrs Slivkins · PDF
Can LLMs predict the convergence of Stochastic Gradient Descent?
Oussama Zekri, Abdelhakim Benechehab, Ievgen Redko · PDF
Can Mamba In-Context Learn Task Mixtures?
Yingcong Li, Xupeng Wei, Haonan Zhao, Taigao Ma · PDF
Can Transformers Solve Least Squares to High Precision?
Jerry Weihong Liu, Jessica Grogan, Owen M Dugan, Simran Arora, Atri Rudra, Christopher Re · PDF
Cross-lingual QA: A Key to Unlocking In-context Cross-lingual Performance
Sunkyoung Kim, Dayeon Ki, Yireun Kim, Jinsik Lee · PDF
DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning
Zijian Zhou, Xiaoqiang Lin, Xinyi Xu, Alok Prakash, Daniela Rus, Bryan Kian Hsiang Low · PDF
Fast Training Dataset Attribution via In-Context Learning
Milad fotouhi, Mohammad Taha Bahadori, Seyi Feyisetan, Payman Arabshahi, David Heckerman · PDF
Fine-grained Analysis of In-context Linear Estimation: Data, Architecture, and Beyond
Yingcong Li, Ankit Singh Rawat, Samet Oymak · PDF
Improve Temporal Awareness of LLMs for Domain-general Sequential Recommendation
Zhendong Chu, Zichao Wang, Ruiyi Zhang, Yangfeng Ji, Hongning Wang, Tong Sun · PDF
In-Context Generalization to New Tasks From Unlabeled Observation Data
Anthony Liang, Pavel Czempin, Yutai Zhou, Stephen Tu, Erdem Biyik · PDF
In-Context Learning from Training on Unstructured Data: The Role of Co-Occurrence, Positional Information, and Training Data Structure
Kevin Christian Wibisono, Yixin Wang · PDF
In-context learning in presence of spurious correlations
Hrayr Harutyunyan, Rafayel Darbinyan, Samvel Karapetyan, Hrant Khachatrian · PDF
In-Context Learning of Energy Functions
Rylan Schaeffer, Mikail Khona, Sanmi Koyejo · PDF
In-Context Principle Learning from Mistakes
Tianjun Zhang, Aman Madaan, Luyu Gao, Steven Zhang, Swaroop Mishra, Yiming Yang, Niket Tandon, Uri Alon · PDF
In-Context Reinforcement Learning Without Optimal Action Labels
Juncheng Dong, Moyang Guo, Ethan X Fang, Zhuoran Yang, Vahid Tarokh · PDF
In-Context Symmetries: Self-Supervised Learning through Contextual World Models
Sharut Gupta, Chenyu Wang, Yifei Wang, Tommi Jaakkola, Stefanie Jegelka · PDF
Learning Fast and Slow: Representations for In-Context Weight Modulation
Andrey Zhmoginov, Jihwan Lee, Max Vladymyrov, Mark Sandler · PDF
Learning Task Representations from In-Context Learning
Baturay Saglam, Zhuoran Yang, Dionysis Kalogerias, Amin Karbasi · PDF
Linear Transformers are Versatile In-Context Learners
Max Vladymyrov, Johannes von Oswald, Mark Sandler, Rong Ge · PDF
LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language
James Requeima, John F Bronskill, Dami Choi, Richard E. Turner, David Duvenaud · PDF
LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law
Toni J.B. Liu, Nicolas Boulle, Raphaël Sarfati, Christopher Earls · PDF
Localized Zeroth-Order Prompt Optimization
Wenyang Hu, Yao Shu, Zongmin Yu, Zhaoxuan Wu, Xiaoqiang Lin, Zhongxiang Dai, See-Kiong Ng, Bryan Kian Hsiang Low · PDF
Many-shot In-Context Learning
Rishabh Agarwal, Avi Singh, Lei M Zhang, Bernd Bohnet, Luis Rosias, Stephanie C.Y. Chan, Biao Zhang, Aleksandra Faust, Hugo Larochelle · PDF
Many-Shot In-Context Learning in Multimodal Foundation Models
Yixing Jiang, Jeremy Andrew Irvin, Ji Hun Wang, Muhammad Ahmed Chaudhry, Jonathan H Chen, Andrew Y. Ng · PDF
Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment
Max Wilcoxson, Morten Svendgård, Ria Doshi, Dylan Davis, Reya Vir, Anant Sahai · PDF
Probing the Decision Boundaries of In-context Learning in Large Language Models
Siyan Zhao, Tung Nguyen, Aditya Grover · PDF
Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars
Zhaoxuan Wu, Xiaoqiang Lin, Zhongxiang Dai, Wenyang Hu, Yao Shu, See-Kiong Ng, Patrick Jaillet, Bryan Kian Hsiang Low · PDF
Retrieval & Fine-Tuning for In-Context Tabular Models
Valentin Thomas, Junwei Ma, Rasa Hosseinzadeh, Keyvan Golestan, Guangwei Yu, Maksims Volkovs, Anthony L. Caterini · PDF
TabMDA: Tabular Manifold Data Augmentation for Any Classifier using Transformers with In-context Subsetting
Andrei Margeloiu, Adrián Bazaga, Nikola Simidjievski, Pietro Lio, Mateja Jamnik · PDF
Task Descriptors Help Transformers Learn Linear Models In-Context
Ruomin Huang, Rong Ge · PDF
Transformers are Minimax Optimal Nonparametric In-Context Learners
Juno Kim, Tai Nakamaki, Taiji Suzuki · PDF
Transformers as Stochastic Optimizers
Ryuichiro Hataya, Masaaki Imaizumi · PDF
Transformers Can Perform Distributionally-robust Optimisation through In-context Learning
Taeyoung Kim, Hongseok Yang · PDF
Transformers Learn Temporal Difference Methods for In-Context Reinforcement Learning
Jiuqi Wang, Ethan H Blaser, Hadi Daneshmand, Shangtong Zhang · PDF
Universal Self-Consistency for Large Language Models
Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, Denny Zhou · PDF
Verbalized Machine Learning: Revisiting Machine Learning with Language Models
Tim Z. Xiao, Robert Bamler, Bernhard Schölkopf, Weiyang Liu · PDF

Accepted papers (39)

☆A Theoretical Understanding of Self-Correction through In-context Alignment

☆An In-Context Learning Theoretic Analysis of Chain-of-Thought

☆Automatic Domain Adaptation by Transformers in In-Context Learning

☆Can large language models explore in-context?

☆Can LLMs predict the convergence of Stochastic Gradient Descent?

☆Can Mamba In-Context Learn Task Mixtures?

☆Can Transformers Solve Least Squares to High Precision?

☆Cross-lingual QA: A Key to Unlocking In-context Cross-lingual Performance

☆DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning

☆Fast Training Dataset Attribution via In-Context Learning

☆Fine-grained Analysis of In-context Linear Estimation: Data, Architecture, and Beyond

☆Improve Temporal Awareness of LLMs for Domain-general Sequential Recommendation

☆In-Context Generalization to New Tasks From Unlabeled Observation Data

☆In-Context Learning from Training on Unstructured Data: The Role of Co-Occurrence, Positional Information, and Training Data Structure

☆In-context learning in presence of spurious correlations

☆In-Context Learning of Energy Functions

☆In-Context Principle Learning from Mistakes

☆In-Context Reinforcement Learning Without Optimal Action Labels

☆In-Context Symmetries: Self-Supervised Learning through Contextual World Models

☆Learning Fast and Slow: Representations for In-Context Weight Modulation

☆Learning Task Representations from In-Context Learning

☆Linear Transformers are Versatile In-Context Learners

☆LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language

☆LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law

☆Localized Zeroth-Order Prompt Optimization

☆Many-shot In-Context Learning

☆Many-Shot In-Context Learning in Multimodal Foundation Models

☆Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment

☆Probing the Decision Boundaries of In-context Learning in Large Language Models

☆Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars

☆Retrieval & Fine-Tuning for In-Context Tabular Models

☆TabMDA: Tabular Manifold Data Augmentation for Any Classifier using Transformers with In-context Subsetting

☆Task Descriptors Help Transformers Learn Linear Models In-Context

☆Transformers are Minimax Optimal Nonparametric In-Context Learners

☆Transformers as Stochastic Optimizers

☆Transformers Can Perform Distributionally-robust Optimisation through In-context Learning

☆Transformers Learn Temporal Difference Methods for In-Context Reinforcement Learning

☆Universal Self-Consistency for Large Language Models

☆Verbalized Machine Learning: Revisiting Machine Learning with Language Models

A Theoretical Understanding of Self-Correction through In-context Alignment

An In-Context Learning Theoretic Analysis of Chain-of-Thought

Automatic Domain Adaptation by Transformers in In-Context Learning

Can large language models explore in-context?

Can LLMs predict the convergence of Stochastic Gradient Descent?

Can Mamba In-Context Learn Task Mixtures?

Can Transformers Solve Least Squares to High Precision?

Cross-lingual QA: A Key to Unlocking In-context Cross-lingual Performance

DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning

Fast Training Dataset Attribution via In-Context Learning

Fine-grained Analysis of In-context Linear Estimation: Data, Architecture, and Beyond

Improve Temporal Awareness of LLMs for Domain-general Sequential Recommendation

In-Context Generalization to New Tasks From Unlabeled Observation Data

In-Context Learning from Training on Unstructured Data: The Role of Co-Occurrence, Positional Information, and Training Data Structure

In-context learning in presence of spurious correlations

In-Context Learning of Energy Functions

In-Context Principle Learning from Mistakes

In-Context Reinforcement Learning Without Optimal Action Labels

In-Context Symmetries: Self-Supervised Learning through Contextual World Models

Learning Fast and Slow: Representations for In-Context Weight Modulation

Learning Task Representations from In-Context Learning

Linear Transformers are Versatile In-Context Learners

LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language

LLMs learn governing principles of dynamical systems, revealing an in-context neural scaling law

Localized Zeroth-Order Prompt Optimization

Many-shot In-Context Learning

Many-Shot In-Context Learning in Multimodal Foundation Models

Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment

Probing the Decision Boundaries of In-context Learning in Large Language Models

Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars

Retrieval & Fine-Tuning for In-Context Tabular Models

TabMDA: Tabular Manifold Data Augmentation for Any Classifier using Transformers with In-context Subsetting

Task Descriptors Help Transformers Learn Linear Models In-Context

Transformers are Minimax Optimal Nonparametric In-Context Learners

Transformers as Stochastic Optimizers

Transformers Can Perform Distributionally-robust Optimisation through In-context Learning

Transformers Learn Temporal Difference Methods for In-Context Reinforcement Learning

Universal Self-Consistency for Large Language Models

Verbalized Machine Learning: Revisiting Machine Learning with Language Models