Outstanding Papers

There were three winners of the outstanding paper award this year, and three runner-up papers.

For the second time ever, ICLR is also awarding a test of time award for papers from ICLR 2015 which have had sustained impact. More on these tomorrow.

Listed below are the winners and runners-up of the outstanding paper awards.

Winners

Safety Alignment Should be Made More Than Just a Few Tokens Deep

Oral Presentation Paper

TL;DR: We identify an underlying problem (shallow safety alignment) that makes current safety alignment vulnerable, and we also propose approaches for mitigations.

Authors: Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, Peter Henderson.

Learning Dynamics of LLM Finetuning

Oral Presentation Paper

TL;DR: The paper propose a novel learning dynamics framework to understand LLM’s behavior during finetuning (e.g., SFT, DPO, and other variants). Some counter-intuitive behavior can be well explained by the proposed framework.

Authors: Yi Ren, Danica J. Sutherland.

AlphaEdit: Null-Space Constrained Model Editing for Language Models

Oral Presentation Paper

TL;DR: We propose a novel model editing method named AlphaEdit to minimize the disruption to the preserved knowledge during editing.

Authors: Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, Tat-Seng Chua.

Honourable mentions

Data Shapley in One Training Run.

Oral Presentation Paper

TL;DR: We develop a new notion of Data Shapley that requires only one model training run.

Authors: Jiachen T. Wang, Prateek Mittal, Dawn Song, Ruoxi Jia.

SAM 2: Segment Anything in Images and Videos.

Oral Presentation Paper

TL;DR: We present Segment Anything Model 2 (SAM 2), a foundation model towards solving promptable visual segmentation in images and videos.

Authors: Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollar, Christoph Feichtenhofer.

Faster Cascades via Speculative Decoding.

Oral Presentation Paper

TL;DR: Faster language model cascades through the use of speculative execution.

Authors: Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Seungyeon Kim, Neha Gupta, Aditya Krishna Menon, Sanjiv Kumar.