Competition Track

Overview

There are 16 official competitions at NeurIPS 2024, across physics and scientific computing, generative AI and LLMs, multi-agent systems and RL, signal reconstruction and enhancement, and responsible AI and security.

This page aims to give a brief introduction to each competition, as well as linking to the relevant resources.

Competition workshops take place on Saturday 14 and Sunday 15 December.

Retrospectives on last year’s competitions are now published in the Datasets and Benchmarks Track.

Physics and Scientific Computing

MyoChallenge

This competition requires participants to learn policies for dexterity and agility in human-like musculo-skeletal models. As in the 2023 challenge, there’s a manipulation track and a locomotion track, with this year’s tasks being more difficult than last year’s.

ML4CFD

In ML4CFD, participants try to model fluid flows over an aerofoil by predicting various regression targets on a point cloud.

An 8x NVIDIA RTX A6000 cluster was made available to participants in need of GPU resources.

Fair Universe HiggsML Uncertainty Challenge

This competition builds on the 2014 Kaggle HiggsML challenge, and focuses on uncertainty quantification in high-energy physics.

BELKA: Chemical Assessment

In BELKA, competitors predict small molecule-protein interactions — a successful method here would be valuable for drug discovery.

Generative AI and Large Language Models

Hacker Cup

This competition consists of a new AI track for Meta’s Hacker Cup coding competition, with participants building systems to solve programming problems within tight time constraints.

LLM Merging

The LLM Merging competition explores ways in which LLMs fine-tuned for specific tasks can be merged together into a generalist LLM, a relatively new area of research.

Edge-Device LLMs

This competition aims to push the boundary of LLM performance on edge devices, where compute resources are limited. Novel methods are encouraged, and so well-understood methods like quantisation of weights are not allowed. One track calls for compressing existing models, the other focuses on training from scratch.

Multi-Agent Systems and Reinforcement Learning

Auto-Bidding in Large-Scale Auctions

This competition explores auto-bidding strategies for online advertising. One track focuses on generative models, the other on reinforcement learning and similar approaches.

Lux AI Season 3

The third edition of this player-vs-player resource gathering competition just launched on Kaggle, and will conclude in March 2025. This version comes with an environment implemented in JAX, allowing for efficient batching of environment steps and easier large-scale training.

Concordia: Cooperative Intelligence of LLM Agents

This competition tests cooperation in AI agents, evaluating skills like promise-keeping, negotiation, reciprocity, reputation, partner choice, compromise, and sanctioning.

Signal Reconstruction and Enhancement

URGENT

The URGENT challenge organisers aim to build universal speech enhancement models. The competition requires participants to build a system to adaptively handle input speech with a variety of distortions, input formats, and acoustic environments.

Weather4Cast

In the Weather4Cast competition, participants forecast rainfall at high spatial and temporal resolution.

Ariel Data Challenge

In the fifth iteration of this astronomical data analysis challenge, participants extract exoplanetary signals (with uncertainty estimates) from simulated raw observational data, ahead of the 2029 European Space Agency Ariel Mission.

Responsible AI and Security

CLAS: LLM/Agent Safety

This AI safety-focused competition challenges participants to automate prompt injection attacks on LLMs and agents. There are tracks for jailbreaking and backdoor trigger recovery.

Image Watermarks Stress Test

Inspired by the WAVES benchmark, this competition aims to stress-test image watermarks. Participants remove invisible watermarks from images, while maintaining image quality.

LLM Privacy

The LLM Privacy competition explores strategies to mitigate privacy risks associated with LLMs, including during fine-tuning and prompt generation. The competition is split into two tracks: Red Team (finding exploits) and Blue Team (defending against vulnerabilities).

More info

For more info on the NeurIPS 2024 competitions, see the announcement on the NeurIPS blog, the official NeurIPS 2024 competition track page, or the accepted competition proposals on OpenReview.