AlgoTune

Can language models optimize the runtime of popular algorithms like gzip compression, AES encryption or SVD? To answer this, we built AlgoTune, a benchmark consisting of more than one hundred widely used math, physics, and computer science functions. For each function, the goal is to write code that is faster than the reference implementation while producing the same outputs as the reference, on a held-out test set of inputs. In addition to the benchmark, we also developed AlgoTuner, an agent which enables language models to iteratively optimize code.

This site contains AlgoTuner trajectories for all AlgoTune tasks. Each entry shows the complete conversation between the model and the AlgoTune environment, including code edits, timing evaluations, and the iterative optimization process.