We compare two recently proposed methods that combine ideas from conformal inference and quantile regression to produce locally adaptive and marginally valid prediction intervals under sample exchangeability.

Accompanying paper:

• Matteo Sesia and Emmanuel J. Candes, "A comparison of some conformal quantile regression methods", 2019. arXiv:1909.05433

The methods we are comparing are described in:

• Yaniv Romano, Evan Patterson, and Emmanuel J. Candes, "Conformalized quantile regression", 2019.

• Danijel Kivaranovic, Kory D. Johnson, and Hannes Leeb, "Adaptive, Distribution-Free Prediction Intervals for Deep Neural Networks", 2019.

The code in this repository is a fork of https://github.com/yromano/cqr.

• python
• numpy
• scipy
• scikit-learn
• scikit-garden
• pytorch

The development version is available here on GitHub:

git clone https://github.com/msesia/cqr-comparison.git

The code available under 'experiments/' in the repository replicates the experimental results in our paper.

• Bike: Bike sharing dataset data set.

• Bio: Physicochemical properties of protein tertiary structure data set.

• Blog: BlogFeedback data set.

• Community: Communities and crime data set.

• Concrete: Concrete compressive strength data set.

• Facebook Variant 1 and Variant 2: Facebook comment volume data set.

• Homes: House sale prices for King County.

• STAR: C.M. Achilles, Helen Pate Bain, Fred Bellott, Jayne Boyd-Zaharias, Jeremy Finn, John Folger, John Johnston, and Elizabeth Word. Tennessee’s Student Teacher Achievement Ratio (STAR) project, 2008.

The Medical Expenditure Panel Survey (MPES) data can be downloaded using the code in the folder /get_meps_data/ under this repository. It is based on this explanation (code provided by IBM's AIF360).

• MEPS_19: Medical expenditure panel survey, panel 19.

• MEPS_20: Medical expenditure panel survey, panel 20.

• MEPS_21: Medical expenditure panel survey, panel 21.

This project is licensed under the MIT License - see the LICENSE file for details.