You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
importnumpyasnpfrompearsonifyimportPearsonifyfromsklearn.svmimportSVCfromsklearn.datasetsimportmake_classificationfromsklearn.model_selectionimporttrain_test_split# Generate synthetic classification datanp.random.seed(42)
X, y=make_classification(
n_samples=1000, n_features=20, n_informative=10, n_classes=2, random_state=42
)
# Split data into train, calibration, and test setsX_train, X_temp, y_train, y_temp=train_test_split(X, y, test_size=0.4, random_state=42)
X_cal, X_test, y_cal, y_test=train_test_split(X_temp, y_temp, test_size=0.5, random_state=42)
# Initialize Pearsonify with an SVC modelclf=SVC(probability=True, random_state=42)
model=Pearsonify(estimator=clf, alpha=0.05)
# Fit the model on training and calibration setsmodel.fit(X_train, y_train, X_cal, y_cal)
# Generate prediction intervals for test sety_test_pred_proba, lower_bounds, upper_bounds=model.predict_intervals(X_test)
# Calculate coveragecoverage=model.evaluate_coverage(y_test, lower_bounds, upper_bounds)
print(f"Coverage: {coverage:.2%}")
# Plot the intervalsmodel.plot_intervals(y_test_pred_proba, lower_bounds, upper_bounds)
Running example.py will generate the following plot:
This plot shows predicted probabilities with 95% confidence intervals, sorted by prediction score.
📖 References
Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied Logistic Regression. John Wiley & Sons.
Tibshirani, R. (2023). Conformal Prediction. Advanced Topics in Statistical Learning, Spring 2023.
📝 License
This project is licensed under the MIT License - see the LICENSE file for details.
About
Lightweight Python package for generating classification intervals in binary classification tasks using Pearson residuals and conformal prediction