Feat Request: Conformal prediction

[txmed82]

Is your feature request related to a problem? Please describe.

I want to be able to give a prediction set with a coverage guarantee beyond the current calibration and TTA.

Describe the solution you'd like
-- Collect softmax on a held out calibration, compute non conformity scores (LAC:1-softmax), take the empirical quintile as threshold and at inference return score(y) <= threshold.
-- Then calibrate a single threshold that bounds an image level loss, put a per voxel uncertainty mask

Potentially:
inferers/conformal_predictor.py
metrics/conformal_risk.py

Describe alternatives you've considered
External libraries like TorchCP

Additional context
https://arxiv.org/pdf/2208.02814

[Whatsonyourmind]

Two design notes that might help scope this.

1. The LAC score (1 − softmax of the true class) gives marginal coverage but not class-conditional coverage. For medical segmentation where rare-class recall is the thing you actually care about, a Mondrian (class-conditional) split — calibrating a separate threshold per label — conditions coverage where it counts, at the cost of needing enough calibration voxels per class (the finite-sample floor bites hardest exactly on the rare classes, so it's worth a per-class count guard before reporting).

2. The two goals listed actually need two different machines with different guarantees. Returning {y : score(y) ≤ q̂} is vanilla split-CP set construction — set-valued, marginal coverage. Calibrating one threshold that bounds an image-level loss is conformal risk control / Learn-then-Test (the monotone-risk variant in arXiv:2208.02814), which is a different calibration loop entirely. I'd keep a split-CP set predictor and a CRC/LTT risk-bound path as separate components rather than sharing one threshold, since conflating them tends to silently give you neither guarantee.

Happy to sketch the calibration loop for either if it's useful.