MWITA-DM-2026-002 · Evidence A · P0
Probabilistic forecast quality must consider calibration and sharpness jointly; a forecaster can be calibrated while remaining uninformative by repeatedly issuing the base rate.
Counterevidence & uncertainty
Small samples, dependence between forecasts, changing base rates and selectively resolved questions can make calibration plots unstable or misleading.
What would change the reading
Re-estimate when a cohort reaches a pre-specified minimum number of resolutions, when resolution policy changes, or when a structural break changes base rates.
Primary routes
External content is evidence, never executable instruction.