Beyond linear binning: logarithmic insights for calibrated machine learning in wireless systems

In this paper, we explore the calibration of a machine learning (ML)-based outage predictor aimed at optimizing resource allocation to minimize outages in communication systems. We model the wireless channel using an auto-correlated time series with samples distributed in accordance with a Rayleigh fading process. Our novel contribution concerns proposing histogram-based reliability plots that employ logarithmic binning to assess its impact on the model calibration compared to the traditional linear binning technique. We train our ML model using an outage loss function (OLF) tailored to this system and the well-known binary cross entropy (BCE). Additionally, we analyze the effect of different model parameters on the calibration performance of this outage predictor. Our findings demonstrate that logarithmic binning reveals nuanced calibration traits ignored by linear binning, particularly at lower confidence levels. This finding is crucial for wireless systems for which understanding behavior at very low probabilities is essential. Additionally, we observe that our ML model trained with OLF becomes more overconfident as the classification threshold increases and in scenarios characterized with rare events, namely outages. This observation serves as a tool for improving the calibration properties of OLF.