We introduce Infinite Hidden Markov (IHM) models to forecasting realized volatilities of crude oil futures markets with exogenous factors. With these IHM models, we lift the restriction of a pre-defined number of regimes and allow for an unknown number of different parameter regimes and breakpoints. We employ two types of infinite hidden Markov models to accommodate structural breaks incurred by policy changes, exogenous shocks, and other factors. We find that IHM-HAR models outperform all other non-switching variants. In regard to forecasting performance, IHM-HAR models with exogenous factors such as realized volatilities of competing futures markets and the S&P500 are superior model choices for short-term forecasts. For longer-term forecasts, the equity channel shows only little positive impact. Evidence of economic gains in portfolio construction based on IHM-HAR forecasts is provided.