Cereal grains are the dominant source of cadmium in the human diet, with rice being to the fore. Here we explore the effect of geographic, genetic, and processing (milling) factors on rice grain cadmium and rice consumption rates that lead to dietary variance in cadmium intake. From a survey of 12 countries on four continents, cadmium levels in rice grain were the highest in Bangladesh and Sri Lanka, with both these countries also having high per capita rice intakes. For Bangladesh and Sri Lanka, there was high weekly intake of cadmium from rice, leading to intakes deemed unsafe by international and national regulators. While genetic variance, and to a lesser extent milling, provide strategies for reducing cadmium in rice, caution has to be used, as there is environmental regulation as well as genetic regulation of cadmium accumulation within rice grains. For countries that import rice, grain cadmium can be controlled by where that rice is sourced, but for countries with subsistence rice economies that have high levels of cadmium in rice grain, agronomic and breeding strategies are required to lower grain cadmium.