The performance of linear multi-user multiple-input multiple-output (MU-MIMO) systems has been extensively studied for classical statistical channel models. In contrast, there is little analysis for ray-based models, which are physically motivated, feature prominently in standards and have been experimentally validated. Thus, we present a novel analysis framework for zero forcing (ZF) and maximal ratio combining (MRC) applicable to such models. Specifically, using a central result for averaging in the angular domain, we derive accurate expressions for ZF signalto-noise ratio and MRC signal, interference and noise powers. The remarkably simple expressions offer the following insights into the effects of the propagation environment. While ZF is robust to parameters such as cluster and subray angle spreads, MRC interference is highly sensitive to them. We show that the performance scales linearly with the number of antennas, and that it degrades with narrow angle spreads and as the propagation moves toward the antenna end-fire. Finally, by evaluating the variance of the MRC interference, we observe that an approximation to the MRC SINR widely used for classical statistical models, is inaccurate in ray-based channels.