Regularized zero-forcing beamforming (RZFB) is an interesting class of linear signal processing problems, which is very attractive for use in large-scale communication networks due its simple visualization as a straightforward extension of the well-accepted zero-forcing beamforming (ZFB). However, unlike ZFB, which is multi-user interference free, RZFB must manage multi-user interference to achieve its high throughput performance. Most existing works focus on the performance analysis of particular RZBF schemes such as the equip-power allocated RZBF under a fixed regularization parameter. This paper is the first work to consider the joint design of power allocation and regularization parameter for RZFB to maximize the worst users' throughput or the quality-of-service awarded energy efficiency under a fixed transmit power constraint. Such designs pose very computationally challenging optimization problems, for which the paper proposes two-stage optimization algorithms of low computational complexity. Their computational and performance efficiencies are substantiated through numerical examples.