Caching and multicast have surged as effective tools to alleviate the heavy load from the backhaul links while enabling content-centric delivery in communication networks. The main focus of work in this area has been on the cache placements to manage the network delay and backhaul transmission cost. An important issue of optimizing the cost efficiency in content delivery has not been addressed. This paper tackles this issue by proposing collaborative multicast beamforming in cache-enabled ultra-dense networks. The objective is to maximize the cost efficiency, which is defined as the ratio of the content throughput to the sum of power consumption and backhaul cost, in providing quality-of-service for content delivery. Zero-forcing beamforming and generalized zero-forcing beamforming are employed to force the multi-content interference to zero or mitigate it while amplifying the desired signals for users. These problems of collaborative multicast beamforming design are computationally difficult. Path-following algorithms, which invoke a simple convex quadratic program at each iteration, are developed for their solution. Numerical results are provided to demonstrate the computational efficiency of the proposed algorithms and also give insights into the impact of caching on the cost efficiency.