We consider utility maximization problems in the downlink cell-free massive multiple-input multiple-output (MIMO) whereby a large number of access points (APs) simultaneously serve a group of users. Four fundamental maximization objectives are of interest: (i) average spectral efficiency (SE), (ii) proportional fairness, (iii) harmonic-rate, and (iv) minimum SE of all users, subject to a sum power constraint at each AP. As considered problems are non-convex, existing solutions normally rely on successive convex approximation (SCA) and use off-the-shelf convex solvers, which implement an interior-point algorithm, to solve derived convex problems. The complexity of such methods scales quickly with the problem size. Therefore, we propose an accelerated projected gradient method to solve the considered problems. Particularly, each iteration of the proposed solution is given in a closed form and only requires the first order oracle of the objective, rather than the Hessian matrix as in known solutions, and thus is much more memory efficient. Numerical results demonstrate that our proposed solution achieves the same utility performance but with far less run-time, compared to the SCA method. Simulation results show that large-scale cell-free massive MIMO has the intrinsic user fairness, i.e. the four utility functions can deliver nearly uniformed services to all users.