Titanium (Ti) and its alloys have been frequently used in dental and orthopedic implants, but the undesired oxide layer easily formed on the surface tends to be the cause of implant failure for Ti-based implants. To address this problem, we herein prepared a phosphorylated Ti coating (TiP-Ti) with a micro/nano hierarchically structured topography on commercially pure Ti implants by a hydrothermal method to improve its osteointegration capacity. The surface morphology, chemical composition, and biological activity were characterized by scanning electron microscopy (SEM), transmission electron microscopy (TEM), X-ray diffraction (XRD), X-ray photoelectron spectroscopy (XPS), atomic force microscopy (AFM), contact-angle measurement, and protein adsorption assay. Osteointegration of TiP-Ti implants in rat tibia was investigated by biomechanical testing, micro-CT and histological analyses. We further explored the proposed mechanism which improves osteointegration of TiP-Ti implants by proliferation, adhesion, and differentiation assays of rat bone marrow mesenchymal stem cells (BMSCs). Our results demonstrated that the improved osteointegration mainly benefited from the better spread and adhesion of BMSCs on the micro/nano hierarchically structured TiP-Ti surfaces compared to hydroxyapatite coated Ti (HA-Ti), the positive control, and untreated Ti (untreated-Ti), the negative control. In conclusion, TiP-Ti surface is a promising candidate implant surface design to accelerate the osteointegration of Ti-based implants in biomedical applications.