Deep point reinforcement learning approach for sustainable communications by UAV and moving interaction station

Unmanned aerial vehicles (UAVs) have emerged as a critical component in the smart city, which can significantly enhance integrated sensing and communication (ISAC) performance. This paper mainly investigates the UAV-to-Vehicle (U2V) communication scenarios, where vehicles are represented as rigid shapes in the radar point cloud (RPC). The moving interaction station (MIS) is proposed to provide the sensing-assisted and wireless charging service for the UAV. The radio knowledge map (RKM) is introduced to improve the communication and energy efficiency of the UAV-ISAC system. Then, a joint optimization problem is formulated to complete the data collection and upload task by adjusting the UAV trajectory and vehicle access. To address this problem, a deep point reinforcement learning (DPRL) algorithm is proposed, which contains an RPC network, an RKM network, and a decision-making module. Herein, the RPC and RKM networks are designed to merge and map the vehicle RPC and RKM into the action spaces. The decision-making module selects actions from the action spaces to optimize the UAV trajectory and vehicle access. Simulation results show that the proposed DPRL algorithm outperforms the benchmarks, achieving approximately a 10.87% increase in channel capacity and a 24.08% enhancement in residual energy.