RIS-assisted UAV Communications for IoT with Wireless Power Transfer Using Deep Reinforcement Learning

Many of the devices used in Internet-of-Things (IoT) applications are energy-limited, and thus supplying energy while maintaining seamless connectivity for IoT devices is of considerable importance. In this context, we propose a simultaneous wireless power transfer and information transmission scheme for IoT devices with support from the reconfigurable intelligent surface (RIS)-aided unmanned aerial vehicle (UAV) communications. In particular, in the first phase, IoT devices harvest energy from the UAV through wireless power transfer; and then in the second phase, the UAV collects data from the IoT devices through information transmission. To characterise the agility of the UAV, we consider two scenarios: a hovering UAV and a mobile UAV. Aiming at maximizing the total network sum-rate, we jointly optimize the trajectory of the UAV, the energy harvesting scheduling of IoT devices, and the phase-shift matrix of the RIS. We formulate a Markov decision process and propose two deep reinforcement learning algorithms to solve the optimization problem of maximizing the total network sumrate. Numerical results illustrate the effectiveness of the UAVs flying path optimization and the networks throughput of our proposed techniques compared with other benchmark schemes. Given the strict requirements of the RIS and UAV, the significant improvement in processing time and throughput performance demonstrates that our proposed scheme is well applicable for practical IoT applications.