Hybrid deep reinforcement learning for enhancing localization and communication efficiency in RIS-aided cooperative ISAC systems

In this paper, we propose a novel framework that combines simultaneous localization and communication (SLAC) using a reconfigurable intelligent surface (RIS) aided integrated sensing and communication (ISAC) systems. Our primary focus is on enhancing resource efficiency in such systems. We introduce Cloud Radio Access Networks (C-RAN) that facilitate collaboration between multiple base stations (BSs), enhancing cooperation benefits for both communication and sensing capabilities. To evaluate localization performance, we formulate an optimization problem to minimize the squared position error bound (SPEB) that reflects the system functional performance by optimizing the transmit beamformer, phase shift and subcarrier assignment under certain constraints. Moreover, in order to adjust the phase shift of the RIS, we propose a RIS aided cooperative ISAC SLAC protocol. This approach utilizes the measurements collected to refine the location and velocity estimates of the agent, as well as to reconstruct the environmental map with enhanced accuracy. However, the high dimensionality of the decision space makes the problem computationally intensive and challenging to navigate using gradient-based or exhaustive search methods. To efficiently tackle these issues, we construct a framework based on Markov Decision Processes (MDP) and address it by introducing a novel algorithm called hybrid deep reinforcement learning algorithm (HDRL). We validate our proposed algorithm through various simulations, demonstrating its effectiveness in improving system performance by comparing with the baseline schemes.