UAV-assisted dynamic avatar task migration for vehicular metaverse services: a multi-agent deep reinforcement learning approach

Avatars, as promising digital representations and" service assistants of users in Metaverses, can enable drivers and" passengers to immerse themselves in 3D virtual services and" spaces of UAV-assisted vehicular Metaverses. However, avatar" tasks include a multitude of human-to-avatar and avatar-toavatar" interactive applications, e.g., augmented reality navigation," which consumes intensive computing resources. It is inefficient" and impractical for vehicles to process avatar tasks locally. Fortunately," migrating avatar tasks to the nearest roadside units (RSU)" or unmanned aerial vehicles (UAV) for execution is a promising" " solution to decrease computation overhead and reduce task processing" latency, while the high mobility of vehicles brings challenges" for vehicles to independently perform avatar migration" decisions depending on current and future vehicle status. To" address these challenges, in this paper, we propose a novel avatar" task migration system based on multi-agent deep reinforcement" learning (MADRL) to execute immersive vehicular avatar tasks" dynamically. Specifically, we first formulate the problem of" avatar task migration from vehicles to RSUs/UAVs as a partially" observable Markov decision process that can be solved by" MADRL algorithms. We then design the multi-agent proximal" policy optimization (MAPPO) approach as the MADRL algorithm" for the avatar task migration problem. To overcome slow" convergence resulting from the curse of dimensionality and nonstationary" issues caused by shared parameters in MAPPO, we" further propose a transformer-based MAPPO approach via" sequential decision-making models for the efficient representation" of relationships among agents. Finally, to motivate terrestrial" or non-terrestrial edge servers (e.g., RSUs or UAVs) to share" computation resources and ensure traceability of the sharing" records, we apply smart contracts and blockchain technologies to" achieve secure sharing management. Numerical results demonstrate" that the proposed approach outperforms the MAPPO" approach by around 2% and effectively reduces approximately" 20% of the latency of avatar task execution in UAV-assisted" vehicular Metaverses.