Joint model deployment and task execution for collaborative edge inference

Intelligent edge inference necessitates latency-critical deep neural network execution. However, serving various inference tasks requires a large pool of candidate DNN models and a higher resource orchestration complexity, which is impractical for resource-limited edge servers. This paper proposes a collaborative edge inference framework, which enables both terminal-edge model partitioning and edge-to-edge (E2E) task migration to accelerate DNN inference. A two-timescale joint optimization problem is formulated. At the large timescale, DNN model deployment is updated in each period based on model popularity, while at the small timescale, task execution is determined in each time slot. The key objective is to minimize the long-term system cost of the inference delay and model deployment overhead, subject to resource limitations and service requirements. This highly nonconvex problem is solved by integrating both the machine learning and the convex optimization theory. For the period-based model deployment, a multi-agent proximal policy optimization (MAPPO)-based algorithm is developed. For each slot, Karush-Kuhn-Tucker (KKT)-based closed-form resource allocation is further embedded into a mixed-integer quadratic programming (MIQP) reformulation, and the resulting inference delays are accumulated to train the MAPPO at the large timescale. Simulation results demonstrate that the proposed method achieves stable convergence and significant gains over baselines under varying demand dynamics, resource configurations, and network scales.