Coordination of power and transportation networks with integrated electricity-hydrogen stations: a reinforcement learning approach considering delayed reward

The increasing adoption of electric vehicles (EVs) and fuel cell EVs (FCEVs) has intensified the coupling between urban transportation and energy systems. However, existing approaches lack effective optimization strategies for integrated transportation–energy systems, particularly in managing integrated electricity-hydrogen stations (IEHSs) to maximize revenue while serving various vehicle types, such as EVs and FCEVs. The present work proposes a dynamic optimization strategy for IEHSs that enhances the operation of IEHSs, power distribution networks, and transportation networks by guiding vehicle charging and fueling decisions. The sensitivity of vehicular traffic to system-level decisions is ensured by establishing a mixed dynamic user equilibrium model incorporating EVs, FCEVs, and conventional-fuel vehicles. An IEHS operation framework that integrates discount coupon allocation and hydrogen production/storage strategies is designed with traffic flow feedback and distribution locational marginal pricing. The inherent delayed reward problem in transportation, where the impacts of pricing decisions on station revenue and traffic congestion manifest over extended time horizons, is addressed by applying a multi-agent deep reinforcement learning approach, featuring a dual-critic architecture based on long short-term memory networks under the sequence markov decision process framework, to redistribute delayed rewards. The results of numerical computations involving real-world traffic and power networks demonstrate that the proposed approach improves IEHS profitability, balances charging demand, and alleviates traffic congestion compared with static and semi-dynamic traffic approaches.