Reachability analysis for incomplete networks of Markov decision processes

Ralf Wimmer, Ernst Moritz Hahn, Holger Hermanns, Bernd Becker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Assume we have a network of discrete-time Markov decision processes (MDPs) which synchronize via common actions. We investigate how to compute probability measures in case the structure of some of the component MDPs (so-called blackbox MDPs) is not known. We then extend this computation to work on networks of MDPs that share integer data variables of finite domain. We use a protocol which spreads information within a network as a case study to show the feasibility and effectiveness of our approach.
Original languageEnglish
Title of host publication9th IEEE/ACM International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011, Cambridge, UK, 11-13 July, 2011
Pages151-160
Number of pages10
DOIs
Publication statusPublished - 2011
Externally publishedYes

Cite this

Wimmer, R., Hahn, E. M., Hermanns, H., & Becker, B. (2011). Reachability analysis for incomplete networks of Markov decision processes. In 9th IEEE/ACM International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011, Cambridge, UK, 11-13 July, 2011 (pp. 151-160) https://doi.org/10.1109/MEMCOD.2011.5970522