Algorithm 972: JMarkov: An integrated framework for Markov chain modeling

Juan F. Pérez, Daniel F. Silva, Julio C. Góez, Germán Riaño, Andrés Sarmiento, Andrés Sarmiento-Romero, Raha Akhavan-Tabatabaei

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Markov chains (MC) are a powerful tool for modeling complex stochastic systems. Whereas a number of tools exist for solving different types ofMCmodels, the first step inMCmodeling is to define themodel parameters. This step is, however, error prone and far from trivial when modeling complex systems. In this article, we introduce jMarkov, a framework for MC modeling that provides the user with the ability to define MC models from the basic rules underlying the system dynamics. From these rules, jMarkov automatically obtains the MC parameters and solves the model to determine steady-state and transient performance measures. The jMarkov framework is composed of four modules: (i) the main module supports MC models with a finite state space; (ii) the jQBD module enables the modeling of Quasi-Birth-and-Death processes, a class of MCs with infinite state space; (iii) the jMDP module offers the capabilities to determine optimal decision rules based on Markov Decision Processes; and (iv) the jPhase module supports the manipulation and inclusion of phase-type variables to representmore general behaviors than that of the standard exponential distribution. In addition, jMarkov is highly extensible, allowing the users to introduce new modeling abstractions and solvers.

Original languageEnglish (US)
Article number29
JournalACM Transactions on Mathematical Software
Volume43
Issue number3
DOIs
StatePublished - Jan 1 2017

Fingerprint

Markov processes
Markov chain
Module
Modeling
Markov Chain Model
Complex Systems
State Space
Quasi-birth-and-death Process
Stochastic systems
Markov Decision Process
Decision Rules
Exponential distribution
Stochastic Systems
System Dynamics
Performance Measures
Large scale systems
Manipulation
Dynamical systems
Trivial
Inclusion

All Science Journal Classification (ASJC) codes

  • Software
  • Applied Mathematics

Cite this

Pérez, J. F., Silva, D. F., Góez, J. C., Riaño, G., Sarmiento, A., Sarmiento-Romero, A., & Akhavan-Tabatabaei, R. (2017). Algorithm 972: JMarkov: An integrated framework for Markov chain modeling. ACM Transactions on Mathematical Software, 43(3), [29]. https://doi.org/10.1145/3009968
Pérez, Juan F. ; Silva, Daniel F. ; Góez, Julio C. ; Riaño, Germán ; Sarmiento, Andrés ; Sarmiento-Romero, Andrés ; Akhavan-Tabatabaei, Raha. / Algorithm 972 : JMarkov: An integrated framework for Markov chain modeling. In: ACM Transactions on Mathematical Software. 2017 ; Vol. 43, No. 3.
@article{10af022b91234e2caecebbafd42bc33c,
title = "Algorithm 972: JMarkov: An integrated framework for Markov chain modeling",
abstract = "Markov chains (MC) are a powerful tool for modeling complex stochastic systems. Whereas a number of tools exist for solving different types ofMCmodels, the first step inMCmodeling is to define themodel parameters. This step is, however, error prone and far from trivial when modeling complex systems. In this article, we introduce jMarkov, a framework for MC modeling that provides the user with the ability to define MC models from the basic rules underlying the system dynamics. From these rules, jMarkov automatically obtains the MC parameters and solves the model to determine steady-state and transient performance measures. The jMarkov framework is composed of four modules: (i) the main module supports MC models with a finite state space; (ii) the jQBD module enables the modeling of Quasi-Birth-and-Death processes, a class of MCs with infinite state space; (iii) the jMDP module offers the capabilities to determine optimal decision rules based on Markov Decision Processes; and (iv) the jPhase module supports the manipulation and inclusion of phase-type variables to representmore general behaviors than that of the standard exponential distribution. In addition, jMarkov is highly extensible, allowing the users to introduce new modeling abstractions and solvers.",
author = "P{\'e}rez, {Juan F.} and Silva, {Daniel F.} and G{\'o}ez, {Julio C.} and Germ{\'a}n Ria{\~n}o and Andr{\'e}s Sarmiento and Andr{\'e}s Sarmiento-Romero and Raha Akhavan-Tabatabaei",
year = "2017",
month = "1",
day = "1",
doi = "10.1145/3009968",
language = "English (US)",
volume = "43",
journal = "ACM Transactions on Mathematical Software",
issn = "0098-3500",
publisher = "Association for Computing Machinery (ACM)",
number = "3",

}

Pérez, JF, Silva, DF, Góez, JC, Riaño, G, Sarmiento, A, Sarmiento-Romero, A & Akhavan-Tabatabaei, R 2017, 'Algorithm 972: JMarkov: An integrated framework for Markov chain modeling', ACM Transactions on Mathematical Software, vol. 43, no. 3, 29. https://doi.org/10.1145/3009968

Algorithm 972 : JMarkov: An integrated framework for Markov chain modeling. / Pérez, Juan F.; Silva, Daniel F.; Góez, Julio C.; Riaño, Germán; Sarmiento, Andrés; Sarmiento-Romero, Andrés; Akhavan-Tabatabaei, Raha.

In: ACM Transactions on Mathematical Software, Vol. 43, No. 3, 29, 01.01.2017.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Algorithm 972

T2 - JMarkov: An integrated framework for Markov chain modeling

AU - Pérez, Juan F.

AU - Silva, Daniel F.

AU - Góez, Julio C.

AU - Riaño, Germán

AU - Sarmiento, Andrés

AU - Sarmiento-Romero, Andrés

AU - Akhavan-Tabatabaei, Raha

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Markov chains (MC) are a powerful tool for modeling complex stochastic systems. Whereas a number of tools exist for solving different types ofMCmodels, the first step inMCmodeling is to define themodel parameters. This step is, however, error prone and far from trivial when modeling complex systems. In this article, we introduce jMarkov, a framework for MC modeling that provides the user with the ability to define MC models from the basic rules underlying the system dynamics. From these rules, jMarkov automatically obtains the MC parameters and solves the model to determine steady-state and transient performance measures. The jMarkov framework is composed of four modules: (i) the main module supports MC models with a finite state space; (ii) the jQBD module enables the modeling of Quasi-Birth-and-Death processes, a class of MCs with infinite state space; (iii) the jMDP module offers the capabilities to determine optimal decision rules based on Markov Decision Processes; and (iv) the jPhase module supports the manipulation and inclusion of phase-type variables to representmore general behaviors than that of the standard exponential distribution. In addition, jMarkov is highly extensible, allowing the users to introduce new modeling abstractions and solvers.

AB - Markov chains (MC) are a powerful tool for modeling complex stochastic systems. Whereas a number of tools exist for solving different types ofMCmodels, the first step inMCmodeling is to define themodel parameters. This step is, however, error prone and far from trivial when modeling complex systems. In this article, we introduce jMarkov, a framework for MC modeling that provides the user with the ability to define MC models from the basic rules underlying the system dynamics. From these rules, jMarkov automatically obtains the MC parameters and solves the model to determine steady-state and transient performance measures. The jMarkov framework is composed of four modules: (i) the main module supports MC models with a finite state space; (ii) the jQBD module enables the modeling of Quasi-Birth-and-Death processes, a class of MCs with infinite state space; (iii) the jMDP module offers the capabilities to determine optimal decision rules based on Markov Decision Processes; and (iv) the jPhase module supports the manipulation and inclusion of phase-type variables to representmore general behaviors than that of the standard exponential distribution. In addition, jMarkov is highly extensible, allowing the users to introduce new modeling abstractions and solvers.

UR - http://www.scopus.com/inward/record.url?scp=85011339690&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85011339690&partnerID=8YFLogxK

U2 - 10.1145/3009968

DO - 10.1145/3009968

M3 - Article

AN - SCOPUS:85011339690

VL - 43

JO - ACM Transactions on Mathematical Software

JF - ACM Transactions on Mathematical Software

SN - 0098-3500

IS - 3

M1 - 29

ER -