Tackling latency via replication in distributed systems

Zhan Qiu, Juan F. Pérez, Peter G. Harrison

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Consistently high reliability and low latency are twin requirements common to many forms of distributed processing; for example, server farms and mirrored storage access. To address them, we consider replication of requests with canceling - i.e. initiate multiple concurrent replicas of a request and use the first successful result returned, canceling all outstanding replicas. This scheme has been studied recently, but mostly for systems with a single central queue, while server farms exploit distributed resources for scalability and robustness. We develop an approximate stochastic model to determine the response-time distribution in a system with distributed queues, and compare its performance against its centralized counterpart. Validation against simulation indicates that our model is accurate for not only the mean response time but also its percentiles, which are particularly relevant for deadline-driven applications. Further, we show that in the distributed set-up, replication with canceling has the potential to reduce response times, even at relatively high utilization. We also find that it offers response times close to those of the centralized system, especially at medium-to-high request reliability. These findings support the use of replication with canceling as an effective mechanism for both fault- and delay-tolerance.

Original languageEnglish (US)
Title of host publicationICPE 2016 - Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering
PublisherAssociation for Computing Machinery
Pages197-208
Number of pages12
ISBN (Electronic)9781450340809
DOIs
StatePublished - Mar 12 2016
Externally publishedYes
Event7th ACM/SPEC International Conference on Performance Engineering, ICPE 2016 - Delft, Netherlands
Duration: Mar 12 2016Mar 16 2016

Conference

Conference7th ACM/SPEC International Conference on Performance Engineering, ICPE 2016
CountryNetherlands
CityDelft
Period3/12/163/16/16

Fingerprint

Farms
Servers
Stochastic models
Scalability
Processing

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Science Applications
  • Hardware and Architecture

Cite this

Qiu, Z., Pérez, J. F., & Harrison, P. G. (2016). Tackling latency via replication in distributed systems. In ICPE 2016 - Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (pp. 197-208). Association for Computing Machinery. https://doi.org/10.1145/2851553.2851562
Qiu, Zhan ; Pérez, Juan F. ; Harrison, Peter G. / Tackling latency via replication in distributed systems. ICPE 2016 - Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering. Association for Computing Machinery, 2016. pp. 197-208
@inproceedings{3b8093a700604f7a9627f09ab2f070ea,
title = "Tackling latency via replication in distributed systems",
abstract = "Consistently high reliability and low latency are twin requirements common to many forms of distributed processing; for example, server farms and mirrored storage access. To address them, we consider replication of requests with canceling - i.e. initiate multiple concurrent replicas of a request and use the first successful result returned, canceling all outstanding replicas. This scheme has been studied recently, but mostly for systems with a single central queue, while server farms exploit distributed resources for scalability and robustness. We develop an approximate stochastic model to determine the response-time distribution in a system with distributed queues, and compare its performance against its centralized counterpart. Validation against simulation indicates that our model is accurate for not only the mean response time but also its percentiles, which are particularly relevant for deadline-driven applications. Further, we show that in the distributed set-up, replication with canceling has the potential to reduce response times, even at relatively high utilization. We also find that it offers response times close to those of the centralized system, especially at medium-to-high request reliability. These findings support the use of replication with canceling as an effective mechanism for both fault- and delay-tolerance.",
author = "Zhan Qiu and P{\'e}rez, {Juan F.} and Harrison, {Peter G.}",
year = "2016",
month = "3",
day = "12",
doi = "10.1145/2851553.2851562",
language = "English (US)",
pages = "197--208",
booktitle = "ICPE 2016 - Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering",
publisher = "Association for Computing Machinery",
address = "United States",

}

Qiu, Z, Pérez, JF & Harrison, PG 2016, Tackling latency via replication in distributed systems. in ICPE 2016 - Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering. Association for Computing Machinery, pp. 197-208, 7th ACM/SPEC International Conference on Performance Engineering, ICPE 2016, Delft, Netherlands, 3/12/16. https://doi.org/10.1145/2851553.2851562

Tackling latency via replication in distributed systems. / Qiu, Zhan; Pérez, Juan F.; Harrison, Peter G.

ICPE 2016 - Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering. Association for Computing Machinery, 2016. p. 197-208.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Tackling latency via replication in distributed systems

AU - Qiu, Zhan

AU - Pérez, Juan F.

AU - Harrison, Peter G.

PY - 2016/3/12

Y1 - 2016/3/12

N2 - Consistently high reliability and low latency are twin requirements common to many forms of distributed processing; for example, server farms and mirrored storage access. To address them, we consider replication of requests with canceling - i.e. initiate multiple concurrent replicas of a request and use the first successful result returned, canceling all outstanding replicas. This scheme has been studied recently, but mostly for systems with a single central queue, while server farms exploit distributed resources for scalability and robustness. We develop an approximate stochastic model to determine the response-time distribution in a system with distributed queues, and compare its performance against its centralized counterpart. Validation against simulation indicates that our model is accurate for not only the mean response time but also its percentiles, which are particularly relevant for deadline-driven applications. Further, we show that in the distributed set-up, replication with canceling has the potential to reduce response times, even at relatively high utilization. We also find that it offers response times close to those of the centralized system, especially at medium-to-high request reliability. These findings support the use of replication with canceling as an effective mechanism for both fault- and delay-tolerance.

AB - Consistently high reliability and low latency are twin requirements common to many forms of distributed processing; for example, server farms and mirrored storage access. To address them, we consider replication of requests with canceling - i.e. initiate multiple concurrent replicas of a request and use the first successful result returned, canceling all outstanding replicas. This scheme has been studied recently, but mostly for systems with a single central queue, while server farms exploit distributed resources for scalability and robustness. We develop an approximate stochastic model to determine the response-time distribution in a system with distributed queues, and compare its performance against its centralized counterpart. Validation against simulation indicates that our model is accurate for not only the mean response time but also its percentiles, which are particularly relevant for deadline-driven applications. Further, we show that in the distributed set-up, replication with canceling has the potential to reduce response times, even at relatively high utilization. We also find that it offers response times close to those of the centralized system, especially at medium-to-high request reliability. These findings support the use of replication with canceling as an effective mechanism for both fault- and delay-tolerance.

UR - http://www.scopus.com/inward/record.url?scp=85020211058&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020211058&partnerID=8YFLogxK

U2 - 10.1145/2851553.2851562

DO - 10.1145/2851553.2851562

M3 - Conference contribution

AN - SCOPUS:85020211058

SP - 197

EP - 208

BT - ICPE 2016 - Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering

PB - Association for Computing Machinery

ER -

Qiu Z, Pérez JF, Harrison PG. Tackling latency via replication in distributed systems. In ICPE 2016 - Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering. Association for Computing Machinery. 2016. p. 197-208 https://doi.org/10.1145/2851553.2851562