Container-based Cloud Virtual Machine Benchmarking

Blesson Varghese, Lawan Thamsuhang Subba, Long Thai, Adam Barker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

27 Citations (Scopus)
800 Downloads (Pure)


With the availability of a wide range of cloud Virtual Machines (VMs) it is difficult to determine which VMs can maximise the performance of an application. Benchmarking is commonly used to this end for capturing the performance of VMs. Most cloud benchmarking techniques are typically heavyweight - time consuming processes which have to benchmark the entire VM in order to obtain accurate benchmark data. Such benchmarks cannot be used in real-time on the cloud and incur extra costs even before an application is deployed.

In this paper, we present lightweight cloud benchmarking techniques that execute quickly and can be used in near real-time on the cloud. The exploration of lightweight benchmarking techniques are facilitated by the development of DocLite - Docker Container-based Lightweight Benchmarking. DocLite is built on the Docker container technology which allows a user-defined portion (such as memory size and the number of CPU cores) of the VM to be benchmarked. DocLite operates in two modes, in the first mode, containers are used to benchmark a small portion of the VM to generate performance ranks. In the second mode, historic benchmark data is used along with the first mode as a hybrid to generate VM ranks. The generated ranks are evaluated against three scientific high-performance computing applications. The proposed techniques are up to 91 times faster than a heavyweight technique which benchmarks the entire VM. It is observed that the first mode can generate ranks with over 90% and 86% accuracy for sequential and parallel execution of an application. The hybrid mode improves the correlation slightly but the first mode is sufficient for benchmarking cloud VMs.
Original languageEnglish
Title of host publicationProceedings of the IEEE International Conference on Cloud Engineering (IC2E), Germany, 2016. Acceptance Rate: 23%
PublisherInstitute of Electrical and Electronics Engineers Inc.
Publication statusPublished - 02 Jun 2016
EventIEEE International Conference on Cloud Engineering (IC2E), 2016 - Berlin, Germany
Duration: 04 Apr 201608 Apr 2016


ConferenceIEEE International Conference on Cloud Engineering (IC2E), 2016

Bibliographical note

Acceptance Rate (17/73): 23%


Dive into the research topics of 'Container-based Cloud Virtual Machine Benchmarking'. Together they form a unique fingerprint.

Cite this