Fast analysis and prediction in large scale virtual machines resource utilisation

Abdullahi Abubakar, Sakil Barbhuiya, Peter Kilpatrick, Vien Ngo, Dimitrios S. Nikolopoulos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

103 Downloads (Pure)


Most Cloud providers running Virtual Machines (VMs) have a constant goal of preventing downtime, increas- ing performance and power management among others. The most effective way to achieve these goals is to be proactive by predicting the behaviours of the VMs. Analysing VMs is important, as it can help cloud providers gain insights to understand the needs of their customers, predict their demands, and optimise the use of resources. To manage the resources in the cloud efficiently, and to ensure the performance of cloud ser- vices, it is crucial to predict the behaviour of VMs accurately. This will also help the cloud provider improve VM placement, scheduling, consolidation, power management, etc. In this paper, we propose a framework for fast analysis and prediction in large scale VM CPU utilisation. We use a novel approach both in terms of the algorithms employed for prediction and in terms of the tools used to run these algorithms with a large dataset to deliver a solid VM CPU utilisation predictor. We processed over two million VMs from Microsoft Azure VM traces and filter out the VMs with complete one month of data which amount to 28,858VMs. The filtered VMs were subsequently used for prediction. Our Statistical analysis reveals that 94% of these VMs are predictable. Furthermore, we investigate the patterns and behaviours of those VMs and realised that most VMs have one or several spikes of which the majority are not seasonal. For all the 28,858VMs analysed and forecasted, we accurately predicted 17,523 (61%) VMs based on their CPU. We use Apache Spark for parallel and distributed processing to achieve fast processing. In terms of fast processing (execution time), on average, each VM is analysed and predicted within three seconds.

Original languageEnglish
Title of host publicationProceedings of the 10th International Conference on Cloud Computing and Services Science, CLOSER 2020
ISBN (Print)9781713840459
Publication statusPublished - 02 Jan 2021
Event10th International Conference on Cloud Computing and Services Science - virtual, online
Duration: 07 May 202009 May 2020


Conference10th International Conference on Cloud Computing and Services Science
Abbreviated titleCLOSER 2020
Cityvirtual, online


Dive into the research topics of 'Fast analysis and prediction in large scale virtual machines resource utilisation'. Together they form a unique fingerprint.

Cite this