Energy efficient resource controller for Apache Storm

Apache Storm is a distributed processing engine that can reliably process unbounded streams of data for real-time applications. While recent research activities mostly focused on devising a resource allocation and task scheduling algorithm to satisfy high performance or low latency requirements of Storm applications across a distributed and multi-core system, finding a solution that can optimize the energy consumption of running applications remains an important research question to be further explored. In this article, we present a controlling strategy for CPU throttling that continuously optimize the level of consumed energy of a Storm platform by adjusting the voltage and frequency of the CPU cores while running the assigned tasks under latency constraints defined by the end-users. The experimental results running over a Storm cluster with 4 physical nodes (total 24 cores) validates the effectiveness of proposed solution when running multiple compute-intensive operations. In particular, the proposed controller can keep the latency of analytic tasks, in terms of 99th latency percentile, within the quality of service requirement specified by the end-user while reducing the total energy consumption by 18% on average across the entire Storm platform.