Resource Management for Graphics Processing Units in Cloud Computing

The persistent development of Cloud computing attracts individuals and organisations to change their IT strategies. According to this development and the incremental demand for the adoption of Cloud computing, Cloud computing providers (CSPs) continuously update their infrastructure to fit this demand, which has led to the introduction of accelerator units to support Cloud computing users’ applications requirements. Graphics Processing Units (GPUs) are well-known types of accelerators since they provide high-performance capabilities with low computational power consumption compared with traditional CPUs. The CSPs considerations in terms of adopting accelerators have resulted in an increase in hardware heterogeneity in the Cloud infrastructure and therefore the resource management techniques have challenged. The complexity of managing a heterogeneous Cloud infrastructure while maintaining the Quality of Service (QoS) is an important issue that needs addressing alongside minimising the infrastructure operational costs. A Cloud infrastructure consumes tremendous amounts of energy with a substantial impact on such operational cost. Thus, new energy-aware management techniques need to be developed to efficiently manage heterogeneous Cloud infrastructures.
This thesis introduces a systematic architecture that consists of several novel components to manage heterogeneous GPU resources made available through Virtual Machines (VMs) in a Cloud environment, considering performance and energy consumption as key factors. To fulfil the goals of the introduced architecture, different research contributions are accomplished. First, a power consumption model is introduced to identify the power consumption for heterogenous GPUs located in a Cloud-based system. Hybrid inputs are used to develop the power model consisting of hardware performance counters and resources utilisation (GPU and memory). Second, an agnostic energy model that aims to directly estimate energy consumption to be applicable for different types of GPUs is then developed. Third, this agnostic energy model is enhanced to estimate energy by reducing the cost of the data collection procedure. This enhancement of the energy model is created by developing a novel end-to-end energy framework aimed to predict the required resources for each GPU application, and then pass the mentioned estimated resources to the heterogeneous GPU energy modeller. Finally, this thesis develops scheduling policies to reduce energy consumption and then to balance the trade-off between energy and performance whilst meeting the Quality of Service (QoS) in a Cloud computing environment. To achieve this goal, the proposed scheduling policies make use of the end-to-end GPU energy framework to obtain the application’s energy consumption and execution time proactively.
The prediction models and the energy framework are evaluated on two different GPUs by a real Cloud computing testbed and show that they are capable to effectively predict power and energy consumption. Moreover, the evaluation of the proposed scheduling policies reveals that they can reduce energy consumption and also support the trade-off between energy saving and performance whilst maintaining the QoS requirements, therefore balancing energy consumption and applications’ performance.