Ph.D. Thesis Colloquium
Please click on the following link to join the Thesis Colloquium:
MS Teams link
ABSTRACT
Serverless Computing is a highly abstract computing service that allows users to design their applications as a workflow of independent stateless functions. The serverless platforms provide event-driven function executions and a powerful pay-as-you-use billing model. The arduous task of function deployment, resource provisioning, management, and system administration is completely undertaken by the serverless service provider, thus providing ease of use to the user. Though serverless platforms show many benefits and are widely adopted by many application domains, they also exhibit certain shortcomings. In a bid to provide clean programming abstractions and ease of use, the existing platforms suffer from significant platform overheads, leading to non-deterministic application execution latency. Shared resources, independent execution of individual workflow components, interaction between various control components, external data exchange, etc. are some of the major contributors to the overheads. In addition, the prevalence of resource heterogeneity in serverless platforms, together with the platform overheads, gives rise to variable performance in serverless function executions. The varying application characteristics further add to the non-determinism of the function executions. In this context, Quality of Service (QoS) can be an important way of conveying application requirements for meeting its execution expectations. As QoS requirements are sensitive to each application domain, to understand its impact, an exploration of how QoS requirements can be handled in serverless platforms is conducted in this work. The study identifies the existing literature for selecting the relevant application domains that are suitable for studying the impact of QoS requirements on resource provisioning, instance sizing, function deployment decisions, and data exchange among functions.
In the first phase, we examine the domain of Image and Video analytics. The applications in this domain are typically deployed in a heterogeneous resources setting akin to the Edge-Cloud continuum. With varying application input size, it is imperative to select resource assignments to functions that meets user-specified QoS requirements. However, in a heterogeneous resource pool comprising Edge and Cloud resources, choosing a resource layer and function resource size that is both cost-efficient and one that meets QoS requirements is a challenging task. In order to address this issue, we present a novel QoS-aware serverless platform that deduces function resource specification in a heterogeneous resource setting. The proposed platform performs function placement across Edge and Cloud layers based on incoming input size and user-specified QoS requirements while prioritizing low-cost function executions. Experimental results based on real-world workloads on a video surveillance application show that the proposed platform brings efficient resource utilization and cost savings at the Cloud by reducing the resource usage by up to 30%.
In the second phase, we examine the domain of Network Function Virtualization. Network Function Virtualization (NFV) and Software Defined Networks (SDNs) allow the network functions to be programmed as software entities that can be deployed on commodity servers in the Cloud, referred to as Virtual Network Functions (VNFs). VNFs are arranged in a workflow to enforce a network policy referred to as Service Function Chain (SFC). Existing solutions examine the SFC deployment on either IaaS or FaaS offerings. IaaS solutions alone lack the dynamism to respond to quick workload changes for network traffic which is similar to streaming data. Existing FaaS solutions exhibit significant platform overheads which are prohibitive to meet the application’s requirements. Further, the variability in the incoming flowlet and payload sizes of applications exacerbates the problem of selecting function resource assignments that both satisfy QoS requirements and one that results in low resource usage. In order to address these shortcomings, we propose a Hybrid Serverless Platform (HSP) encompassing IaaS and FaaS in its function deployment strategy. Specifically, The IaaS components handle the steady state load, whereas the FaaS components activate during the dynamic change associated with scaling to minimize service loss. The proposed HSP controller takes resource provisioning decisions based on user-specified QoS requirements and the application’s flow statistics. The HSP controller’s design exploits data locality in SFC realization, reducing data-transfer times and platform overheads between VNFs. The proposed platform provides up to 35% resource savings as compared to a pure IaaS deployment and up to 55% lower end-to-end SFC execution times as compared to a baseline FaaS implementation with minimal loss of flowlet service.
In the third phase, we examine the domain of Web Inferencing which constitutes an important class of event-driven applications that lend themselves to being composed as a workflow of functions. Typically, web inferencing applications are deployed across a heterogeneous setting comprising Edge and Cloud resources. Though web inferencing workflows have seen an extensive adoption of serverless computing, they suffer from the overheads of the serverless computing platforms, which increases the application’s end-to-end latency. Existing literature addresses the problem of high platform overheads by a technique called ‘Function Fusion’, which combines consecutive serverless functions in a workflow into a single logical group. However, existing solutions are limited by the peak resource sizes of functions that dictate the sizing of fusion groups; they are insensitive to the varying application inputs and QoS requirements of the inferencing applications. In order to address these issues, we propose a novel fusion criteria called ‘RightFusion’ that performs cost-efficient function fusion in serverless platforms that manage a heterogeneous resource pool. The proposed platform uses incoming application characteristics and user-specified QoS requirements to dynamically arrive at function fusion decisions that are both cost-efficient and QoS-abiding. Experimental evaluation of representative inferencing applications shows that RightFusion reduces resource usage by above 35% at Edge and Cloud while meeting the QoS requirements compared to the PeakFusion baseline.
In conclusion, this study demonstrates the value of considering application QoS in serverless computing. The QoS specifications enable the platform to undertake resource management and function placement decisions that meet user expectations. In our studies, QoS requirements from different application domains have translated to different design decisions that address various problems, including resource right-sizing, function fusion, request ordering, etc., resulting in cost and resource savings. Current Serverless platforms take off the burden of resource provisioning and placement for execution from the user. However, they are agnostic to the expectations of applications. Applications, on the other hand, suffer from non-deterministic overheads from these platforms. Given the differences in differing applications, QoS provides a mechanism to express application requirements that the serverless platform can use for decision-making with regard to application execution. The current exploration exposes that QoS is application-specific and requires different methods to deal with it; emphasizing the need for customization in serverless platform design. It also brings to the fore the value of considering the integration of learning approaches in serverless controller’s decision-making. This study considered cost and performance as the primary QoS attributes for each of the domains examined. Other QoS features, like security, privacy, reliability, jurisprudence, etc., are yet to be explored.
