A Flexible Architecture for Structural Reliability Auditing to the Clouds

Abstract:

Today’s cloud computing systems pervasively rely on redundancy techniques to enhance reliability and availability. In complex multi-layered hardware/software stacks, however, seemingly independent components used redundantly might share deep, hidden dependencies. These common dependencies may potentially result in unexpected correlated failures, thus undermining redundancy efforts.

Heading off correlated failures is extremely challenging in cloud-scale systems for the following three reasons. First, infrastructure components and their dependencies underlying cloud-scale redundant systems are typically very complex. Second, it is non-trivial to efficiently and accurately determine the common dependencies potentially resulting in correlated failures within cloud-scale system environment. Third, application-level cloud services (e.g., iCloud) renting multiple cloud providers for redundancy cannot identify dependencies shared by these rented cloud providers, since no cloud provider is willing to disclose such sensitive information.

This thesis proposes Independence-as-a-Service (or INDaaS), a practical and flexible architecture to audit the independence of redundant systems proactively, thus avoiding potential correlated failures fundamentally. INDaaS first utilizes pluggable dependency acquisition modules to automatically collect the structural dependency information (including network, hardware, and software level dependencies) from a variety of sources underlying the audited cloud services. With this information in hand, INDaaS then evaluates the independence of redundant systems of interest using pluggable auditing modules, offering various performance, precision, and data secrecy trade-offs. For redundant systems across multiple cloud providers unwilling to share their full structural information with others, INDaaS can leverage privacy-preserving set intersection cardinality protocols to quantify the independence of the redundant systems without leaking any sensitive information of the cloud providers. We evaluate the practicality of INDaaS with three case studies that audit realistic network, hardware, and software dependency structures, and evaluate the effectiveness and efficiency of INDaaS through large-scale datasets.

Ph.D. Thesis: PDF