Systems Colloquium
Thursday, February 18, 2010
10:30 a.m., AKW 200

Sign up to meet with speaker.

Speaker: Harsha V. Madhyastha, UCSD

Title: Taming the Scale and Costs of (Really) Large Distributed Systems

Abstract: Over the last decade, the penetration of broadband Internet access and the commoditization of server hardware have dramatically increased. These trends have resulted in planetary-scale distributed applications that span millions of end-hosts and data centers that encompass tens of thousands of servers. Such a large scale makes it hard to deal with one of basic challenges in building any system: ensuring good performance while minimizing cost. In this talk, I will present solutions that address this challenge for applications deployed in two different settings: across the wide-area Internet and within data centers.

First, I will present iPlane, an information plane designed to serve as the source of path information for all applications on the Internet. iPlane continually measures the Internet from several hundred geographically distributed vantage points to maintain an up-to-date map of the Internet's structure. iPlane annotates this map with ISP routing policies that it infers and link properties that it measures such as latency and loss rate. Unlike previous attempts at measurement of the Internet at scale, iPlane's focus is on using its measurements along tens of millions of paths to estimate path performance on the remaining tens of billions of unmeasured paths in the Internet. Using the data it gathers, iPlane can accurately predict the properties of the path between arbitrary end-hosts in the Internet thus eliminating the need for measurement by any application.

Second, I will talk about BICMIC, a model that automates the process of determining the cluster configuration best suited to any particular data center application. BICMIC makes two primary contributions. First, it identifies the properties of any cluster that need to be modeled to accurately estimate the application performance that the cluster can yield. BICMIC is an acronym for these properties---(network) bandwidth, IO bus bandwidth, compute, memory, IOPS, and (storage) capacity. Second, by studying several popular applications ranging from MapReduce jobs to backup-to-the-cloud to webservices such as photo sharing and product search, we determine the granularity at which applications need to be modeled: as a set of input-compute-output units. These units capture the compute and IO requirements of applications as well as the interaction between application components. Our model combines these two sources of information to identify how various cluster configuration decisions should be combined to make a given deployment cost-effective. Examples of configuration decisions include under-utilization of storage devices, caching of data in SSDs or DRAM, use of low-power CPUs, and separation of storage and compute into separate server farms.

Bio: Harsha V. Madhyastha is a postdoctoral scholar at the University of California San Diego. He previously received his Ph.D. and M.S. degrees from the University of Washington and his B.Tech. degree from the Indian Institute of Technology Madras, all in Computer Science and Engineering. He has been a recipient of the Best Paper Award at the ACM SIGCOMM Internet Measurement Conference. His research interests span all aspects of distributed and networked systems.