Stephanie Wang, UC Berkeley: “A Distributed Memory Layer for General-Purpose Execution”

Position: PhD Candidate and Software Engineer

Current Institution: UC Berkeley / Anyscale

Abstract: A Distributed Memory Layer for General-Purpose Execution

Distributed applications have become the norm. The cloud has provided ready and increasingly affordable access to compute resources which in principle allows anyone to build a distributed application. However the design and implementation of distributed applications is still limited to a handful of experts. This is because distributed computing poses a myriad of problems that are simpler or nonexistent on a single process or machine such as scheduling communication and fault tolerance. My work has focused on the problem of building a general-purpose distributed system that can act as a common substrate for current and future applications. By building one system that can support many different applications I hope to ease the burden on developers reduce duplicated effort between distributed systems and promote interoperability between currently disparate applications. A key challenge in this effort is ensuring fault tolerance and reliability without sacrificing performance or flexibility. Thus my work has focused on building a fast and reliable distributed memory layer for general-purpose execution. As part of this work I have designed novel systems and techniques for transparent data recovery and distributed memory management. This work has been realized in the open-source Ray project a general-purpose execution system that is used today for a wide variety of applications from big data processing to reinforcement learning.

Bio:

Stephanie is a PhD candidate at UC Berkeley advised by Ion Stoica. She holds a BS in computer science and math and an MEng in computer systems from MIT where she was advised by Frans Kaashoek and Nickolai Zeldovich. Her research focuses on the problem of designing and building a general-purpose distributed system. Towards this end she is also a lead committer for the open-source Ray project and a software engineer at the startup Anyscale.