Projects
The ProcessCache system provides a generic facility for automatically memoizing the work of a broad class of multi-process Linux programs. ProcessCache caches results and transparently determines when cached results can be used and when re-execution is necessary. ProcessCache generalizes previous work on forward build systems, to go beyond software builds to other multi-process programs like shell scripts and bioinformatics workflows. ProcessCache supports unmodified Linux binaries, using the ptrace mechanism to trace system calls and determine program inputs. The system utilizes a custom asynchronous runtime (developed in Rust) to obviate latent ptrace overhead. Our experiments show that ProcessCache can automatically provide incremental computation to existing programs, accelerating workloads from 1.06x to 65x. I functioned as technical lead for this project, managing the project and maintaining ownership of it. The open-source code for the project can be found here.
DetTrace is a reproducible container abstraction for Linux implemented entirely in userspace. All computation that occurs inside a DetTrace container is a pure function of the initial file system state of the container. Reproducible containers can be used for a variety of purposes, including replication for fault-tolerance, reproducible software builds, and reproducible data analytics. We use DetTrace to achieve, in an automatic fashion, reproducibility for 12,130 Debian package builds, containing over 800 million lines of code, as well as bioinformatics and machine learning workflows. We show that, while software in each of these domains is initially irreproducible, DetTrace brings reproducibility without requiring any hardware, OS, or application changes. My largest contribution to the system was extending the container's sequential scheduler to run in parallel in system-call-free regions of code while still maintaining a deterministic scheduling, bringing the overhead down to just 2% for some of our compute-bound benchmarks. The open-source code can be found here, and our ASPLOS paper can be found here.
During the summer of 2020, I interned with the VMware Research Group. I worked somewhat outside my normal realm, but still with the Rust programming language, on Distributed Differential Datalog (D3log). D3log is a typed Datalog Rust engine built upon a timely dataflow computation model extended to work across machines. My largest contribution was adapting the distributed API to allow for incremental, on-the-fly reconfiguration of the nodes within the network, improving fault-tolerance. The open-source code can be found here.