GraalDoss: Direct object snapshotting and sharing for cloud-native applications

Abstract

Modern cloud-computing providers operate on a pay-as-you-use billing model, with computing power and memory being the most important and expensive resources. Due to resource costs, cloud-native applications should start fast while minimizing startup time and memory footprint over multiple application instances. However, modern workloads consist of large amounts of data, often requiring initialization which introduces repeated CPU work across application instances. Current cloud-native solutions that pre-initialize application code and data operate at application-build time to enable sharing during execution. However, these solutions do not consider data that becomes available or can only be initialized during application execution. We present Doss, a direct object snapshotting and sharing mechanism for cloud-native applications. Doss snapshots the state of the object graph directly from the executing language-runtime heap. This allows Doss to achieve constant deserialization overhead with memory mappings. Doss shares warmed-up data snapshots across compatible language-runtime instances, reducing the memory overhead of the system, and avoiding cold starts. We implement GraalDoss in Java as part of GraalVM. GraalDoss maintains a constant data-cache memory overhead across multiple application instances, eliminating costly data initialization. In microservice applications, GraalDoss reduces the total memory footprint by 44% for 8 microservice instances and improves first-response times by 51%. In natural language processing applications, GraalDoss improves total execution times by several orders of magnitude.