SourceFS: The Fastest Way to Checkout and Build Android and Beyond

Slow Builds and Code Checkouts

Today's connected devices are powered by some of the largest codebases ever developed.

The latest Linux kernel has 65 Million lines of code while the Android AOSP has 120M+, and this is just the foundation. Real-world device codebases are far larger – add the code for hardware support, custom features, services, additional OSs and a smartphone quickly tops 200M lines of code, while an Electric Vehicle exceeds 500M. And these codebases continue to grow throughout a device’s lifetime, with each software update.

Every code checkout pulls millions of files and hundreds of GB, every build runs hundreds of thousands of steps. And because dependency graphs at this scale are imperfect, even a small change can trigger a massive rebuild or, worse, produce an incorrect result. 

The impact: hours of developer time lost every day and $millions wasted on ever increasing CI/CD compute requirements — or more often, configurations that are not tested due to compute constraints.

The Source File System (SourceFS)

Not another build system. SourceFS is a high-performance filesystem virtualization layer that seamlessly drops into your workflows, tools, and CI – delivering unparalleled speedups with zero migration overhead. And here's how it works. 

At its core, SourceFS virtualizes everything, materializes on demand, and does all of this transparently – so neither users nor the rest of the system notice the difference.

‍

‍

SourceFS accelerates code checkouts by over 10x. It does this by creating a virtual file representation of the entire codebase on checkout, and materializing files on-demand, in an instant, if they are needed. This eliminates the need to download hundreds of GB of untouched code, drastically reducing disk space requirements. Given that most changes only address a small fraction of the codebase, most files are never materialized. All this is seamlessly integrated with Git and Repo for a frictionless onboarding experience.

SourceFS accelerates builds by up to 10×. Every build step runs in a lightweight sandbox that records all inputs, outputs, and environment. This covers 99% of the build – not just compilation, but linking, packaging, signing, generating docs, and more. As the build runs, any step that exactly matches a prior record is skipped and the results are automatically reused; any new or invalidated steps – for example from your local changes – are recorded for future use.

Under the hood, SourceFS packs novel algorithms, advanced virtualization, high-performance caching and replay, efficient sandboxing, and a state-of-the-art backend with near-zero overhead. Most of this is written in Rust and powered by decades of kernel and operating system advancements and expertise.

Fast Builds, Efficient Storage and Cost Savings

The results of checking out code and building in a SourceFS environment are truly impressive.

Fast checkouts, even when compared to the most optimised standard way of downloading the code, are over 20x faster. The SourceFS code checkout gives developers a working git tree, and aside from running in a SourceFS environment, everything else is the same workflows developers are already used to. 

In addition to speed, a non-obvious superpower is the fact that now a codebase takes a tiny amount of disk space. This is game changing for device developers as most device makers maintain multiple device codebases, for each product line and sometimes even for each product version.

Fixing widescale bugs, common across multiple codebases, or even just checking out another device codebase to see how something is done is seamless and similar to working on a tiny github repository. 

‍

Fast builds are what truly makes a difference to developer productivity. With SourceFS builds complete 9x faster on a regular developer machine. This sets a new standard as it enables developers to get their sword fighting time back and speeds-up the lengthy feedback loop on CI/CD pipelines.

Even compared to other fast build solutions described below, SourceFS achieves significantly better performance, by being able to replay almost all the build steps - across not just compilers, but also all the other tools used in a device codebase – think packaging, linking, documentation and a lot more. 

‍

‍Cost savings is what makes a difference for organizations. The ability to do more with the same compute budget or reduce the costs where they have grown out of control is what is financially important.

To understand the impact on builds, it is important to size this to the constraints in fast-paced device organizations., where developers can’t wait over 2hrs for a build to finish. In these organizations developers have access to the biggest and most powerful compute nodes available.

While these nodes complete builds in a more reasonable amount of time, significantly lower than what SourceFS achieves on a standard machine, their associated costs are substantially higher. Here compared to SourceFS, at better performance, we note cost savings that can range even beyond 14x.

Alternatives: Why They Fall Short

SourceFS builds on prior efforts. Each approach listed below has its own place, yet in our experience they all fall short when faced with the complexity of a modern device codebase.

Migrating to a new build system (Bazel, Buck2)

SourceFS delivers the performance gains of modern build systems like Bazel or Buck2 – while also accelerating checkouts – all without requiring any migration.

Moving a device codebase to a new build system is a massive undertaking that even well-resourced teams have abandoned midway. Further still, this complexity multiplies when real devices — think electric vehicles — span multiple operating systems (Yocto, Android, QNX), each with its own bespoke build system.

Using a compiler wrapper (REClient, Goma)

An intermediate step short of migrating to a new build system is to add a compiler wrapper that enables caching and replay via a remote-execution protocol. However, wrappers cover only a subset of build actions, so they speed up only part of the build – and don’t help checkouts. They’re also brittle as they rely on parsing command-line flags and correctly inferring inputs/outputs, which can break in unexpected ways.

Next Steps

We are rapidly expanding SourceFS support to more real world device codebases, including support for other operating systems, including Yocto, in parallel with optimising performance even further.

Fast builds and checkouts are the first superpowers in our mission to transform how software for smart devices is developed and maintained – making it simpler, faster, and more cost-effective. Next up: even more performance and broader device-codebase support — stay tuned!