Friday, December 20th 2019

Considering New Operating Systems

Software's history holds the key to an evolutionary step for cloud infrastructure.

App and Data Storage History

Databases were popularized by the client-server boom of the 90s. Enterprises wanted a centralized place where many systems could interact with their data all at the same time.

The database contract: Define a consistent data schema and in turn enterprise applications get a single server for all their data that is accessible to multiple clients at once.

By the 2000s, applications were comprised of rich clients (e.g. big Java apps) and thin clients (e.g. a PHP page). Both types of clients still read data directly from databases.

Rich clients had far more features than thin clients, but their distribution had to be managed so clients could stay in sync with the server. (e.g. a database migration required every computer running that rich client to have the latest version in order to function).

Thin clients could be instantly distributed via the stateless HTTP protocol, but could not provide the majority of features rich clients offered. Thin clients were only capable of static presentation and very simple interaction. To overcome this, thin clients forced business logic to be shifted into rich backends running on central servers.

Apps Today

Practially no applications today use a database’s built-in user system, each instead creating their own user system, business logic, permissions, and sometimes even locking mechanisms to serve data to clients.

These clients no longer directly access data, and instead rely on rich backends to read data from the database on their behalf, which causes the duplication of a large amount of business logic that databases once provided.

Every cloud app today is siloed and all reimplement the same core patterns—create, read, update, and delete—dozens or hundreds of times in order to provide a nonstandard API that only their clients can consume. (We grew very familiar with just how non-standard these APIs are when we began connecting to many external SaaS apps.)

The Bet

Rethink reimagines cloud app architecture from first principles.

Rethink decouples the data layer from the application layer by providing a developer-friendly cloud filesystem. By removing the need for a single, rigid database, it allows software to return to a rich client model with a thin backend. This also cuts out an enormous amount of duplicated business logic present in today’s cloud applications.

On the data layer, the cloud file system can replace a central database entirely. Querying can be provided via a MacOS Spotlight-like indexing functionality and SQLite support. It can read and write any type of data, far exceeding databases’ primarily numerical and textual support, it provides read/write locks (ACID), and even supports real-time collaboration for any file via Websockets.

Today’s application backends keep data consistent by defining table schemas and migrations; Rethink’s filesystem accomplishes the same by allowing rich client applications to specify and maintain file structure, just as desktop apps have for decades.

Not only can a filesystem replace a database, but files have several distinct advantages too. Files are a lot more user-friendly: they provide a skeuomorphic interface to data, they can represent a complete unit of work that is user-portable, and they are interoperable by default when using multi-decade–old file standards. Files represent an abstract interface that allows applications to store any data they need together, while databases depend on external services to store binary files at scale.

This new platform shifts the business logic typically run on servers today back onto client devices. This makes Rethink incredibly scalable because users’ edge devices provide the majority of application compute instead of Rethink’s cloud backend. Because of this, deploying an app on Rethink can be done in one-click and developers don’t need to think about costly hosting or complex network setups.

With the near-universal prevalence of Chromium—an open-source browser engine that powers Google Chrome and Microsoft Edge, as well as the relatively recent release of HTML5, the web is so fully featured that we can make the return to rich clients without any distribution trade offs. These new clients can even directly access desktop hardware and run at near-native speeds with WebAssembly. A WebAssembly core also enables these cloud apps to be written in many more languages than just JavaScript.

Cloud filesystem-powered applications return to the efficient model of direct data access, skipping the APIs and databases typically required by cloud apps and gain further advantages by leveraging the latest web technologies to complete a true cloud OS.