RFD 223 Web Console Architecture

The console is a static JS bundle that runs in the browser and calls to Nexus directly. This has numerous advantages. There is no JavaScript running on the server. There is nothing but static files to deploy, no extra servers to monitor.

This option requires some sort of access token to be stored in the browser so the console app can talk to Nexus. The standard approach is to store a relatively short-lived session ID in a cookie and have the server maintain a table linking session IDs to user IDs and some metadata to help with expiration. Using the HttpOnly attribute on the cookie prevents malicious scripts from stealing it, but because cookies are automatically sent along with requests to our domain, we also need to use standard CSRF mitigations. See [rfd169] for a detailed discussion.

A console server would manage sessions and proxy authenticated API requests from the browser client to Nexus. We would use the session cookie to look up the session, which would point to an API token, and then we’d forward the request to Nexus with the token attached.

A console server written in Rust would be limited to the following functionality:

A console server written in JS could also provide some of the features detailed below.

Rather than giving the console server its own key-value store for sessions, we would rely on the CockroachDB instance which is already set up for the control plane in order to take advantage of replication.

Rather than talking directly to CockroachDB for session storage, the console could use client credentials ([oauth-2.0-spec] §4.4) to hit a dedicated Nexus sessions endpoint with a session ID and get back a token. This is not very different from the Cockroach solution — simply replace the arrow from console server to CockroachDB with an arrow to Nexus — and has some advantages:

This is only included for completeness. If the console server had its own database, it would have to be replicated across racks, a problem we are already doing a bunch of work to solve with the control plane CockroachDB instance.

The security issues with Node and npm have prompted the development of alternative JS runtimes like Deno, which is an written in Rust and wraps V8. Deno would probably be a lot safer than Node for running a long-running server. Deno does not use npm, and making apps written for Node work on Deno takes some work.

However, we can probably get all the benefits of server-side JS without a long-running server. Next.js, SvelteKit, and Remix are designed not to rely on Node specifically, and can also run in so-called "serverless" environments like Cloudflare Workers, which execute JavaScript using the V8 engine directly.

The Deno team maintains high quality Rust bindings for V8 that are core to their product. We may be able to use that to execute JS on the server without bringing in Node or Deno. There is prior art for using the V8 Rust bindings to render React components server-side. This is a bit out there, but could make sense for us as an incremental optimization on top of a solid client-side console. It could also make sense in the context of adding a general serverless worker feature to the rack.

This is the weakest advantage, so it’s only first to get it out of the way. Sometimes you want to run the exact same code on client and server. The most common example is input validation: you have to validate input on the server no matter what because you can’t trust clients, but ideally you can also run the same validation client-side so the user doesn’t need a server round-trip to know their input is invalid. In order to share this code, you need to be running JS on the server.

There is a basic set of problems client-side web apps tend to have around data fetching. They don’t fundamentally break an app, but they can degrade user experience a little or a lot depending on how many components on the page are fetching data. There are client-side mitigations, but they tend to be partial and/or complicated, so the state of the art here is various forms of server integration. Here I characterize the loading spinner problems and several server-side mitigations: server-side rendering, server-integrated frameworks, and Server Components. These solutions are all React-specific because that’s what we’re using and what I’m most familiar with, and all require Node because they rely on running React code on the server. However, this problem is common to all client-side web apps, and once you get the shape of the problem in React world you can easily find similar approaches in other JS frameworks as well as the Rust + Wasm frameworks. For example, SvelteKit has file-based routing and async navigation like Remix and Next.js.

Over time single-page app devs have come to understand that letting components define their own data requirements by fetching data at render time is the best way to architect apps for ease of reasoning. (Alternate approaches where data requirements are spelled out in a central place and components ask for bits and pieces have led to a lot of pain.) But doing data fetching at the component level comes with downsides:

SSR means doing the work required to render the page server-side (including data fetching, in sophisticated implementations) so that initial pageload can be a complete page with no loading spinners. The result of the initial render is sent to the browser as HTML, but then React spins up on the client and everything from there is business as usual. Time to paint is faster because API requests originating from the frontend server have fewer hops (maybe the server is faster at rendering too, but that’s not the main factor). SSRed HTML can also be cached (server-side or client-side), but for data-driven views of live systems like ours, this is of limited utility because cached HTML gets outdated quickly.

Note that SSR is a technical term with a narrow meaning. The other two solutions below also involve something that could be called server-side rendering, but typically SSR refers only to the initial page load. This is one reason it’s not a slam dunk for improving perceived performance: it only affects the initial pageload and not subsequent client-side navigation (i.e., most navigation), which is still going to use async fetches and loading states. The reason it only affects initial pageload is because server integration with the rest of the app’s lifecycle requires more substantial architecture changes, while SSR of the initial render can usually be turned on with just a few lines of code.

Another reason it’s not a slam dunk is that perceived performance with SSR may actually be worse (depending on request latencies) due to user psychology around partial loading and spinners. For example, a client-rendered site where a page shell with a loading state shows up in 200ms and content pops in at 800ms might feel the same as or faster than an SSR page that takes 500ms to do both steps simultaneously because, in the latter case, the user sees nothing for the entire 500ms whereas in the former there is at least some response at 200ms. SSR has to be much faster than CSR in order to have a clear perceived performance benefit.

In December 2020, the React team previewed a new feature of React currently in development called Server Components [server-components]. They’ve been working on it since then, but I’m not aware of any indication of when it will be ready. With SSR, the entire app is rendered server-side initially and then after that then entire app is rendered client-side. Server Components allow you to tell parts of the app to always render on the server — the source for those components is never sent to the client, only the result of rendering. Future renders also have to happen on the server. The idea is that highly interactive parts remain on the client, and parts that render infrequently (or for which renders are triggered by HTTP requests already) go on the server.

If this sounds like classic server-rendered HTML, it should: the goal of Server Components is explicitly to preserve the capability to make native-feeling web apps afforded by SPA frameworks while making them work more like classic web apps when prudent. There is some consensus across languages that this is likely to be a valuable approach going forward: Server Components are quite a bit like Phoenix LiveView over in Elixir/Erlang world and Laravel Livewire in PHP land. It’s all rather complex and only tangentially related to this RFD, so don’t worry too much about it, but I’m including details below for anyone interested.

Server components address the loading spinner problems by moving all API requests server-side where they will presumably have much shorter round trips. The basic structure is the same — the requests can still happen in series rather than in parallel — they just make everything faster and render all at once at the end. Here is how Server Components address each part of the problem:

My overall take is that there’s a lot of complexity here and a lot of risk for apps to overdo it. Deciding what goes on the server and what goes on the client can add a lot of conceptual overhead. I expect that the best use of server components will be made by library developers who expose a constrained version of it to app developers, rather than app developers using them directly ad hoc. Regardless, taking advantage of this feature (when it comes out) whether directly or through mediating libraries would require a Node server.

Remix is just one framework, but I’m going to let it stand in for the best of what’s possible right now. It uses file-based routing like Next.js, but it’s simpler and more grounded in fundamental web APIs like Fetch [remix-next-1] [remix-next-2]. I’m presenting these as an alternative to Server Components, but they will be able to integrate and take advantage of Server Components without app devs having to change their code.

React by itself is sort of the View and the View-Controller layer of the old MVC model, but it is silent on two essential app functions: routing and data fetching. Countless libraries for these are available. What we’re using in the console right now is React Router (by far the most popular option) for the former and React Query for the latter.

Remix is a library by the React Router devs that solves routing and data fetching together in an integrated way. In a Remix app, you define the data requirements for each route in a loader function that runs server-side and sends its result down to the client. If a route needs to make multiple requests and combine them, only the aggregated result of that is sent to the client. "Route" somewhat counterintuitively here means the set of layouts corresponding to the set of route segments. For example, in the route /team/david there would be a file for the /team part (e.g., a page wrapper) and a file for the /david part (the page content) and each can specify their own data requirements. The data requirements for /team/david are simply the respective data requirements for /team and /data, and these two loaders run in parallel on the server.

A second key feature is that when the user navigates to a route, the client does not show the target page until after the loaders have completed. In plain React, the pattern is: switch immediately to the new view, kick off fetches (with loading spinners), then render when the data comes back. In Remix it’s: kick off all loaders (show a global loading indicator if you want, but by default do not) and only render the new page after the data comes back.

There is more to Remix, but route-based data requirements and async navigation are sufficient to address the loading spinner problems: