RFD 3
Oxide Hiring Process
RFD
3
Updated

At Oxide, we are taking a big swing: rethinking the definition of the server-side computer, combining the wisdom of hyperscaler systems with the energy of the open data center — and delivering it as a broadly consumable product. Our most deeply held technical belief is in the integration of hardware and software: we believe that the best and most reliable infrastructure comes when hardware and software are designed to work with one another. When we seek those to join us — regardless of role — we are looking for those who share this worldview: that our systems are best when we design them together.

But how does one assess candidates for such positions? This is an age-old question without a formulaic answer: designing, building, selling, and supporting computing systems is itself too varied to admit a single archetype.

Most obviously, this kind of endeavor is intellectually challenging; it demands minds that not only enjoy the thrill of solving puzzles, but can also stay afloat in a sea of numbing abstraction. This raw capacity, however, is insufficient; there are many more nuanced skills that successful Oxide employees must possess. For example, engineering is an almost paradoxical juxtaposition of collaboration and isolation: successful engineers are able to work well with (and understand the needs of!) others, but are also able to focus intensely on their own. This contrast extends to the conveyance of ideas, where they must be able to express their own ideas well enough to persuade others, but also be able to understand and be persuaded by the ideas of others — and be able to implement all of these on their own. They must be able to build castles of imagination, and yet still understand the constraints of a grimy reality: they must be arrogant enough to see the world as it isn’t, but humble enough to accept the world as it is. Each of these is a balance, and for each, long-practicing engineers will cite colleagues who have been ineffective because they have erred too greatly on one side or another.

This balance — which can be thought of as the balance between work on one’s own and work with others — exists in every role we might add to the company. Determining this in a candidate defies assessment that is pat or rote; we must assess prospective employees without the luxury of firm criteria. This document is an attempt to pull together accumulated best practices over our collective careers; while it shouldn’t be inferred to be overly prescriptive, where it is rigid, there is often a painful lesson behind it.

In terms of evaluation mechanism: using in-person interviewing alone can be highly unreliable and can select predominantly for surface aspects of a candidate’s personality. While we advocate (and indeed, insist upon) interviews, they should come relatively late in the process; as much assessment as possible should be done by allowing the candidate to show themselves as they truly work: on their own, via their creations.

Traits to evaluate

How does one select for something so nuanced as balance, especially when the road ahead is unknown? We must look at a wide-variety of traits, presented here in the order in which they are traditionally assessed:

  • Aptitude

  • Education

  • Motivation

  • Values

  • Integrity

Aptitude

As the ordering implies, there is a temptation in traditional hiring to focus on aptitude exclusively: to use an interview exclusively to assess a candidate’s pure pulling power. While this might seem to be a reasonable course, it in fact leads down the primrose path to pop quizzes about algorithms seen primarily in software engineering interview questions. These assessments of aptitude are misplaced: work is not, in fact, a spelling bee, and one’s ability to perform during an arbitrary oral exam may or may not correlate to one’s ability to actually design, build, sell, or support production systems. We believe that aptitude is better assessed where it is forced to be exercised: based on the work that one does on one’s own. As such, candidates should be asked to provide three samples of their works: a work sample, a writing sample, and an analysis sample.

Work sample

The ultimate measure of us all is in the work we do, and as such a work sample can be the truest way to assess a candidate’s ability. While the exact nature of the work sample will vary based on domain, all candidates should be encouraged to be as detailed as possible. For engineers, this should be source code, schematics, or designs; for operations, it might be a detailed discussion of their work on a program or product; for customer-facing roles like sales, solutions or support, it might be the concrete details of a customer engagement. Regardless of the kind of work sample being asked for, multiple samples should be requested (e.g., 2-3), and concrete details should be requested. The prompt asking for work samples should generally be customized for the domain, but not made so specific that it only has applicability within a single role. (That is, we would generally have a single work sample prompt for each of engineering, operations, sales, solutions, etc.)

If, when adding a new role, there is a question about what would constitute a reasonable work sample, trusted practitioners outside of Oxide should be consulted as to what they seek in prospective peers.

Writing sample

As a company, we believe that good writing is found in the best practitioners, regardless of role: the act of writing — like so much else that we do — requires not only the ability to create wholly new material, but also the ability to reflect, correct, revise, and polish.

To assess a candidate’s writing ability, they should be asked to provide a writing sample. As with the work sample, the specifics in the prompt will vary based on domain: for an engineer, it might be a block comment or design document; for someone in sales, it might be a detailed product description or a customer use case; for operations, it might be a meeting summary or an internal request to an engineering team. As with the work sample, if there is a question about what constitutes a writing sample for a new domain, practitioners outside of Oxide should be consulted. And as with the work sample, multiple samples should be requested.

Analysis sample

A significant challenge at Oxide is dealing not with a system of our own creation, but rather one that has been presented to us. How do we reason about what we see in front of us? When we are trying to understand a system rather than create it, we must flip our disposition: instead of a creator of our own heaven and earth, we must become a scientist, attempting to reason about a foreign world. In having candidates only focus on their ability to build, analytical skills are often left unexplored. And while this can be explored conversationally (e.g., asking for "debugging war stories" is a classic — and often effective — interview question for engineers), an oral description of recalled analysis doesn’t necessarily allow the true depths of a candidate’s analytical ability to be plumbed.

To assess analytical ability, candidates should be asked to provide an analysis sample: a written analysis from the candidate. As with the work and writing sample, the specifics of the prompt will vary by domain: for engineers, this should be the analysis that the candidate performed of a system behaving pathologically; for sales, this might be the research the candidate performed of a prospect and their fit with the product before reaching out to them; for operations, this might be the examination of an existing process with an eye towards improvement; for product management, this might be the exploration of a new system or of a new aspect of the system. Regardless of domain, the sample should contain enough concrete detail to assess analytical ability, with the candidate asked to recall details to the best of their ability.

Other samples

Depending on the domain, other samples may be optionally requested.

Presentation sample

Especially as technologists become more senior, oral presentation of work becomes an important vehicle for communication — and increasingly, conferences of all sizes are posting all videos online. As a result, many candidates can reasonably supply a sample of a technical presentation. People can be very self-conscious about their presentations, so applicants should be assured to not worry about any imperfections — and to not be concerned if it’s brief or informal.

The aim here is severalfold. First, this lets everyone get the same information at once: it is not unreasonable that the contents of a talk that a candidate would point to would be similar to a conversation that they would have otherwise had several times over the day as they are asked about their experience; this minimizes that repetition.

Second, it shows how well the candidate teaches. Assuming that the candidate is presenting on a domain that isn’t intimately known by every member of the audience, the candidate will be required to instruct at some level. Teaching requires both technical mastery and empathy — and a pathological inability to teach may point to deeper problems in a candidate.

Third, if the presentation features questions, it can show how well the candidate fields questions about their work. It should go without saying that the questions themselves shouldn’t be trying to find flaws with the work, but should be entirely in earnest; seeing how a candidate answers such questions can be very revealing about character.

Engagement sample

For customer-facing roles, it can be helpful for a sample of a complete engagement with a customer. Depending on the role, that might span from earliest activity through product sale to long past deployment. The intent behind this sample for customer-facing roles is to get to the love of customers that we hold dear: this sample allows a candidate to show their own empathy for the customer, and their commitment to the customer’s long-term success.

Education

We are all born uneducated — and our own development is a result of the informal education of experience and curiosity, as well as a better structured and more formal education. To assess a candidate’s education, both the formal and informal aspects of education should be considered.

Formal education

Formal education is easier to assess by its very formality: a candidate’s education is relatively easily evaluated if they had the good fortune of discovering their interest and aptitude at a young age, had the opportunity to pursue and complete their formal education in their chosen domain, and had the further good luck of attending an institution that one knows and has confidence in.

But one should not be bigoted by familiarity: there are many terrific practitioners who attended little-known schools or who took otherwise unconventional paths. The completion of a formal education is much more important than the institution: the strongest candidate from a little-known school or otherwise unconventional path is almost assuredly stronger than the weakest candidate from a well-known school.

In other cases, it’s even more nuanced: there have been many later-in-life converts to the beauty and joy of computers, and such candidates should emphatically not be excluded merely because they discovered their calling later than others. For those that concentrated in entirely non-technical disciplines, further probing will likely be required, with greater emphasis on their technical artifacts.

The most important aspect of one’s formal education may not be its substance so much as its completion. Like engineering itself, there are many aspects of completing a formal education that aren’t necessarily fun: classes that must be taken to meet requirements; professors that must be endured rather than enjoyed; subject matter that resists quick understanding or appeal. In this regard, completion of a formal education represents the completion of a significant task. Inversely, the failure to complete one’s formal education may constitute an area of concern. There are, of course, plausible life reasons to abandon one’s education prematurely (especially in an era when higher education is so expensive), but there are also many paths and opportunities to resume and complete it. The failure to complete formal education may be entirely reasonable — but it may also indicate deeper problems, and should be understood.

Informal education

Learning is a life-long endeavor, and much of one’s education will be informal in nature. Assessing this informal education is less clear, especially because (by its nature) there is little formally to show for it — but candidates should have a track record of being able to learn on their own, even when this self-education is arduous. One way to probe this may be with a simple question: what is an example of something that you learned that was a struggle for you?

Motivation

Motivation is often not assessed in the interview process, which is unfortunate because it dictates so much of what we do and why. For Oxide in particular, it is critically important to find those who feel a deep and intrinsic motivation to solve the problems of a complicated computing system. This is essential exactly because these technical problems are so challenging: they push back on us all, and it takes great internal drive to rise to one’s feet again and again.

But how to truly determine motivation? For many companies, this might be tricky to assess, but at Oxide we have the luxury of a clear mission — and the good fortune that those proficient in our domain tend to be very drawn both to that mission and to others similarly motivated. So simply asking candidates the simple question of why they want to work for Oxide can be revealing: we would expect the answer to be phrased in terms of the mission that we have set for ourselves, the problems we have in tackling it, and/or the team that we have already attracted.

One cautionary note: because our mission is so clear, it can attract some who find it superficially appealing — but in fact will struggle when the problems become spiritually crushing. Motivation should be assessed as much by looking at a candidate’s career (educational, professional or otherwise): their career arc should be clearly driving them to solve the problems that we solve.

Values

Values are often not evaluated formally at all in the hiring process — yet shared values are essential for long-term success. Some questions should be asked to suss out a candidate’s values, e.g.:

  • What work have you done that you are particularly proud of and why?

  • When have you been happiest in your professional career and why?

  • When have you been unhappiest in your professional career and why?

But simply inquiring as to someone’s values is insufficient, because at Oxide, our values are clearly elucidated:

Candor

Humor

Teamwork

Courage

Optimism

Thriftiness

Curiosity

Resilience

Transparency

Diversity

Responsibility

Urgency

Empathy

Rigor

Versatility

We must get to the degree that candidates share our values. This can be done by asking for written answers to values-oriented questions. For example:

  • For one of these values, describe a time in which the value has been reflected in your work.

  • For one of these values, describe a time in which the values were violated by your organization or work — and how it was resolved.

  • For a pair of these values, describe a time in which they came into tension for you, and how you resolved it.

Integrity

In an ideal world, integrity would not be something we would need to assess in a candidate: we could trust that everyone is honest and trustworthy. This view, unfortunately, is naïve with respect to how malicious bad actors can be; for any organization — but especially for one that is biased towards trust and transparency — it is essential that candidates be of high integrity: an employee who operates outside of the bounds of integrity can do nearly unbounded damage to an organization that assumes positive intent.

There is no easy or single way to assess integrity for people with whom one hasn’t endured difficult times. By far the most accurate way of assessing integrity in a candidate is for them to already be in the circle of one’s trust: for them to have worked deeply with (and be trusted by) someone that is themselves deeply trusted. But even in these cases where the candidate is trusted, some basic verification is prudent.

Mechanics of evaluation

While there may be preexisting relationships with candidates, and we may have conversations with candidates to give them context about Oxide or describe our hiring process, every candidate will ultimately submit written material by which they will be evaluated.

Candidate-submitted material

The candidate should submit the following:

  • Work sample (at least one, no more than three)

  • Writing sample (no more than one per category)

  • Analysis sample (at least one, no more than three)

  • Other samples, per domain (at least one, no more than three)

  • Written answers to eight questions:

    1. What work have you found most challenging in your career and why?

    2. What work have you done that you are particularly proud of and why?

    3. When have you been happiest in your professional career and why?

    4. When have you been unhappiest in your professional career and why?

    5. For one of Oxide’s values, describe an example of how it was reflected in a particular body of your work.

    6. For one of Oxide’s values, describe an example of how it was violated in your organization or work.

    7. For a pair of Oxide’s values, describe a time in which the two values came into tension for you or your work, and how you resolved it.

    8. Why do you want to work for Oxide?

Lest this seem arduous, note that much of this material is likely to be preexisting (e.g., samples of prior work). As for the balance, it has been our experience that candidates themselves find answering these self-reflective questions to be helpful to their own job search process, entirely independently of any specific opportunity.

Evaluating candidate-submitted material

Once material is submitted, it should be evaluated. The process by which materials are evaluated is substantive enough to deserve separate treatment; the details of this process can be found in [rfd147], but in short: the evaluation of materials comprises the bulk of our candidate evaluation process. Anyone interested in Oxide should be instructed that we take the materials very seriously — and that we tend to have far more candidates than available positions. (It is not altogether uncommon for candidates that are well-regarded by Oxide employees to submit milquetoast materials; candidates should not assume that familiarity with the team will be a substitute for materials evaluation.)

Interview preparation

Where candidates are promising, they should be advanced to conversations with the team. Before interviews, candidates should be given access to RFDs, as discussed in [rfd35]. Getting complete access to the RFDs gives candidates the opportunity to understand Oxide much more concretely, with the understanding that after seeing the gritty details of our endeavor, a candidate may reasonably choose to withdraw their candidacy. There is no shame in this: we want new additions to Oxide to be well-informed about the true state of the company — and it is helpful to all if a candidate has decided themselves that they are not a fit.

Scheduling

Interviews should be scheduled with members of the Oxide team. We ask candidates to provide at least nine hours of time, generally over three three-hour blocks consisting of three slots of one hour each. To schedule interviews, Oxide employees will book an interview slot, opting into conversations with the candidate as they see fit. Some candidates may wish to provide more such blocks, and in rare cases, we may ask for more blocks. We try to balance respect for the candidate’s time with our desire to be thorough, and we have generally found nine hours of interviews to be sufficient without being overly burdensome.

Pre-meeting

Everyone on the interview schedule should read the candidate-submitted material, and a pre-meeting should then be held to discuss approach: based on the written material, what are the things that the team wishes to better understand? And who will do what?

Materials

Because every employee of Oxide submitted materials when they joined the company, we are afforded a unique opportunity: in addition to interviewers reading a candidate’s materials, candidates are offered the interviewers' materials in advance of their conversation. (That is, the materials that the interviewers provided when they themselves came to Oxide.) These are provided in confidence: the candidate should understand that these are private, and that they are a way for them to get to know the people who are interviewing them. We have found that this twist — which even for our idiosyncratic company felt iconoclastic when it was suggested by an early Oxide employee — has had an extraordinary effect on the tenor of conversations: the conversation much more closely approximates a conversation between future peers.

Interviews

Interviews shouldn’t necessarily take one form; interviewers should feel free to take a variety of styles and approaches — but should generally refrain from "gotcha" questions and/or questions that may conflate surface aspects of intellect with deeper qualities (e.g., Microsoft’s infamous "why are manhole covers round?"). Not all interviews will cover all traits, and several traits (e.g., aptitude, education) are multifaceted and nuanced: approaching these from different perspectives can be helpful. (Mixing interview styles over the course of the day can also be helpful for the candidate, who doesn’t feel trapped in a single kind of interview.) Should an interviewer learn something during their interview that they feel merits more exploration by another interviewer later in the schedule, they shouldn’t hesitate to communicate their perspective to that later interviewer: the need to fully explore a candidate’s fit trumps any desire to not have one interview influence another.

After the interview

After the interview, the candidate should be discussed by those who interviewed them. The objective isn’t necessarily to get to consensus first (though that too, ultimately), but rather to areas of concern. In this regard, the post-interview conversation must be handled carefully: the interview is deliberately constructed to allow broad contact with the candidate, and it is possible than someone relatively junior or otherwise inexperienced will see something that others will miss. The meeting should be constructed to assure that this important data isn’t suppressed; bad hires can happen when reservations aren’t shared out of fear of disappointing a larger group!

In general, those interviewing a candidate should be asked to enter the meeting with one of three positions: Hire, Do not hire, or Insufficient information.

However the meeting is mechanically run, it is important that if anyone is a Do not hire, these concerns be treated very seriously: it is saying that the aptitude, education, motivation, values and/or integrity of the candidate are in serious doubt or are otherwise unacceptable. Those who believe Do not hire should be asked for the dimensions that most substantiate their position. Especially where these reservations are around values or integrity, a single Do not hire should raise serious doubts about a candidate: the risks of bad hires around values or integrity are far too great to ignore someone’s judgment in this regard!

Ideally, however, no one has the position of Do not hire, and through a combination of screening and candidate self-selection, everyone believes Hire and the discussion can be brief, positive and forward-looking!

If, as is perhaps most likely, there is some mix of Hire and Insufficient information, the discussion should focus on the information that is missing about the candidate. If other interviewers cannot fill in the information about the candidate (and if it can’t be answered by the corpus of material provided by the candidate), the group should together brainstorm about how to ascertain it. Should a follow-up conversation be scheduled? Should the candidate be asked to provide some missing information? Should some aspect of the candidate’s background be explored? The collective decision should not move to Hire as long as there remain unanswered questions preventing everyone from reaching the same decision.

Reference checking

During the process — especially for those hires who are not already well-known by an employee — references may wish to be checked. Reference checking should be viewed as an opportunity to get more details on a candidate that aren’t otherwise captured in their materials or their interview. Importantly, it should never be viewed as a substitute for those things, and it should be understood that there will be conditions in which reference checks are difficult. (And in particular, candidates who plan to leave their current employer may not be able to reasonably offer their current employer as a reference.)

In terms of the mechanics of conducting a reference check, there is not necessarily one way to do it — but in the course of a conversation, one should seek to understand the conditions under which a candidate struggled as well as when they thrived. By putting a reference check relatively later in the process (after materials, work review, interview, etc.), it can (hopefully) be made more effective: by the time one is checking references, the candidate should be known pretty well — and there should be a pretty crisp idea about open questions. Similarly, if something in the reference check conversation doesn’t correspond with what is known, that should be more readily apparent.

Post-evaluation

Timely rejection

Whenever the decision has been made to not move forward with a candidate — either because they are not a fit at all or not a fit now — a timely rejection should be sent to the candidate.

In general, the time and thought put into the rejection should reflect the candidate’s depth and effort in our process: candidates that didn’t bother with the materials or clearly didn’t take them seriously should be given a terse rejection; those who clearly put effort into it but are as clearly not a fit should be given a more thoughtful response; and those who are known and valued or those whom we interview but are deemed not a fit due to time or needs should be given a personalized, honest, clear response. For these most personalized responses, an earnest offer for a follow-up phone conversation should be included: this commitment to the candidate makes clear that despite them not being a fit for us now, we value them as an individual, or a friend, or even a potential future co-worker.

Assessing the assessment process

It is tautologically challenging to evaluate one’s process for assessing engineers: one lacks data on the candidates that one doesn’t hire, and therefore can’t know which candidates should have been extended offers of employment but weren’t. As such, hiring processes can induce a kind of ultimate survivorship bias in that it is only those who have survived (or instituted) the process who are present to assess it — which can lead to deafening echo chambers of smug certitude. One potential way to assess the assessment process: ask candidates for their perspective on it. Candidates are in a position to be evaluating many different hiring processes concurrently, and likely have the best perspective on the relative merits of different ways of assessing engineers.

Of course, there is peril here too: while many organizations would likely be very interested in a candidate who is bold enough to offer constructive criticism on the process being used to assess them while it is being used to assess them, the candidates themselves might not realize that — and may instead offer bland bromides for fear of offending a potential employer. Still, it has been our experience that a thoughtful process will encourage a candidate’s candor — and we have found that the processes described here have been strengthened by listening carefully to the feedback of candidates. More reflection on the history and implementation of this process can be found in [oxf-s3e31].

References