RFD 70 Capacity, Allocation, and Utilization

In-use: resources that are currently in use, as in a process or application is consuming that CPU, Disk, or Memory.

Provisioned: resources that have been provisioned by a user, these can be either in-use or idle, depending on the application load.

Reserved: resources that have been reserved, these are being held by a quota, they can be provisioned, or yet to be provisioned.

Capacity: The limit we actually have.

Utilization: What is actually being used versus what is provisioned (typically a percentage: in-use / provisioned OR provisioned / reserved. Which ever equation we are using to calucate this, we need to always tell the user what we used to get it.

Threshold: the customer designated threshold for when they want to be alerted, this can be ⇐ capacity. The default will be 75% capacity.

Overprovisioning: Allowing operators to provision more of a resource than its capacity. ("Selling more seats than there are on the plane.") Compare with bursting. For example, if a server has 64 CPUs, a system might allow an operator to provision 72 CPUs, which would be 13% overprovisioned. Note that this can apply to CPUs, memory, storage, IOPS, or any other resource. This is usually done to improve utilization (improving economic efficiency) when operators expect that not all resources that are allocated will be used. But if all allocated resources are used, quality of service degrades significantly.

Bursting: allowing applications to use more resources than they’ve been allocated when these resources are available. ("Giving empty seats away to existing customers", maybe?) Compare with overprovisioning. For example, an instance might be allocated 4 CPU-seconds per second. If the system has extra CPU time available, it can allow the instance to use 8 CPU-seconds in a second.

Storage

CPU

Memory

Network: This is more appropriate under health, like other cloud providers we should do the "right thing" here and limit the network bandwidth with the size of the instance

Transparently show data. This does not mean show the customer the raw data, that would not be actionable. But frame the data in a way that is understandable. Make sure the data is clear, accurate, and actionable. The user can also export any data via the API, that way they can integrate easily into any existing systems they have.

Provide data so users can appropriately make decisions for themselves.

Make the "right thing" the "easy thing".

Have action buttons for doing “the right thing” based on the data. (Examples below.) This could include “resizing an over-sized disk” based on actual usage patterns or “resizing an over-sized instance” based on usage patterns.

Always inform all users working on a project of the changes made to resources in the project and by whom with a customizable message sent at the time an action took place. (Examples below.) For example, if a rack admin(s) decides to resize an over-sized disk for a project, the users in the project should get an email saying something like: “I resized your disk X in project Y because over the past Z days you were only utilizing 15% of it. This new size should last you a projected 200 days before needing to be resized at your current write rate.” This should be part of a longer form audit log including the timestamp, user, action, and any other relevant data.

Do not over-provision or allow for over-provisioning. Do not allow a lie to cover up what should otherwise be a conversation between teams. Our goal (and value to the customer) is to help them achieve great utilization of their infrastructure through data, not over-subscription.

In each scenario, show the data users mostly likely want to see with some most likely actions. (Examples below.)

Allow rack(s) admins to set thresholds for when they will be notified they are near capacity.

When in doubt allow for sending multiple users or individual users messages. This should be a one-click button to open an email to the specific users or user. We can even pre-populate it, but still allow for customization, if we have enough data to know what is going on. Something like “Your project, X, is only utilizing 10% of the disk space you provisioned. Can you give me the reasoning behind your allocation or permission to resize to X GB so we can free up disk space for other projects? Thank you!” This way rack(s) admins who want to communicate first before resizing something can do so, OR they can resize it for them and send an email after.

Empower developers with the data to do the right thing so rack admin(s) may not need to intervene. (Examples below). Notify developers when their disks or instances are under-utilized with a one-click button to resize if appropriate.

Quotas are a way to prevent individual users and teams from over-allocating all the resources for themselves.

Focus not only on over-sized instances and disks but also under-sized instances and disks. (Examples below.)

Dashboard. The standard metrics shown on a rack(s) admins dashboard should always include visuals for the limited resources as far as what is in-use, what has been provisioned, reserved, and a comparison to their threshold and capacity. (Examples below).

Views. They should then be able to drill down into tables by user, team, and project, to see where the numbers are coming from (examples below). These tables should automatically be sorted by percent utilized but allow for changing the sort based on clicking a table header or whatever the standard table visual UX is today.

Utilization. If a project’s utilization is 45% or less (we should likely in the future make this number customizable but for now let’s just call it this) there should be handy buttons for doing the “the right thing” whether it is notifying a team to take action, resizing a disk for a project, or resizing an instance. In the event something is resized by someone other than a project member, for instance a rack(s) admin, the project users should be notified. The rack(s) admin should be able to customize the message that gets sent to the project members so everyone is on the same page. Something like “Your instance X in project Y has been resized since you were only using Z CPU and A GB Mem for the past X days. Thank you!”

Inventory. Based on the remaining capacity for a fleet or rack, we can do the math to show rack admin(s) how many instances of what size are remaining.

Forecasting. Where available it would be nice to forecast based on the past what future utilization/allocation will look like. (Examples below.) Seeing as this would take an analytical model we should do what we can easily for V1 and continue work on this thereafter.