The Billing API builds on the API concepts in [rfd4] and allows operators to modify and track the cost of Oxide resources that are consumed by end users. Purchasers of Oxide racks are providing compute services to their teams internally and need a way to track who is using what, with pricing as well. When it comes to who is using what, please refer to [rfd45] on metrics. This RFD focuses on how our customers will charge their internal customers for resources.
Operators will want an easy way to see what each organization, individual projects, and users are "spending" on their resources. Every resource in the Oxide API has a price that can be modified by users with the IAM permission for modifying the billing. IAM permissions are covered in [rfd43].
The Billing API allows our customers to have a chargeback model for their internal customers (the organizations and developers). This has the following benefits:
Increased awareness of how much it really costs to do projects
Increase resource flexibility — it’s much easier to add more resources to meet demand if you are billing for their costs and know exactly where the demand is coming from
In the user interface, we should make it easy for folks with the permission to view billing at a fleet level to see exactly where the highest consumption of resources is coming from. They should be able to view billing on an organizational level and project level, as well as being able to modify (if they have permission) the cost of individual resources across the entire fleet.
Individual users can view their own spending for their projects if they have the IAM permission to do so, by default when a user creates a project they have the permission to view how much the project is spending. They can then grant this IAM permission to other users with access to the project.
Billing accounts, where a bill goes for an organization or project, are covered in [rfd44].
Endpoints
The endpoints for the Billing API are as follows:
GET /system/billing - This is named this way since some metrics will also be
under / system as per <<rfd45>>
This endpoint returns all the current costs for resources
under the API
GET /system/billing/{resource_type} - This endpoint returns the individual cost
for a specific type of resource. For
example /vpc gets the cost of a vpc.
PUT /system/billing/{resource_type} - This endpoint updates the cost for the
resource type. The type of resource,
machine-type if it is a VM, etc, is
included in the body for the request.
GET /organizations/{org_name}/billing - Returns the billing summary for the
organization.
GET /projects/{project_name}/billing - Returns the billing summary for the
project.As you can see above the DELETE method is not implemented for the billing API.
If you would like to set the cost of a resource type to 0 you can do so but
you cannot delete the resource’s billing entirely.
Users can choose whether the resource is billed per second, per minute, or per day when they configure the cost of a resource.
Updates
When the cost of a resource is changed, organizations and projects will use that new cost for the current billing cycle as well as future billing cycles. There is no way, currently, to change what was previously billed. We should keep in mind that we should not build this API in such a way that prevents us from ever allowing people to change what was previously billed in the future.
For v1 the functionality will work this way but in the future we should keep in mind allowing the customer to change when the new pricing takes effect.
Per region billing
We will want to allow customers to be able to set prices to be different in different regions. The API should be built in such a way that is possible even though it may not be necessary for the initial set of racks delivered.
Instance Service Classes
As defined in [rfd4], an instance can be in the "standard" service class or "spot" service class. The default is "standard". All instances are required to have a particular service class. These have different billing settings.
Invoicing
As per [rfd44], billing accounts are set up for organizations. Each organization has a default billing account and organization admins can change the billing account for projects in their organization to something other than the default billing account. Invoices will be sent once a month and the bill for resources will be reset to zero at the end of each month.
Invoices will be sent to the billing account for the project. As per [rfd55], we will have configurable alerts and notifications for billing depending on the user’s permissions. If a user has view access to billing for projects or organizations, they can configure how they would like to be notified for the bills.
Past Invoices
Users can view past invoices in the user interface. We will save all invoices for easily retrieving them later.
Connection to third party tools
For customers with existing systems for tracking billing for their customers, we will provide API endpoints for retrieving billing data at their convenience. We should make sure to expose all the data they need to integrate into their existing system. This includes:
Itemized lists of what resource was charged at what price
Filtering based off organization or project
The daily rate as well as the total cost
The type of resource and the project/organization details of where it came from
As well as everything from the sample invoice below.
If a lot of customers use the same third party tool for integration we should help make the integration ourselves by building it into the product.
Crediting
Our customers will have their own internal SLAs to hold for their customers. In the event one of these SLAs is broken they will need to credit their customers for the resources consumed during that time.
We should make it easy for our customers to do this through the UI. Customers should be able to set a time for resources for an organization or project where they should not be billed. The time should have a start and end date and time. They can add as many of these time segments as they need. These are known as credits. They should each have a name and a description so when later it shows up in an invoice or though the API customers know what they were credited for and the reason.
When a user creates a credit for something like an instance in a specific region, all the dependencies for that resource are also credited. So for example, if an instance is going to be credited for not meeting a specific SLA, the disks and network resources will also be credited back to the user.
Unused quota
In order to disabuse our customer’s internal customers of the notion of free, their internal customers can view the unused quota and the cost spent on it in their dashboard as a widget.
Sample invoice
Here is how a hypothetical invoice with a credit will look:
| Date | Region | Description | Resource | Project | Usage Quantity | Rate | Cost |
|---|---|---|---|---|---|---|---|
10/01/2020 12AM - 10/31/2020 12PM | Mars | Usage of 4 CPU, 18 GB Mem Instances for Project Adventure | Instance: 4 CPU, 16 GB Memory | Adventure | 15 | $0.10 per minute | $69,750.00 |
10/01/2020 12AM - 10/31/2020 12PM | Mars | Usage of 500GB Virtual Disks for Project Adventure | Disk: 500GB | Adventure | 15 | $0.05 per minute | $33,750.00 |
10/15/2020 11PM - 10/16/2020 4:30AM | Mars | Credit for downgraded instances in Region Mars | Instance: 4 CPU, 16 GB Memory | Adventure | 15 | $0.10 per minute | -$515.62 |
10/15/2020 11PM - 10/16/2020 4:30AM | Mars | Credit for downgraded instances in Region Mars | Disk: 500GB | Adventure | 15 | $0.05 per minute | -$257.81 |
User Flow
When an organization is setup they must setup a billing account for that organization. This is covered in [rfd44]. That billing account (if it is the only one) then becomes the default billing account. If they have more than one, the user can decide which billing account is the default billing account.
When a developer creates a project it gets billed automatically to the default billing account for the org. If the org has more than one billing account, the org admin can change which billing account is used for individual projects but by default projects get billed to the default billing account.
By default, email notifications will be set up for the users with organization billing permissions. The IAM roles and permissions are defined in [rfd43]. At the end of every month these users get notified by email with their invoice.
Users (developers and operators) can create and manage alerts for billing. Users with IAM permissions for a project can set alerts up to let members of the project know if they go over a certain budget for that project for a month. The same budget alerts can be put on organizations and alert admins of that organization when they are close to hitting their budget and when they do hit their budget. The alert should allow configuring a threshold for when the notifications start sending.
External References
[rfd4] Oxide Computer Co. RFD 4: User Facing API. 2019.
[rfd43] Oxide Computer Co. RFD 43: Identity and Access Management. 2020.
[rfd44] Oxide Computer Co. RFD 44: Hierarchy and API for User Management and Organizations. 2020.
[rfd45] Oxide Computer Co. RFD 44: User System-level API. 2020.
[rfd52] Oxide Computer Co. RFD 52: Quota Policies. 2020.
[rfd55] Oxide Computer Co. RFD 55: User Alerts. 2020.