Custom Billing Adapter
Connect a billing platform vBilling has no built-in adapter for: stream signed CloudEvents to it with the webhook adapter, push usage vBilling cannot measure through the ingest API, or write a billing adapter in Go.
| Approach | Code to write | Use it for |
|---|---|---|
| Webhook adapter | None in vBilling, an HTTPS receiver on your side | A data platform, a Kafka bridge or a homegrown rating engine |
| Ingest API | An HTTP client in your service | Usage vBilling cannot see: inference tokens, Slurm accounting, storage |
| Go adapter | A Go package in your fork of vBilling | A billing platform with its own API for customers, meters and usage |
Webhook adapter
The webhook adapter POSTs CloudEvents 1.0 batches in structured mode to one endpoint, up to 500 events per request. Keep the signing secret in a Secret under the key secret, and list webhook next to any other adapter (for example '{metronome,webhook}'):
kubectl -n vbilling-system create secret generic vbilling-webhook \
--from-file=secret=./webhook-secret
helm upgrade --install vbilling deploy/helm/vbilling -n vbilling-system \
--set adapters='{webhook}' \
--set webhook.url=https://billing.example.com/vbilling \
--set webhook.existingSecret=vbilling-webhook \
--set region=us-east-1 --set clusterName=us-east-1a| Helm value | Environment variable | Effect |
|---|---|---|
webhook.url | WEBHOOK_URL | Endpoint that receives the batches. Required when webhook is in adapters. |
webhook.existingSecret (key secret) | WEBHOOK_SECRET | Signs every request with HMAC-SHA256. Without it, requests are not signed. |
webhook.headers | WEBHOOK_HEADERS (k=v,k=v) | Extra request headers. The chart sets them as a plain environment value, not from a Secret. |
region, clusterName | REGION, CLUSTER_NAME | Form the CloudEvents source: vbilling/<region>/<clusterName>. |
Event types
| type | Sent | data |
|---|---|---|
com.vcluster.vbilling. | For every event committed to the ledger, collector and ingested alike: every metric, including informational (vcluster_gpu_utilization, vcluster_gpu_downtime_hours) and custom ones | One usage event |
com.vcluster.vbilling. | Before the tenant's first usage event, again when its details change, and after a restart | The tenant: id, display_name, email, currency, region, project, class, plan, clusters, metadata, provider_ids, created_at |
com.vcluster.vbilling. | Once the tenant has had no tenant clusters for OFFBOARD_GRACE (default 1h) | The tenant |
com.vcluster.vbilling. | Each time vBilling starts, before any usage | Every metric definition: code, name, unit, description, billable, group_keys, custom |
A usage event's CloudEvent id is its usage event ID (vb1_...); the others are tenant-<tenant>-<unix time>, offboard-<tenant>-<unix time> and catalog-<unix time>. Each envelope also carries specversion 1.0, source, subject (the tenant ID; the catalog has none), time (a usage event's window_end), datacontenttype application/json and, on usage events, dataschema. A usage batch, formatted and trimmed to one private node's GPU-hours for one minute:
[{
"specversion": "1.0",
"id": "vb1_5d0f8e2c9a7b4e1f3c6a8d2b0e9f7a41",
"source": "vbilling/us-east-1/us-east-1a",
"type": "com.vcluster.vbilling.usage.v1",
"subject": "acme-ai",
"time": "2026-10-05T10:01:00Z",
"datacontenttype": "application/json",
"dataschema": "https://github.com/vClusterLabs-Experiments/vbilling/blob/main/docs/schema/usage-event.v1.json",
"data": {
"schema_version": "vbilling.usage/v1",
"id": "vb1_5d0f8e2c9a7b4e1f3c6a8d2b0e9f7a41",
"tenant": "acme-ai", "metric": "vcluster_gpu_hours",
"quantity": 0.133333333, "unit": "gpu-hours",
"window_start": "2026-10-05T10:00:00Z", "window_end": "2026-10-05T10:01:00Z",
"region": "us-east-1", "sku": "NVIDIA-H100-80GB-HBM3", "resource_id": "gpu-node-1",
"dimensions": {"billing_mode": "private_node", "capacity_type": "reserved",
"gpu_profile": "full", "gpu_type": "NVIDIA-H100-80GB-HBM3",
"node": "gpu-node-1", "tenant_cluster": "vcluster-acme-ai-train"},
"properties": {"gpu_devices": 8, "physical_gpus": 8},
"source": "collector", "recorded_at": "2026-10-05T10:01:04Z"
}
}]Headers and signature
POST /vbilling HTTP/1.1
Content-Type: application/cloudevents-batch+json
User-Agent: vbilling/0.2
X-VBilling-Signature: t=1791194464,v1=<hex HMAC-SHA256>The signature uses the t=...,v1=... scheme of Stripe's Stripe-Signature: an HMAC-SHA256, keyed with your secret, over the timestamp, a period and the raw request body. Verify it on the raw bytes before parsing JSON and reject stale timestamps. A retry is signed again with a new timestamp.
import hashlib, hmac, time
def verify(body: bytes, header: str, secret: str, tolerance: int = 300) -> bool:
fields = dict(p.split("=", 1) for p in header.split(",") if "=" in p)
ts, sig = fields.get("t", ""), fields.get("v1", "")
if not ts.isdigit() or abs(time.time() - int(ts)) > tolerance:
return False
mac = hmac.new(secret.encode(), ts.encode() + b"." + body, hashlib.sha256).hexdigest()
return hmac.compare_digest(mac, sig)Responses, retries and dead letters
| Your endpoint returns | vBilling |
|---|---|
| 2xx | Counts the batch as delivered and moves the cursor on. Return it only once the batch is stored. |
| 400, 413 or 422 | Treats the payload as rejected for good: it splits the batch until each rejected event is alone, parks those as dead letters and delivers the rest. |
| Anything else, or no answer within 30s | Retries the whole batch with exponential backoff (1s, doubling up to 5 min) while later events wait. Auth errors, 404, 409, 429 and 5xx never dead-letter an event. |
id. A batch can arrive twice: after a timeout, a restart before the cursor was saved, a dead-letter replay or a cursor rewind. A usage event keeps its ID for the same window, so upsert on it. Tenant and catalog IDs carry a timestamp: keep the latest. Return 2xx for types you ignore. A catalog event your endpoint rejects with 400, 413 or 422 is logged and skipped; any other failure holds delivery to the endpoint until the catalog event is accepted.Dead letters stay on the ledger volume until a replay clears them. Add -H "Authorization: Bearer $API_TOKEN" when the API is protected:
curl -s localhost:8080/api/v1/destinations/webhook/dead-letters \
| jq '.dead_letters[] | {reason, id: .event.id}'
curl -s -X POST localhost:8080/api/v1/destinations/webhook/dead-letters/replay
# send the retained ledger (RETENTION, default 7 days) to a new endpoint
curl -s -X POST localhost:8080/api/v1/destinations/webhook/cursor -d '{"seq": 0}'Ingest API
Usage vBilling cannot measure itself, such as inference tokens from a model gateway or Slurm accounting, goes in through POST /api/v1/events. Accepted events are committed to the same ledger as collector events and fan out to every adapter, the webhook and your own adapter included. Declare the metric, enable ingest with a token, and send closed windows:
helm upgrade vbilling deploy/helm/vbilling -n vbilling-system --reuse-values \
--set metering.customMetrics='inference_output_tokens:tokens:model' \
--set api.existingSecret=vbilling-api # keys: api-token, ingest-token
curl -X POST localhost:8080/api/v1/events -H "Authorization: Bearer $INGEST_TOKEN" \
-H "X-VBilling-Client: model-gateway" -d '{"events": [{
"tenant": "acme-ai", "metric": "inference_output_tokens", "quantity": 1250000,
"window_start": "2026-10-05T10:00:00Z", "window_end": "2026-10-05T10:01:00Z",
"dimensions": {"model": "llama-3.3-70b"}}]}'- Ingest is off until
INGEST_TOKEN(orAPI_TOKEN) is set. Up to 5,000 events or 10 MiB per request, as{"events": [...]}or a bare array. - The metric must be built in or declared in
CUSTOM_METRICS(code:unit[:group_key|group_key]). Custom metrics are billable, so billing adapters create them at bootstrap. window_endmay be at most 5 minutes in the future andwindow_startat most 34 days old.regiondefaults toREGION, andsourcebecomesingest:<X-VBilling-Client>.- Send a stable
id, or let vBilling derive one from the event's identity, which excludes the quantity. An ID seen in the last 48 hours is reported as a duplicate and dropped, so send each window once, after it closes. - Events are validated one by one: the response lists accepted IDs, duplicates and rejections with their index and reason, and is a 400 only when no event passed.
See Configuration and Custom metrics for the full reference.
Go adapter
For a billing platform with its own API, implement the Destination interface in a Go package. vBilling's packages are under internal/, so the adapter lives in your fork of the repository, at internal/destinations/<name>, and ships in your own image.
// internal/destinations/destination.go (package destinations)
type Destination interface {
Name() string
Bootstrap(ctx context.Context, metrics []usage.MetricDef) error
EnsureTenant(ctx context.Context, t usage.Tenant) error
RemoveTenant(ctx context.Context, t usage.Tenant) error
SendEvents(ctx context.Context, events []usage.Event) error
}| Method | Called | Contract |
|---|---|---|
Name | Wherever the adapter is named: status, logs, /api/v1/destinations/<name> | Return the name you register, the one operators list in adapters (ADAPTERS). A name may be listed once. |
Bootstrap | When delivery starts, with the whole catalog (built-in, informational and custom metrics) | Create meters or billable metrics. No event is delivered until it succeeds; failures are retried with backoff. |
EnsureTenant | Before a tenant's first event reaches the adapter, when its details change, and after a restart | Create or update the customer. A Permanent error is logged and delivery continues; any other error retries the batch. |
RemoveTenant | Once a tenant has had no tenant clusters for OFFBOARD_GRACE | Decide what offboarding means. Never delete a customer with open invoices. An error is retried on the next pass. |
SendEvents | With batches of up to 500 events from the ledger, and with every parked dead letter on a replay | Deliver them. The error you return decides what happens next (below). |
vBilling retries every method freely, so each must be idempotent. Three optional interfaces add capabilities:
| Interface | Method | Purpose |
|---|---|---|
EventFilter | Accepts(metric string) bool | Skip metrics the platform should not see. destinations.BillableOnly drops the informational ones. Skipped events are counted, never retried. |
Reconciler | RecordedTotals(ctx context.Context, q TotalsQuery) ([]Total, error) | Report what the platform recorded per tenant and metric, so GET /api/v1/reconcile can compare it with the ledger. |
CustomerResolver | TenantForCustomer(ctx context.Context, customerID string) (string, bool) | Map a platform customer ID back to a tenant for inbound billing webhooks. Only the Stripe and Metronome adapters' webhooks (/webhooks/stripe, /webhooks/metronome) consult it today; any other billing platform sets billing states with PUT /api/v1/billing-states/{tenant}. |
Errors decide what happens to events
| SendEvents returns | vBilling |
|---|---|
nil | Counts every event as delivered and moves the cursor on. |
&destinations.PartialError{Rejected: rejected}, mapping event IDs to reasons | Dead-letters the listed events and counts the rest as delivered. Return it only when nothing in the batch needs a retry. |
destinations.Permanent(err) | Splits the batch until each rejected event is alone, then dead-letters it. |
| Any other error | Retries the whole batch with backoff while later events wait. Use it for auth errors, rate limits and outages: dead-lettering usage because a key expired would lose revenue. |
destinations.StatusError applies the same rule to HTTP responses: 400, 413 and 422 are permanent, everything else is retried.
Idempotency. Event IDs are deterministic: vb1_ plus a SHA-256 of the event's identity (tenant, metric, window, region, project, SKU, resource ID and dimensions), never of its quantity. A window keeps its ID through restarts, backfills, replays and a second replica. Send ev.ID as the platform's idempotency key and treat "already exists" as success. A retried batch repeats events you already delivered; destinations.SentCache skips them without a request. Events can also arrive late (backfill up to MAX_BACKFILL, default 6h, tenant API gap fills, replays), so accept backdated timestamps, or reject too-old events one by one with a PartialError, as the Stripe adapter does for Stripe's 35-day limit.
A minimal adapter
internal/destinations/acme/adapter.go, for a platform that takes JSON over HTTPS. Its settings come from the environment in the factory:
// Package acme ships usage to the Acme billing API.
package acme
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"net/http"
"os"
"github.com/vclusterlabs-experiments/vbilling/internal/config"
"github.com/vclusterlabs-experiments/vbilling/internal/destinations"
"github.com/vclusterlabs-experiments/vbilling/internal/usage"
)
func init() { destinations.Register("acme", factory) }
func factory(cfg *config.Config) (destinations.Destination, error) {
url, key := os.Getenv("ACME_URL"), os.Getenv("ACME_API_KEY")
if url == "" || key == "" {
return nil, errors.New("ACME_URL and ACME_API_KEY must be set")
}
return New(url, key), nil
}
type Adapter struct {
url, key string
http *http.Client
sent *destinations.SentCache
}
func New(url, key string) *Adapter {
return &Adapter{url: url, key: key, http: destinations.NewHTTPClient(),
sent: destinations.NewSentCache(100_000)}
}
func (a *Adapter) Name() string { return "acme" }
// Accepts skips informational metrics (GPU utilization, GPU downtime).
func (a *Adapter) Accepts(metric string) bool { return destinations.BillableOnly(metric) }
func (a *Adapter) Bootstrap(ctx context.Context, metrics []usage.MetricDef) error {
for _, m := range metrics {
if !m.Billable {
continue
}
meter := map[string]any{"name": m.Name, "unit": m.Unit, "group_by": m.GroupKeys}
if err := a.post(ctx, "/meters/"+m.Code, meter); err != nil {
return err
}
}
return nil
}
func (a *Adapter) EnsureTenant(ctx context.Context, t usage.Tenant) error {
return a.post(ctx, "/customers/"+t.ID, map[string]any{
"name": t.DisplayName, "email": t.Email, "currency": t.Currency,
})
}
// RemoveTenant keeps the customer: its final usage still has to be invoiced.
func (a *Adapter) RemoveTenant(ctx context.Context, t usage.Tenant) error { return nil }
func (a *Adapter) SendEvents(ctx context.Context, events []usage.Event) error {
rejected := map[string]string{}
for _, ev := range events {
if a.sent.Has(ev.ID) {
continue // delivered before this batch was retried
}
err := a.post(ctx, "/usage", map[string]any{
"idempotency_key": ev.ID, "customer": ev.Tenant, "meter": ev.Metric,
"quantity": ev.Quantity, "timestamp": ev.WindowStart, "sku": ev.SKU,
"region": ev.Region, "dimensions": ev.Dimensions,
})
switch {
case err == nil:
a.sent.Add(ev.ID)
case destinations.IsPermanent(err):
rejected[ev.ID] = err.Error() // dead-lettered; the rest is delivered
default:
return err // the batch is retried with backoff
}
}
if len(rejected) > 0 {
return &destinations.PartialError{Rejected: rejected}
}
return nil
}
func (a *Adapter) post(ctx context.Context, path string, body any) error {
b, err := json.Marshal(body)
if err != nil {
return err
}
url := a.url + path
req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(b))
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+a.key)
req.Header.Set("Content-Type", "application/json")
resp, err := a.http.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
msg, _ := io.ReadAll(io.LimitReader(resp.Body, 64<<10))
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
return nil
}
return destinations.StatusError(http.MethodPost, url, resp.StatusCode, msg)
}
var (
_ destinations.Destination = (*Adapter)(nil)
_ destinations.EventFilter = (*Adapter)(nil)
)Register it with a blank import next to the built-in adapters in cmd/vbilling/main.go:
// Register the remaining built-in adapters.
_ "github.com/vclusterlabs-experiments/vbilling/internal/destinations/lago"
_ "github.com/vclusterlabs-experiments/vbilling/internal/destinations/noop"
_ "github.com/vclusterlabs-experiments/vbilling/internal/destinations/webhook"
_ "github.com/vclusterlabs-experiments/vbilling/internal/destinations/acme"Test it the way internal/destinations/webhook/adapter_test.go does: point the adapter at an httptest server and check that a 422 returns a PartialError and a 503 a retryable error. Then build and deploy your image:
go test -race ./internal/destinations/acme/
make test lint
make docker-push IMAGE=registry.example.com/vbilling TAG=acme-1
helm upgrade --install vbilling deploy/helm/vbilling -n vbilling-system \
--set adapters='{acme}' \
--set image.repository=registry.example.com/vbilling --set image.tag=acme-1The chart has no generic extra environment value: add ACME_URL, and ACME_API_KEY from a Secret, to the container's env in deploy/helm/vbilling/templates/statefulset.yaml in your fork.
Dry runs with noop
--set adapters='{noop}' delivers nowhere and logs every event, for example [noop] event tenant=acme-ai metric=vcluster_gpu_hours qty=0.133333333 gpu-hours sku=NVIDIA-H100-80GB-HBM3 region=us-east-1 window=10:00:00. Use it to check metering before a billing platform is connected, or list it next to an adapter you are building.