Lago

Usage-based billing on open source from end to end. Lago rates and invoices, vBilling meters your tenant clusters and sends Lago the events, and both can run on a laptop next to your tenant clusters.

What vBilling sets up

In LagoCreated by vBilling
Billable metricsOne per vBilling metric (vcluster_gpu_hours, vcluster_cpu_core_hours, ...), summing a field named after it (gpu_hours, cpu_core_hours, ...)
Planvcluster-standard (billing.defaultPlanCode) with one charge per metric at $0, for you to price
CustomersOne per tenant: external_id is the tenant ID, with its name, email, currency and metadata
Subscriptionssub-<tenant ID> on the tenant's plan (vbilling.vcluster.com/plan, or the default plan)
EventsBatches of up to 100. transaction_id is the vBilling event ID, so a resent event is never counted twice.

Prices, invoices, taxes and the payment provider stay in Lago.

Run Lago locally

The repository ships a Docker Compose file for Lago 1.17, with the UI on :8081 and the API on :3000 (vBilling's own dashboard uses :8080). Lago signs its tokens with an RSA key: generate one, and never commit it. From the repository root:

git clone https://github.com/vClusterLabs-Experiments/vbilling.git && cd vbilling openssl genrsa 2048 > deploy/lago/lago_rsa.key echo "LAGO_RSA_PRIVATE_KEY=$(base64 -i deploy/lago/lago_rsa.key | tr -d '\n')" > deploy/lago/.env docker compose -f deploy/lago/docker-compose.yml --env-file deploy/lago/.env up -d

Open http://localhost:8081 and sign up to create the organization. Then copy the API key from Developers, API keys into a file (paste it, press Enter, then Ctrl-D):

cat > ./lago-api-key

For a shared or production setup, run Lago in Kubernetes with its Helm chart (see the Lago docs) and point lago.apiURL at its API Service.

Install vBilling with Lago

kubectl create namespace vbilling-system kubectl -n vbilling-system create secret generic vbilling-lago --from-file=api-key=./lago-api-key helm upgrade --install vbilling deploy/helm/vbilling -n vbilling-system \ --set adapters='{lago}' \ --set lago.apiURL=http://lago-api.lago-system.svc.cluster.local:3000 \ --set lago.existingSecret=vbilling-lago
Helm valueEnvironment variableDefaultEffect
lago.apiURLLAGO_API_URLhttp://lago-api.lago-system.svc.cluster.local:3000Lago API, reachable from the vBilling pod. Use http://host.docker.internal:3000 for Lago in Docker on the same machine as a vind control plane cluster.
lago.existingSecretSecret with the key api-key. Use it instead of lago.apiKey.
lago.apiKeyLAGO_API_KEYLago API key.
billing.defaultPlanCodeDEFAULT_PLAN_CODEvcluster-standardPlan that vBilling creates, and that tenants without a plan annotation subscribe to.
billing.currencyBILLING_CURRENCYUSDCurrency of the plan and of tenants without a currency annotation.

Check that vBilling bootstrapped Lago and subscribed the first tenants:

kubectl -n vbilling-system logs statefulset/vbilling | grep -E '\[(bootstrap|lago)\]' [bootstrap] created metric "vcluster_gpu_hours" (id=...) [bootstrap] created plan "vcluster-standard" (id=...) with ... charges [lago] subscribed tenant vcluster-team-a-team-a to plan vcluster-standard curl -s http://localhost:3000/api/v1/customers -H "Authorization: Bearer $(cat ./lago-api-key)" | jq '.customers[].external_id'

Configure pricing

Every charge on vcluster-standard starts at $0, so invoices stay at $0.00 until you price them. In the Lago UI, open Plans, vCluster Standard, and set a price per unit on each charge. vBilling sends every dimension as an event property, so charge filters can price one metric several ways:

To price tenants differently, create more plans and set vbilling.vcluster.com/plan on their tenant clusters. Lago pricing lists the charges to configure, with example price lists.

Events in Lago

Each event carries the quantity under the metric's field name, which the billable metric sums, and every dimension as a property (trimmed here):

{ "transaction_id": "vb1_3f9c...", "external_subscription_id": "sub-vcluster-team-a-team-a", "code": "vcluster_gpu_hours", "timestamp": 1791216000, "properties": { "gpu_hours": 0.25, "sku": "NVIDIA-H100-80GB-HBM3", "capacity_type": "on-demand", "billing_mode": "shared", "region": "us-east-1", "tenant_cluster": "vcluster-team-a-team-a", "vcluster": "vcluster-team-a-team-a" } }

The timestamp is the window start. vcluster repeats tenant_cluster so that charge filters written for v0.1 keep working.

Offboarding

When a tenant has been gone for billing.offboardGrace (default 1h), vBilling terminates its subscription and Lago issues the final invoice.

Good to know

Run it all on a laptop with vind

vind runs vCluster in Docker, so a control plane cluster, three tenant clusters, vBilling and Lago fit on one machine. Start Lago as above, then, from the repository root:

# a control plane cluster in Docker, with metrics-server vcluster use driver docker vcluster create vbilling-host --connect=true kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml kubectl patch deployment metrics-server -n kube-system --type=json \ -p='[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--kubelet-insecure-tls"}]' # three tenant clusters inside it vcluster use driver helm vcluster create team-alpha --namespace vcluster-team-alpha --connect=false vcluster create team-beta --namespace vcluster-team-beta --connect=false vcluster create team-gpu --namespace vcluster-team-gpu --connect=false # vBilling, sending to Lago on the host kubectl create namespace vbilling-system kubectl -n vbilling-system create secret generic vbilling-lago --from-file=api-key=./lago-api-key helm upgrade --install vbilling deploy/helm/vbilling -n vbilling-system \ --set adapters='{lago}' \ --set lago.apiURL=http://host.docker.internal:3000 \ --set lago.existingSecret=vbilling-lago

Within a minute or two the log shows a window line for the three tenant clusters and a subscription per tenant:

kubectl -n vbilling-system logs statefulset/vbilling [controller] starting (adapters=[lago], window=1m0s, reconcile=30s, region=default) API listening on :8080 [controller] window ...: ... events for 3 tenant cluster(s) (...) [lago] subscribed tenant vcluster-vcluster-team-alpha-team-alpha to plan vcluster-standard

Open the dashboard with kubectl -n vbilling-system port-forward svc/vbilling 8080 and http://localhost:8080/, and the customers in Lago at http://localhost:8081. To clean up:

vcluster use driver docker vcluster delete vbilling-host docker compose -f deploy/lago/docker-compose.yml down -v

Troubleshooting