Using the GPU Inference Cost Calculator
Enter billed hours for one GPU and measured throughput for one active GPU. Use the share of billed time spent serving traffic to account for idle capacity.
Worked example
Two GPUs billed for 100 hours each cost 400 at 2 per GPU-hour. At 50% active time and five requests per second per GPU, they serve 1.8 million requests.
Example inputs
- Number of GPUs
- 2
- Billed hours per GPU for the period
- 100
- Cost per GPU-hour
- 2
- Requests per second per active GPU
- 5
- Share of billed time serving requests (%)
- 50
Example result
- GPU cost for the period
- 400
- Billed GPU-hours
- 200
- Estimated requests served
- 1,800,000
- Cost per request
- 0.00022222222
Method and assumptions
Billed GPU-hours include idle time. Estimate served requests from active seconds and measured throughput per GPU.
Model billable usage
The starting rates are illustrative. For cost estimates, use your provider’s billing units, region and plan, and keep all prices in the same currency.
Common question
What if a single model replica spans several GPUs?
Divide the measured throughput of the whole replica by its GPU count before entering throughput per GPU. Include all billed GPUs in the fleet count.