Technology

GPU Inference Cost Calculator

Estimate billed GPU-hours, how many inference requests your fleet can serve, and the average GPU cost per request.

01

Your inputs

02

Your result

Ready when you are.

Adjust the inputs, then calculate to see your result here.

Calculated on your device.

GET MORE FROM YOUR RESULT

Using the GPU Inference Cost Calculator

Enter billed hours for one GPU and measured throughput for one active GPU. Use the share of billed time spent serving traffic to account for idle capacity.

Worked example

Two GPUs billed for 100 hours each cost 400 at 2 per GPU-hour. At 50% active time and five requests per second per GPU, they serve 1.8 million requests.

Example inputs

Number of GPUs
2
Billed hours per GPU for the period
100
Cost per GPU-hour
2
Requests per second per active GPU
5
Share of billed time serving requests (%)
50

Example result

GPU cost for the period
400
Billed GPU-hours
200
Estimated requests served
1,800,000
Cost per request
0.00022222222

Method and assumptions

Billed GPU-hours include idle time. Estimate served requests from active seconds and measured throughput per GPU.

Model billable usage

The starting rates are illustrative. For cost estimates, use your provider’s billing units, region and plan, and keep all prices in the same currency.

Common question

What if a single model replica spans several GPUs?

Divide the measured throughput of the whole replica by its GPU count before entering throughput per GPU. Include all billed GPUs in the fleet count.