Compute: VMs, containers and serverless

Every design has boxes labelled "service", and sooner or later someone asks what they run on. Virtual machines, Kubernetes, serverless containers and functions all run your code; they differ in how they scale, what you pay for while nothing happens, how long a request may take, and how much of the platform your team must operate. This guide explains each, how the clouds' versions compare, and which to choose, with a chooser and code you can run.

1. Four ways to run code

Each step down the table hands more of the work to the provider, and takes away some control.

2. How scaling to zero works

The serverless models start instances when requests arrive and stop them when traffic stops. That's what makes them cheap for spiky traffic, and what causes cold starts.

3. The products

4. Differences at a glance

5. When to choose which, and why

1. Is it a stateless HTTP service? Start with serverless containers (Cloud Run, Container Apps, or Fargate on AWS). You ship a container, nothing else to run.
2. Is it triggered by events (a file upload, a queue message, a schedule) and short? A function is the least code and the least cost.
3. Many services, a platform team, special needs (GPUs, sidecars, long-lived connections, custom scheduling)? Kubernetes.
4. Need the whole machine (a database you run, licensed software, kernel tuning)? VMs.
5. Steady, high traffic all day? Always-on capacity (VMs, Kubernetes, or minimum instances) is usually cheaper than paying per request; scale-to-zero pays off when traffic is spiky or often idle.

Try it: choose for your requirements

6. One day of traffic, three ways, in code

A service's traffic for each hour of a day, at 0.2 seconds per request. Little's law turns requests per second into requests in flight. VMs sized for the peak run all day; autoscaled containers follow the traffic hour by hour; functions run one request per environment, so they need far more environment-hours, but each is billed only while busy and nothing while idle. The program was run in all three languages; each prints the output below it.

import math

# Requests per second in each hour of one day, and how long each request takes.
RPS = [2, 1, 1, 0, 0, 1, 5, 20, 60, 90, 100, 95, 110, 100, 90, 85, 80, 70, 60, 40, 25, 15, 8, 4]
SECONDS_PER_REQUEST = 0.2
PER_INSTANCE = 20          # requests one VM or container instance handles at once

# Little's law: requests in flight = arrival rate x time each one takes.
in_flight = [rps * SECONDS_PER_REQUEST for rps in RPS]
peak = max(in_flight)
print(f"peak in flight: {peak:.0f} requests")

# VMs: sized once for the peak plus 30% headroom, running all day.
vms = math.ceil(peak * 1.3 / PER_INSTANCE)
print(f"VMs sized for peak: {vms} instances x 24 h = {vms * 24} instance-hours")

# Containers that autoscale each hour (minimum one instance), like Kubernetes or Cloud Run with min instances = 1.
containers = sum(max(1, math.ceil(c / PER_INSTANCE)) for c in in_flight)
print(f"autoscaled containers: {containers} instance-hours")

# Functions: one request per execution environment, billed only while running.
busy_seconds = sum(RPS) * 3600 * SECONDS_PER_REQUEST
print(f"functions: {busy_seconds:,.0f} busy seconds = {busy_seconds / 3600:.1f} environment-hours, "
      f"peak {math.ceil(peak)} environments at once")

idle = sum(1 for rps in RPS if rps == 0)
print(f"idle hours: {idle}; scale-to-zero pays nothing then, and the next request is a cold start")

// Requests per second in each hour of one day, and how long each request takes.
const RPS = [2, 1, 1, 0, 0, 1, 5, 20, 60, 90, 100, 95, 110, 100, 90, 85, 80, 70, 60, 40, 25, 15, 8, 4];
const SECONDS_PER_REQUEST = 0.2;
const PER_INSTANCE = 20; // requests one VM or container instance handles at once

// Little's law: requests in flight = arrival rate x time each one takes.
const inFlight = RPS.map((rps) => rps * SECONDS_PER_REQUEST);
const peak = Math.max(...inFlight);
console.log(`peak in flight: ${peak.toFixed(0)} requests`);

// VMs: sized once for the peak plus 30% headroom, running all day.
const vms = Math.ceil((peak * 1.3) / PER_INSTANCE);
console.log(`VMs sized for peak: ${vms} instances x 24 h = ${vms * 24} instance-hours`);

// Containers that autoscale each hour (minimum one instance), like Kubernetes or Cloud Run with min instances = 1.
const containers = inFlight.reduce((sum, c) => sum + Math.max(1, Math.ceil(c / PER_INSTANCE)), 0);
console.log(`autoscaled containers: ${containers} instance-hours`);

// Functions: one request per execution environment, billed only while running.
const busySeconds = RPS.reduce((a, b) => a + b, 0) * 3600 * SECONDS_PER_REQUEST;
console.log(`functions: ${Math.round(busySeconds).toLocaleString("en-US")} busy seconds = ${(busySeconds / 3600).toFixed(1)} environment-hours, ` +
  `peak ${Math.ceil(peak)} environments at once`);

const idle = RPS.filter((rps) => rps === 0).length;
console.log(`idle hours: ${idle}; scale-to-zero pays nothing then, and the next request is a cold start`);

import java.util.Arrays;
import java.util.Locale;

public class Main {
    // Requests per second in each hour of one day, and how long each request takes.
    static final int[] RPS = {2, 1, 1, 0, 0, 1, 5, 20, 60, 90, 100, 95, 110, 100, 90, 85, 80, 70, 60, 40, 25, 15, 8, 4};
    static final double SECONDS_PER_REQUEST = 0.2;
    static final int PER_INSTANCE = 20; // requests one VM or container instance handles at once

    public static void main(String[] args) {
        // Little's law: requests in flight = arrival rate x time each one takes.
        double[] inFlight = Arrays.stream(RPS).mapToDouble(rps -> rps * SECONDS_PER_REQUEST).toArray();
        double peak = Arrays.stream(inFlight).max().orElse(0);
        System.out.printf("peak in flight: %.0f requests%n", peak);

        // VMs: sized once for the peak plus 30% headroom, running all day.
        int vms = (int) Math.ceil(peak * 1.3 / PER_INSTANCE);
        System.out.printf("VMs sized for peak: %d instances x 24 h = %d instance-hours%n", vms, vms * 24);

        // Containers that autoscale each hour (minimum one instance), like Kubernetes or Cloud Run with min instances = 1.
        int containers = Arrays.stream(inFlight).mapToInt(c -> Math.max(1, (int) Math.ceil(c / PER_INSTANCE))).sum();
        System.out.printf("autoscaled containers: %d instance-hours%n", containers);

        // Functions: one request per execution environment, billed only while running.
        double busySeconds = Arrays.stream(RPS).sum() * 3600 * SECONDS_PER_REQUEST;
        System.out.printf(Locale.US, "functions: %,.0f busy seconds = %.1f environment-hours, peak %d environments at once%n",
                busySeconds, busySeconds / 3600, (int) Math.ceil(peak));

        long idle = Arrays.stream(RPS).filter(rps -> rps == 0).count();
        System.out.printf("idle hours: %d; scale-to-zero pays nothing then, and the next request is a cold start%n", idle);
    }
}

peak in flight: 22 requests
VMs sized for peak: 2 instances x 24 h = 48 instance-hours
autoscaled containers: 25 instance-hours
functions: 764,640 busy seconds = 212.4 environment-hours, peak 22 environments at once
idle hours: 2; scale-to-zero pays nothing then, and the next request is a cold start

7. Say it in the interview

8. Common mistakes

9. Check yourself

10. Sources and further reading

Product facts checked on 3 November 2026. Limits and features change; check before relying on them.