MIG vs MPS vs Time-Slicing: Sharing One GPU for vLLM

Time-slicing divides time: each pod runs as many processes on the underlying GPU as it likes, and the GPU provides an equal share of time to all of them.

MPS divides space through a userspace control daemon that allows memory and compute resources to be explicitly partitioned, and enforces those limits per workload.

MIG divides the silicon: each instance's processors have separate and isolated paths through the entire memory system, with the on-chip crossbar ports, L2 cache banks, memory controllers and DRAM address busses assigned uniquely to one instance. NVIDIA calls that memory and fault isolation at the hardware layer, and never uses the phrase for MPS.

Its costs mirror that: fixed profiles, geometry changes that need the GPU free of user workloads, CAP_SYS_ADMIN and a driver-handle daemon stop to turn MIG mode on, and a per-card instance ceiling that varies by product, so 7 is not universal. Our RTX PRO 6000 vs H100 vs L40S comparison has those counts; isolating a workload from its host is a different control, priced in confidential GPU inference.

The README sentence to bring to a design review: nothing special is done to isolate workloads granted replicas from the same underlying GPU, each has access to the GPU memory, and they share a fault-domain, so if one crashes they all do.

The second misunderstanding is arithmetic. replicas multiplies a resource's advertised count; it does not divide capacity. A node with 8 GPUs at replicas: 10 advertises 80 nvidia.com/gpu, and a pod requesting two gets two scheduling tokens and no extra compute: more than one time-sliced GPU carries no proportional guarantee.

failRequestsGreaterThanOne turns that silent non-event into an UnexpectedAdmissionError: kubelet reports nvidia.com/gpu: 2 as too large because the maximum request size for shared resources is 1. NVIDIA recommends true and ships false for backwards compatibility, so a time-sliced node has it off unless you set it.

apiVersion: v1
kind: ConfigMap
metadata:
  name: time-slicing-config
data:
  any: |-
    version: v1
    flags:
      migStrategy: none
    sharing:
      timeSlicing:
        renameByDefault: true
        failRequestsGreaterThanOne: true
        resources:
        - name: nvidia.com/gpu
          replicas: 4

Wiring it into the operator is a create and a patch:

kubectl create -n gpu-operator -f time-slicing-config.yaml
kubectl patch clusterpolicies.nvidia.com/cluster-policy \
  -n gpu-operator --type merge \
  -p '{"spec": {"devicePlugin": {"config": {"name": "time-slicing-config", "default": "any"}}}}'

renameByDefault: true advertises nvidia.com/gpu.shared, which keeps whole-card workloads off a sliced node. Set it false and the product label is suffixed instead: nvidia.com/gpu.product = A100-SXM4-40GB-SHARED.

DCGM-Exporter also stops associating metrics to containers once time-slicing is enabled with the NVIDIA Kubernetes device plugin, and an equal share of time is not a service level objective, which puts a shared card and P95 TTFT targets in tension.

The device plugin README at v0.20.0 has one line on this: sharing with MPS is currently not supported on devices with MIG enabled. That is a statement about the plugin, not the hardware. The MIG User Guide says CUDA MPS is supported on top of MIG, the 48-client maximum simply lowered proportionally to the Compute Instance size.

They agree once you notice what the plugin daemon does first. It sets EXCLUSIVE_PROCESS, and the MIG guide says that mode is not supported in MIG mode because multiple MPS servers are used, one per MIG GPU instance. NVIDIA does not state that as its reason, so treat it as two documents agreeing, not causation. Outside Kubernetes the combination is a loop over instance UUIDs, one control daemon and pipe directory each:

for mig_device in "${MIG_DEVICES[@]}"; do
  export CUDA_MPS_PIPE_DIRECTORY=/tmp/$mig_device
  mkdir -p "$CUDA_MPS_PIPE_DIRECTORY"
  sudo CUDA_VISIBLE_DEVICES=$mig_device \
    CUDA_MPS_PIPE_DIRECTORY=/tmp/$mig_device \
    nvidia-cuda-mps-control -d
  CUDA_MPS_PIPE_DIRECTORY=/tmp/$mig_device \
    CUDA_VISIBLE_DEVICES=$mig_device \
    my-cuda-app &
done

Inside the plugin the pairing that works is mixed MIG strategy plus time-slicing: the supported resource types are nvidia.com/gpu and any emerging from the mixed strategy. A nvidia.com/mig-3g.40gb can be time-sliced. It cannot run MPS.

From the same README: with both time-slicing and MPS, the same sharing method is applied to all GPUs on a node, and you cannot configure sharing on a per-GPU basis. They are also mutually exclusive, and the plugin picks MPS when both blocks appear. The isolation decision therefore lives in the node pool, not the pod spec, at a granularity of one config key per node: a multi-key ConfigMap plus the nvidia.com/device-plugin.config label. The devices selector does not help; the plugin logs that customising it is not yet supported and ignores it.

The MPS row ends arguments: all MPS client behavior will be attributed to the MPS server process by system monitoring and accounting tools, nvidia-smi and NVML named explicitly. Tell an assessor Department A's inference is separable from Department B's while the node label says mps or time-slicing, and NVIDIA's documentation contradicts you. On-premise there is no per-instance invoice either.

Here is the line that causes the outage. In vLLM v0.27.1 the worker computes its requested memory as the total memory in its initial snapshot times gpu_memory_utilization, rounded up. Total, not free. The snapshot reads the accelerator runtime directly, with no branch for time-slicing, MPS or a replica count, and the default is 0.92.

The docstring is unambiguous that the split is your job: the limit is per-instance, it does not matter whether another vLLM instance is running on the same GPU, and two instances on one GPU should each be set to 0.5. Nothing in the sharing layer does that for you, because the replica count never reaches the serving process. The second replica reads the whole card's total memory, asks for 92 percent of it, and hits one of two failures depending on timing.

ValueError: Free memory on device cuda:0 (12.4/79.1 GiB) on startup is less than
desired GPU memory utilization (0.92, 72.8 GiB). Decrease GPU memory utilization
or reduce GPU memory used by other processes.

AssertionError: Error in memory profiling. Initial free memory 61.3 GiB, current
free memory 68.9 GiB. This happens when other processes sharing the same container
release GPU memory while vLLM is profiling during initialization. To fix this,
ensure consistent GPU memory allocation or isolate vLLM in its own container.

The first is the polite one: the co-tenant already held memory at startup and the check caught it. The second fires the other way round, when a co-tenant releases memory mid-profile and free memory ends up higher than the worker measured. The rule that prevents both: gpu_memory_utilization at or below 1 over the replica count, minus headroom, on every pod.

# one of four replicas on an 80 GB H100: the 0.92 default would claim the whole card
vllm serve /models/qwen3-8b \
  --gpu-memory-utilization 0.22 \
  --max-model-len 8192

# resources:
#   limits:
#     nvidia.com/gpu.shared: 1

The dedicated-card version of that memory equation is derived in full in debugging vLLM KV cache OOM.

Under MIG the instance is the device: nvidia-smi reports each MIG device's own memory total, so on an 80 GB H100 the card row reads 87MiB of 81559MiB while each 3g.40gb device row reads 44MiB of 40448MiB.

Memory slices are roughly eighths of the card and SM slices roughly sevenths, which is why the seven-instance geometry addresses 70 GB and leaves an eighth unassigned.

A 10 GB instance holds weights, activations and the KV cache, and the memory-versus-concurrency arithmetic in our Kimi K3 GPU sizing analysis shows how fast the cache side grows. Seven tenants each get a slice that will not serve a useful model; three 2g.20gb instances partition the card more honestly. Nor can you span slices: NCCL is not supported with MIG, and with driver R570 only P2P between MIG instances on the same GPU is supported. If prefill and decode contention is the real problem, split the phases across pools instead, as in prefill and decode disaggregation.

Work backwards from the isolation requirement, never forwards from the pod count.

Write the same four lines down whichever row you land on, because none appear in a pod spec:

This week, print the four sharing-related labels on every GPU node and check them against what you have already told an assessor.

kubectl get nodes -o custom-columns=\
NODE:.metadata.name,\
SHARING:.metadata.labels.nvidia\\.com/gpu\\.sharing-strategy,\
REPLICAS:.metadata.labels.nvidia\\.com/gpu\\.replicas,\
MIG:.metadata.labels.nvidia\\.com/mig\\.strategy,\
MPS:.metadata.labels.nvidia\\.com/mps\\.capable

If that column reads time-slicing on a node where two departments run, fix the document before you move the workload. For the platform decisions around this one, start at our AI development tools pillar.