Kubernetes Device Plugin to DRA Migration: The Go or No-Go Call

Dynamic Resource Allocation has been stable in Kubernetes since v1.34, the NVIDIA GPU Operator docs now tell you to stop deploying the device plugin, and the extended-resource bridge between the two went GA in v1.37. So the move lands on roadmaps. Meanwhile the GPU nodes work: pods ask for nvidia.com/gpu: 1, the device plugin hands over a card, and no one has filed a ticket. We have walked several teams through this decision and sent most of them away without migrating yet. The blocker is almost never the Kubernetes version. What stops them is the compute type they provision with and the way they currently share cards.

This post is about the go or no-go call and the order of operations if you go. For the billing side of GPUs, our GPU cost allocation playbook covers the chargeback models and the instrumentation, and it picks up where this one stops.

The version floor is three numbers

Half the confusion in search results traces back to people quoting one version number for a feature that landed across three releases. Pin all three before you plan anything.

Kubernetes core DRA APIs graduated to stable in v1.34, which is how the Amazon EKS release notes record it under "Dynamic Resource Allocation (DRA) Core APIs (GA)". The upstream DRA concept page lists the feature as stable since v1.35 instead. Both are right. The API group settled in 1.34, and in 1.35 the DynamicResourceAllocation gate was locked on, so you can no longer turn it off. Blogs that pick one of those and present it as the answer are the reason your team is arguing.

The driver has its own floor, and it is stricter than either. NVIDIA's DRA driver install guide lists "Kubernetes v1.34.2 or later with a resource.k8s.io DeviceClass API available", "NVIDIA GPU driver version 580 or later", and "An underlying container runtime that supports CDI and is configured to use CDI for injecting DRA-allocated devices." The patch version matters. A cluster on 1.34.0 does not qualify.

| Floor | Value | Where it comes from | |---|---|---| | Control plane | v1.34.2 | NVIDIA DRA driver prerequisites | | Gate locked on | v1.35 | Upstream DRA concept page | | Extended resource bridge | v1.36 (beta, on by default); GA in v1.37 | Kubernetes feature gates reference | | GPU driver | 580 | NVIDIA DRA driver prerequisites | | Node runtime | CDI configured | NVIDIA DRA driver prerequisites |

The driver also moved house, which trips up anyone copying a year-old Helm command. The v0.4.0 release notes say "The DRA Driver for NVIDIA GPUs has moved from the NVIDIA org to kubernetes-sigs as part of its donation to CNCF." The chart now lives at oci://registry.k8s.io/dra-driver-nvidia/charts/dra-driver-nvidia-gpu, and the current release is v0.5.1, dated 12 October. AWS's own install example still pins 0.4.1, so read the chart values for whatever version you install rather than trusting any guide's flags.

What DRA replaces, and what it leaves alone

The device plugin advertises a counter. Kubernetes docs on device plugins state the two limits that follow from that: "Extended resources are only supported as integer resources and cannot be overcommitted," and "Devices cannot be shared between containers." The scheduler knows a node has four of something called nvidia.com/gpu. It does not know whether those are L4s or H100s, how much memory each has, or how they sit on the bus.

DRA swaps the counter for objects. Drivers publish ResourceSlice objects describing each device with attributes, you define a DeviceClass, and pods bind through a ResourceClaim or a ResourceClaimTemplate. AWS summarises the difference as "Rich attributes via ResourceSlice objects including GPU model, memory, driver version, and topology" against "Integer count of nvidia.com/gpu extended resources," and lists CEL-based filtering by model or memory as available on one side and "Not supported" on the other. If you have been encoding GPU model into node labels and nodeSelectors to get workloads onto the right silicon, that is the pain DRA fixes.

The migration posts tend to leave out one fact: the device plugin API is not deprecated. Its docs say "Feature state: Stable since Kubernetes v1.26" and nothing anywhere announces a removal release. There is no deadline here, which makes this unlike the real cutoffs we write about, such as Kubernetes extended support billing. Nobody is going to break your GPU nodes next quarter. Treat the migration as a capability purchase and ask what you are buying.

Your sharing mode decides whether you can move at all

We run this test first, and it stops most teams. If you share cards today, you are using static MIG profiles, MPS, or time-slicing through the device plugin, and all three are generally available there. The AWS comparison table puts "(GA) Static MIG, MPS, time-slicing" against "(Alpha) Dynamic MIG, MPS, time-slicing" for the DRA driver, and the surrounding text is direct about it: "Dynamic MIG and time-slicing are in alpha state in the NVIDIA DRA driver." The same page recommends that teams using MIG or time-slicing stay on "static configuration with the DRA driver or using the NVIDIA device plugin." NVIDIA's own guide is a shade softer on one point: a shared DRA claim can time-slice a card without any gate, but custom time-slice intervals need the alpha TimeSlicingSettings gate, and dynamic MIG is alpha outright.

So a team that shares GPUs and migrates in full this quarter moves a production dependency from GA to alpha. Do not do that. Either run whole-card allocation on DRA and leave shared nodes on the device plugin, or wait.

The alpha gates also fight each other. NVIDIA documents that DynamicMIG "conflicts with PassthroughSupport, NVMLDeviceHealthCheck, and MPSSupport." Picking dynamic MIG costs you the device health check in the same move, and the install guide says that neither the gRPC probe nor DCGM reports DRA allocation health today. You would be running partitioned cards with less visibility than you have now.

Upstream is closing this gap. Partitionable devices reached beta in v1.36, described in the v1.36 DRA update as "native DRA support for dynamically carving physical hardware into smaller, logical instances (such as Multi-Instance GPUs) based on workload demands." Beta upstream with an alpha driver gate on top is still a poor home for a shared inference fleet. Ask again after the gate defaults change.

On EKS the blocker is your compute type

AWS draws the line at how nodes get provisioned, and it is a hard line. From the EKS device management guide: "The NVIDIA DRA driver is not currently supported with EKS Auto Mode. Use the NVIDIA device plugin with EKS Auto Mode, or with Karpenter when you use dynamic capacity provisioning."

Read that against how you run GPUs. Auto Mode means the answer is no today, whatever version you are on. Karpenter is conditional: "Karpenter supports DRA with both dynamic and static capacity provisioning as of Karpenter version 1.15, though dynamic provisioning requires enabling Karpenter feature gates." Managed node groups and self-managed nodes are the clean paths.

Two node-level details will bite you regardless of provisioner. AWS is unambiguous on the first: "When using the NVIDIA DRA driver for GPU device management, do not deploy it alongside the NVIDIA device plugin on the same node. Doing so can cause silent oversubscription of the underlying devices to multiple pods on the same node." You get two pods on one card with no error anywhere, and you find out from garbage results or an out-of-memory kill.

The second catches Bottlerocket users. The EKS-optimized Bottlerocket NVIDIA variants ship the device plugin on by default, so you disable it in node user data:

# Bottlerocket GPU node user data. Requires Bottlerocket 1.63.0 or later;
# on earlier versions the built-in device plugin cannot be disabled.
[settings.kubelet-device-plugins.nvidia]
enabled = false

That setting arrived in Bottlerocket 1.63.0. If your AMI is older, you upgrade the AMI before you touch DRA, because there is no other way to get the built-in plugin out of the way.

On GKE the blocker is the node pool spec

Google's floor is higher and the shape of the work is different. The GKE DRA setup guide requires that you "Have a GKE Standard cluster that runs version 1.35 or later," and the guide is written for Standard without a word about Autopilot. If your GPU workloads run on Autopilot, treat this migration as out of scope until Google documents a path.

On Standard you give up managed drivers and take on three labels. You pass gpu-driver-version=disabled in the --accelerator flag and install the NVIDIA driver yourself. You add gke-no-default-nvidia-gpu-device-plugin=true to keep GKE's own plugin off the pool, nvidia.com/gpu.present=true so the driver DaemonSet lands, and cloud.google.com/gke-nvidia-gpu-dra-driver=true, because, in the guide's words, "The cluster autoscaler uses this node label to identify nodes that run the DRA driver for GPUs." Miss the third one and your DRA pool stops scaling, which looks like a scheduler problem for a day before anyone checks labels.

We would not attempt this on your existing GPU pool. Build a new pool with the labels and the disabled driver install, and leave the old one serving traffic.

Keep nvidia.com/gpu working while you move

The bridge exists and few teams use it. Extended resource support reached beta in v1.36 and graduated to stable in v1.37, per the Kubernetes feature gates reference, and upstream states the intent plainly: it "allows for a gradual transition to DRA, meaning cluster operators can migrate clusters to DRA but let application developers adopt the ResourceClaim API on their own schedule." The DRAExtendedResource gate is on by default from v1.36.0, and NVIDIA's guide says earlier versions need it enabled manually.

One detail costs an afternoon if you miss it. NVIDIA's managed gpu.nvidia.com DeviceClass "does not set spec.extendedResourceName to nvidia.com/gpu." The legacy name needs a DeviceClass of your own:

# Illustrative. Read the driver name off your own cluster before applying:
# kubectl get resourceslice -o jsonpath='{.items[*].spec.driver}'
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: legacy-nvidia-gpu
spec:
  extendedResourceName: nvidia.com/gpu
  selectors:
  - cel:
      expression: device.driver == "gpu.nvidia.com"

With that in place, a pod still asking for nvidia.com/gpu: 1 gets satisfied by the DRA driver, and teams that want attribute selection move to a ResourceClaimTemplate when they are ready. This is the single highest-value thing on the list, because it decouples the cluster change from every application team's backlog.

Two gaps to plan around. The upstream concept page says the scheduler "doesn't support preemption for DRA resources," so if you rely on priority classes to evict low-priority GPU jobs for training runs, that mechanism stops working for anything allocated through a claim. And DRA allocation health is not reported by the gRPC probe or DCGM, with NVMLDeviceHealthCheck still alpha and off by default.

The migration order we would run

Treat this as node pool work rather than a Helm upgrade. NVIDIA's GPUCluster workflow is documented for new installations, and the guide states that "An in-place migration from a standalone DRA driver Helm release or from ClusterPolicy to GPUCluster is not supported." Its own advice when a ClusterPolicy already exists is to use a new cluster. On most fleets a new node pool is enough, but there is no upgrade command here.

1. Answer the go or no-go. Auto Mode, Autopilot, Karpenter dynamic capacity, or any production MIG and time-slicing means no for now. Write down the date you will re-check.
2. Get to the floors on your existing pool: v1.34.2 or later, driver 580 or later, CDI configured, Bottlerocket 1.63.0 or later if that is your AMI.
3. Stand up a separate GPU node pool with the DRA driver and no device plugin on those nodes. Keep the old pool taking traffic.
4. Add the legacy DeviceClass so nvidia.com/gpu requests keep working on the new pool.
5. Move one non-critical workload, then one that benefits from attribute selection, and confirm ResourceSlice objects and claims look right before anything else follows.
6. Replace your priority-based GPU preemption with queue admission, or accept that it no longer applies.

Our view in one line: migrate for attribute-based selection and topology awareness, which the device plugin cannot do at all, and leave sharing where it is, because the device plugin currently does that better. Agent and inference platforms get the most out of this, for the reasons in our post on sandboxing AI agents on Kubernetes, where picking the right card for an untrusted workload is part of the isolation design.

Which half of your pools should move

The decision is cheap to get wrong in both directions. Migrating a shared GPU fleet onto alpha driver gates buys you outages you will debug for a week. Staying put while your team hand-maintains nodeSelectors across four GPU generations wastes engineering time every sprint.

We do this work with platform teams: auditing which of your GPU pools qualify, building the parallel pool, writing the DeviceClass set and the claim templates, and leaving a runbook your on-call can follow. If you are holding a GPU fleet and an opinion you are not sure about, talk to us and we will start by sorting your pools into the half that can move now and the half that should wait.