Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 172 additions & 0 deletions content/en/blog/_posts/2026/dra-136-update.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
---
layout: blog
title: "Kubernetes v1.36: More Drivers, New Features, and the Next Era of DRA"
slug: dra-136-updates
draft: true
date: XXXX-XX-XX
author: >
The DRA team
---

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it will be great to include some information on adoption and gaps still left comparing to Device Plugin. Maybe a couple of words on available DRA drivers. So end users may make sense of this blog post.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a little section about the availability of drivers. I'm a little worried that by mentioning some drivers here, we might be forgetting others that also should be included. But I can ask in the device management chat if someone knows about other drivers that should be included.

I need to think a bit more about the gaps vs Device Plugin.

Dynamic Resource Allocation (DRA) has fundamentally changed how we handle hardware
accelerators and specialized resources in Kubernetes. In the v1.36 release, DRA
continues to mature, bringing a wave of feature graduations, critical usability
improvements, and new capabilities that extends the flexibility of DRA to native
resources like memory and CPU, and support for ResourceClaims in PodGroups.

We have also seen significant momentum in driver availability. Both the
[NVIDIA GPU](https://github.com/NVIDIA/k8s-dra-driver-gpu)
and Google TPU DRA drivers are being transferred to the Kubernetes project, joining the
[DRANET](https://github.com/kubernetes-sigs/dranet)
driver that was added last year.

Whether you are managing massive fleets of GPUs, need better handling of failures,
or simply looking for better ways to define resource fallback options, the upgrades
to DRA in 1.36 have something for you. Let's dive into the new features and graduations!

## Feature graduations

The community has been hard at work stabilizing core DRA concepts. In Kubernetes 1.36,
several highly anticipated features have graduated to Beta and Stable.

**Prioritized List (Stable)**

Hardware heterogeneity is a reality in most clusters. With the
[Prioritized List](https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#prioritized-list)
feature, you can confidently define fallback preferences when requesting
devices. Instead of hardcoding a request for a specific device model, you can specify an
ordered list of preferences (e.g., "Give me an H100, but if none are available, fall back
to an A100"). The scheduler will evaluate these requests in order, drastically improving
scheduling flexibility and cluster utilization.

**Extended Resource Support (Beta)**

As DRA becomes the standard for resource allocation, bridging the gap with legacy systems
is crucial. The DRA
[Extended Resource](https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#extended-resource)
feature allows users to request resources via traditional extended resources on a Pod.
This allows for a gradual transition to DRA, meaning application developers and
operators are not forced to immediately migrate their workloads to the ResourceClaim
API.

**Partitionable Devices (Beta)**

Hardware accelerators are powerful, and sometimes a single workload doesn't need an
entire device. The
[Partitionable Devices](https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#partitionable-devices)
feature, provides native DRA support for carving physical hardware into smaller,
logical instances (such as Multi-Instance GPUs). This allows administrators to
safely and efficiently share expensive accelerators across multiple Pods.

**Device Taints (Beta)**

Just as you can taint a Kubernetes Node, you can now apply taints directly to specific DRA
devices.
[Device Taints and Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#device-taints-and-tolerations)
empower cluster administrators to manage hardware more effectively. You can taint faulty
devices to prevent them from being allocated to standard claims, or reserve specific hardware
for dedicated teams, specialized workloads, and experiments. Ultimately, only Pods with
matching tolerations are permitted to claim these tainted devices.

**Device Binding Conditions (Beta)**

To improve scheduling reliability, the Kubernetes scheduler can now use the
[Binding Conditions](https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#device-taints-and-tolerations)
feature to delay committing a Pod to a Node until its required external resources—such as attachable
devices or FPGAs—are fully prepared. By explicitly modeling resource readiness, this
prevents premature assignments that can lead to Pod failures, ensuring a much more robust
and predictable deployment process.
Comment on lines +71 to +78
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Device Binding Conditions part looks good to me.


Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to include kubernetes/enhancements#4680 in the feature blog.

ref: docs PR is #54420

Suggested change
**Resource Health Status (Beta)**
Knowing when a device has failed or become unhealthy is critical for
workloads running on specialized hardware. With
[Resource Health Status](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#device-health-monitoring),
Kubernetes now exposes device health information directly in the Pod
Status through the `allocatedResourcesStatus` field. When a DRA driver
detects that an allocated device has become unhealthy, it reports this
back to the kubelet, which surfaces it in each container's status.
In 1.36, the feature graduates to beta (enabled by default) and adds
an optional `message` field providing human readable context about the
health status, such as error details or failure reasons. DRA drivers
can also configure per device health check timeouts, allowing different
hardware types to use appropriate timeout values based on their
health reporting characteristics. This gives users and controllers
crucial visibility to quickly identify and react to hardware failures.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I've added your proposal for now, but do you think we can shorten it a bit and make it just one paragraph? There is a large number of features and we don't want the blog post to be too long. Focus just on the benefits of this feature and what it enables and leave the details to the DRA docs which we link to. Also, including that it is graduating to beta in 1.36 is already given from the context.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I forgot to add this in the first draft, it is of course something we should include in the blog.

**Resource Health Status (Beta)**

Knowing when a device has failed or become unhealthy is critical for workloads running on
specialized hardware. With
[Resource Health Status](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#device-health-monitoring),
Kubernetes now exposes device health information directly in the Pod Status through the
`allocatedResourcesStatus` field. When a DRA driver detects that an allocated device
has become unhealthy, it reports this back to the kubelet, which surfaces it in each
container's status.

In 1.36, the feature graduates to beta (enabled by default) and adds an optional `message`
field providing human readable context about the health status, such as error details or
failure reasons. DRA drivers can also configure per device health check timeouts,
allowing different hardware types to use appropriate timeout values based on their
health reporting characteristics. This gives users and controllers crucial visibility
to quickly identify and react to hardware failures.

## New Features

Beyond stabilizing existing capabilities, v1.36 introduces foundational new features
that expand what DRA can do.

**ResourceClaim Support for Workloads**

To optimize large-scale AI/ML workloads that rely on strict topological scheduling, the
[ResourceClaim Support for Workloads](add_link_here)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section looks good, thanks!

Right now I'm anticipating the link to be /docs/concepts/scheduling-eviction/dynamic-resource-allocation/#workload-resource-claims. I'll follow up once #54596 merges.

feature enables Kubernetes to seamlessly manage shared resources across massive sets
of Pods. By associating ResourceClaims or ResourceClaimTemplates with PodGroups,
this feature eliminates previous scaling bottlenecks, such as the limit on the
number of pods that can share a claim, and removes the burden of manual claim
management from specialized orchestrators.

**DRA for Native Resources**

Why should DRA only be for external accelerators? In v1.36, we are introducing the first
iterations of using the DRA API to manage Kubernetes native resources (like CPU and
Memory). By bringing CPU and memory allocation under the DRA umbrella with the DRA
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I don't think this needs to be capitalized.

Suggested change
Memory). By bringing CPU and memory allocation under the DRA umbrella with the DRA
memory). By bringing CPU and memory allocation under the DRA umbrella with the DRA

[Native Resources](add_link_here)
feature, users can leverage DRA's advanced placement, NUMA-awareness, and prioritization
semantics for standard compute resources, paving the way for incredibly fine-grained
performance tuning.

**DRA Resource Availability Visibility**

One of the most requested features from cluster administrators has been better visibility
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to improve this section to mention the actual API name (ResourcePoolStatusRequest), the feature gate, and the alpha status — consistent with how other features in this post reference their API objects and maturity level.

ref: docs PR is #54456

Suggested change
One of the most requested features from cluster administrators has been better visibility
One of the most requested features from cluster administrators has been better visibility
into hardware capacity. The new
[ResourcePoolStatusRequest](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#resource-pool-status)
API (alpha, behind the `DRAResourcePoolStatus` feature gate) allows you to query
the availability of devices in DRA resource pools. By creating a
ResourcePoolStatusRequest object, you get a point-in-time snapshot of device counts
— total, allocated, available, and unavailable — for each pool managed by a given
driver. This enables better integration with dashboards and capacity planning tools.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added your suggestion, but made some small adjustments to make it similar to the other features mentioned. Let me know if you want to make some changes to it.

into hardware capacity. The new
[DRAResourcePoolStatus](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#resource-pool-status)
feature allows you to query the availability of devices in DRA resource pools. By creating a
`ResourcePoolStatusRequest` object, you get a point-in-time snapshot of device counts
— total, allocated, available, and unavailable — for each pool managed by a given
driver. This enables better integration with dashboards and capacity planning tools.

**List Types for Attributes**

With
[List Types for Attributes](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#list-type-attributes),
DRA can represent device attributes as typed lists (int, bool, string, and version), not
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit to match the names of the new fields:

Suggested change
DRA can represent device attributes as typed lists (int, bool, string, and version), not
DRA can represent device attributes as typed lists (`ints`, `bools`, `strings`, and `versions`), not

just scalar values. This helps model real hardware topology, such as devices that belong
to multiple PCIe roots or NUMA domains.

This feature also extends `ResourceClaim` constraint behavior to work naturally
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This shouldn't be formatted as code:

Suggested change
This feature also extends `ResourceClaim` constraint behavior to work naturally
This feature also extends ResourceClaim constraint behavior to work naturally

with both scalar and list values: `matchAttribute` now checks for a non-empty
intersection, and `distinctAttribute` checks for pairwise disjoint values.
It also introduces `includes()` function in DRA CEL, which lets device selectors keep working
more easily when an attribute changes between scalar and list representations.

**Device Allocation Ordering through Lexicographical Ordering**
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The section title feels a bit repetitive. Would something like 'Lexicographical Device Allocation' or 'Lexicographical Ordering for Device Allocation' be better?

The content looks good to me. Thank you!


The Kubernetes scheduler has been updated to evaluate devices using lexicographical
ordering based on resource pool and ResourceSlice names. This change empowers drivers
to proactively influence the scheduling process, leading to improved throughput and
more optimal scheduling decisions. To support this capability, the ResourceSlice
controller toolkit now automatically generates names that reflect the exact device
ordering specified by the driver author.

Copy link
Copy Markdown
Contributor

@everpeace everpeace Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to include kubernetes/enhancements#5491 if it's worth putting in the feature blog.

ref: docs PR is #54561

Suggested change
**List Types for Attributes**
With
[List Types for Attributes](/docs/concepts/scheduling-eviction/dynamic-resource-allocation/#list-type-attributes),
DRA can represent device attributes as typed lists (int, bool, string, and
version), not just scalar values. This helps model real hardware topology, such
as devices that belong to multiple PCIe roots or NUMA domains.
This feature also extends `ResourceClaim` constraint behavior to work naturally
with both scalar and list values: `matchAttribute` now checks for a non-empty
intersection, and `distinctAttribute` checks for pairwise disjoint values.
It also introduces `includes()` function in DRA CEL, which lets device selectors keep working
more easily when an attribute changes between scalar and list representations.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I forgot this one, it is definitely worth including. Added your suggestion.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to my comment on #54567 (comment), do you think we could make it a bit more focused on just the benefits of the feature and leave some of the details to the DRA documentation? And see if we can keep it to a single paragraph?

## What’s next?

This cycle introduced a wealth of new DRA features, and the momentum continues.
Our focus remains on progressing existing features toward beta and stable releases
while enhancing DRA's performance, scalability, and reliability. Additionally,
integrating DRA with Workload-Aware and Topology-Aware Scheduling will be a key
priority over the coming releases.


## Getting involved

A good starting point is joining the WG Device Management
[Slack channel](https://kubernetes.slack.com/archives/C0409NGC1TK) and
[meetings](https://docs.google.com/document/d/1qxI87VqGtgN7EAJlqVfxx86HGKEAc2A3SKru8nJHNkQ/edit?tab=t.0#heading=h.tgg8gganowxq),
which happen at US/EU and EU/APAC friendly time slots.

Not all enhancement ideas are tracked as issues yet, so come talk to us if you want to help or have some ideas yourself!
We have work to do at all levels, from difficult core changes to usability enhancements in kubectl, which could be picked up by newcomers.