Improve the data format in CRD(ComposabilityRequest)

[hase1128]

I propose two improvement for the data format.

Proposal1:
Some users don't need (or want) to specify “targetNode” every time

1. It is tedious to specify for each node when there are many servers.
2. User only needs to have the number of GPUs in the entire K8s cluster(number of GPUs per node is not important for this user)

To satisfy this demand, we might need to design something like this:

1. The user specifies the number of GPUs in the cluster itself. 
2. After that, the CDI Operator may need to actually determine the number of GPUs for each node

Of course, we will keep the existing functionality so that users can still directly specify the number of GPUs per node.

Proposal2:
Allows user to specify a variety of device types.
GPU, PCIeSSD, NIC, CXL Memory, etc.

[mgazz]

@hase1128 thank you for creating the issue.

I agree no Proposal2.

Regarding Proposal1, I agree that defining a targetNode is tedious. A couple of considerations on this.

User only needs to have the number of GPUs in the entire K8s cluster

In this case, does the user specify the number of GPUs in the whole cluster, or the number of GPUs that they require for the specific ComposabilityRequest?

Users might not even have visibility of the nodes or even the GPUs in the cluster. For this reason specifying the number of GPUs in the entire cluster becomes hard without knowledge of the current state.

(number of GPUs per node is not important for this user)
I agree this is one of the cases to be supported, but we still need to consider the case where a user might want a specific configuration. Without making the CRD too complicated we could consider the addition of a parameter that describes the constraint in how we allocate resources (colocated,distributed,...) with a default value that describes the use-case in Proposal1 (no constraints).

[hase1128]

@mgazz

I agree no Proposal2.

Did you want to write "I agree on Proposal2."?

In this case, does the user specify the number of GPUs in the whole cluster, or the number of GPUs that they require for the specific ComposabilityRequest?

I expected to specify the number of required GPUs for the entire cluster.

By the way, should I create a ComsabilityRequest for each node? Does it make sense to specify multiple target nodes in a single ComposabilityRequest and keep updating that single ComposablityRequest when the user wants to change the cluster configuration?

In cases like this, I was considering "user can also specify the number of GPUs for the entire cluster in a single ComposabliltyRequest."
If the user only specifies the number of GPUs for the entire cluster in the manifest, the spec for the number of GPUs for the entire cluster will be updated.(targetNode is not updated in spec) . Regarding status, targetNode information is updated.

If the user specifies the targetnode directly, the targetnode information is also updated in spec.

This is my current thinking, and we need to discuss.

Users might not even have visibility of the nodes or even the GPUs in the cluster. For this reason specifying the number of GPUs in the entire cluster becomes hard without knowledge of the current state.

Would it also be difficult for such a user to specify a targetNode? In other words, it is difficult to create a ComsablityRequest itself.

Such a user may simply request the required GPU in a device plugin or resource claim within the yaml of the application's pod.

[mgazz]

Did you want to write "I agree on Proposal2."?

Correct, sorry about that.

I expected to specify the number of required GPUs for the entire cluster.

We can surely take this use-case in consideration. My main worry is the case where two different users want to specify the number of GPUs for the whole cluster. In this case, the two requests might be in conflict and we would need to manage this scenario in the operator. Maybe allowing only one ComposabilityRequest that describes the GPUs in the entire cluster. And block any other creation of ComposabilityRequests.

By the way, should I create a ComsabilityRequest for each node? Does it make sense to specify multiple target nodes in a single ComposabilityRequest and keep updating that single ComposablityRequest when the user wants to change the cluster configuration?

The ComposabilityRequest is created by the User/Service and in an ideal situation they would not specify any targetNode. (They might not know or care about the underlining composable infrastructure or where the device is attached as long as it is usable.) Setting the targetNode was to simplify a first implementation and for demonstration purposes.

In your opinion, and considering your scenarios, what would users/services would like/need to specify in the ComposabilityRequest?

The idea of having a ComposabilityRequest for the whole cluster might make sense if only a single service updates it. Like for instance a scheduler plugin. In this case the user has no knowledge of the management of composable resources.

If we allow K8s cluster's users to create and modify ComposabilityRequests, having a single ComposabilityRequest means that in the operator we will have to manage conflicts on concurrent updates on the same resource from multiple users. This is a design choice that might suit us, we just need to understand the implications.

Would it also be difficult for such a user to specify a targetNode? In other words, it is difficult to create a ComsablityRequest itself.

It would be difficult if the targetNode remains a mandatory field of the ComposabilityRequest. The ComposabilityRequest, should contain the devices that the user desires. The way I see it. moving away from just a PoC, the targetNode should be optional.

Such a user may simply request the required GPU in a device plugin or resource claim within the yaml of the application's pod.

Yes, a ResourceClaim could generate a ComposabilityRequest that describes the resources needed. Or the operator itself could process the ResourceClaim and connect the devices to fulfill the claim.

[hase1128]

My main worry is the case where two different users want to specify the number of GPUs for the whole cluster. In this case, the two requests might be in conflict and we would need to manage this scenario in the operator. Maybe allowing only one ComposabilityRequest that describes the GPUs in the entire cluster. And block any other creation of ComposabilityRequests.

Yes, we need to consider such feacher.

In your opinion, and considering your scenarios, what would users/services would like/need to specify in the ComposabilityRequest?

The idea of having a ComposabilityRequest for the whole cluster might make sense if only a single service updates it. Like for instance a scheduler plugin. In this case the user has no knowledge of the management of composable resources.

If we allow K8s cluster's users to create and modify ComposabilityRequests, having a single ComposabilityRequest means that in the operator we will have to manage conflicts on concurrent updates on the same resource from multiple users. This is a design choice that might suit us, we just need to understand the implications.

Below is a user scenario.
A shopping website performs AI inference (using GPUs) to recommend products to end users.
If a shopping event is scheduled for a specific date and time and it is known in advance that many GPUs will be required, the K8s administrator will manually update the ComposablityRequest. In that case, the K8s administrator will specify the number of GPUs required across the cluster.

Below is a service scenario.
Security system performs AI inference (using GPUs) to identify suspicious people from image data of security camera.
The GPU load varies depending on the number of people. If Pods are increased by the HPA and there are no free GPUs, the service that manages the resources will automatically update the ComposablityRequest. In that case, it will specify the nodes on which the Pod can be deployed.

Since there are scenarios like this, we need to consider confliction case. Thank you for your good point.

It would be difficult if the targetNode remains a mandatory field of the ComposabilityRequest. The ComposabilityRequest, should contain the devices that the user desires. The way I see it. moving away from just a PoC, the targetNode should be optional.

Yes, I think targetNode should be optional.