KEP: Multi level Cluster Queues #1093
Conversation
|
Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
k8s-ci-robot
added
kind/feature
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: KunWuLuan The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
cncf-cla: yes
✅ Deploy Preview for kubernetes-sigs-kueue ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
k8s-ci-robot
added
the
needs-ok-to-test
|
Hi @KunWuLuan. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
k8s-ci-robot
added
the
size/L
|
Not sure plan A or B is better. Maintain a tree is easier when there are a lot of cluster queues in cluster, but it will cost more time when try to update the tree because we always have to update the whole tree. |
| - Allow users to define weights for different job of him/her. | ||
| ### Non-Goals | ||
|
|
||
| - Cohort will not be deprecated because Kueue is forward campatible. |
There was a problem hiding this comment.
Not deprecating (or significantly limiting) cohorts would allow a kind-of 2 dimensional hierarchy with hard to understand logic (and even more complex code).
| This proposal allow cluster admins to define a multi-level hierachy for cluster queues. Multi-level of cluster queues allow admins and users to define different dequeue policies for different node. This will allow Kueue to manage more levels of resources. | ||
| ## Motivation | ||
| Systems like Yarn allow creating a hierarchy of fair sharing, which allows modeling deeper organizational structures with fair-sharing. | ||
| Kueue currently supports three organizational levels: Cohort (models a business unit), ClusterQueue (models divisions within a business unit), namespace (models teams within a division). However fair-sharing is only supported at one level, within a cohort. This is not convinent if there are more than one level in an organization or if some users want to manage the resource consumption of his/her own jobs. |
There was a problem hiding this comment.
How would exactly fair sharing work on multi-level hierarchy? How is it different from having a flattened structure with multiplied/compounded weights (from root to leaf)?
There was a problem hiding this comment.
in every node in cluster queue tree, cluster queues will have their guaranteed resources( min weight or nominal resouces ), parent cluster queues' min resources must larger than the sum of their children's
if any node's usage is more than min, then workloads in it can be preempted by other node in tree
KunWuLuan
force-pushed
the
kep/multi-level-cluster-queue
branch
from
f6e1a95
to
9a4ceb3
Compare
| - Cohort will not be deprecated because Kueue is forward campatible. | ||
| ## Proposal | ||
| We will extend cluster queues to allow a cluster queue to be the parent of another cluster queue. And allow admins to define weights for all these cluster queues. | ||
| ### PlanA |
There was a problem hiding this comment.
How do you plan to handle cluster-to-cluster authentication?
|
/close |
|
@alculquicondor: Closed this PR. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
This proposal allow cluster admins to define a multi-level hierachy for cluster queues. Multi-level of cluster queues allow admins and users to define different dequeue policies for different node. This will allow Kueue to manage more levels of resources.
What type of PR is this?
/kind feature
What this PR does / why we need it:
Systems like Yarn allow creating a hierarchy of fair sharing, which allows modeling deeper organizational structures with fair-sharing.
Kueue currently supports three organizational levels: Cohort (models a business unit), ClusterQueue (models divisions within a business unit), namespace (models teams within a division). However fair-sharing is only supported at one level, within a cohort. This is not convinent if there are more than one level in an organization or if some users want to manage the resource consumption of his/her own jobs.
Special notes for your reviewer:
Does this PR introduce a user-facing change?