Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[k8s] Show currently available GPUs on Kubernetes cluster #2839

Closed
romilbhardwaj opened this issue Dec 5, 2023 · 4 comments · Fixed by #3499
Closed

[k8s] Show currently available GPUs on Kubernetes cluster #2839

romilbhardwaj opened this issue Dec 5, 2023 · 4 comments · Fixed by #3499
Labels
k8s Kubernetes related items

Comments

@romilbhardwaj
Copy link
Collaborator

User requested - in addition to showing list of supported GPUs, they would like to see realtime utilization for GPUs so they can know how many GPUs they can launch. Kubernetes dashboard is nice, but doesn't show live GPU usage.
@romilbhardwaj romilbhardwaj added the k8s Kubernetes related items label Dec 5, 2023
@gmittal
Copy link
Collaborator

gmittal commented Dec 5, 2023

following

@michaelzhiluo
Copy link
Collaborator

Potentially misunderstanding this here - does the user mean like 5/8 gpus are used on the cluster (where 5 gpus are occupied by other pods)?

In this case, it can be done by dynamically fetching node.status.allocatable_capacity in kubectl get nodes for the nvidia.com/gpu field.

@xzrderek xzrderek mentioned this issue Jan 20, 2024
Closed
5 tasks
@github-actions GitHub Actions
Copy link

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

@github-actions github-actions bot added the Stale label Apr 14, 2024
@romilbhardwaj
Copy link
Collaborator Author

Bumping this up

@romilbhardwaj romilbhardwaj mentioned this issue Apr 30, 2024
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
k8s Kubernetes related items
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[k8s] Realtime GPU availability of kubernetes cluster in sky show-gpus skypilot-org/skypilot
3 participants
@gmittal @romilbhardwaj @michaelzhiluo