Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Nvidia extension #2882

Draft
wants to merge 10 commits into
base: dev
Choose a base branch
from
Draft

Nvidia extension #2882

wants to merge 10 commits into from

Conversation

Realmbird
Copy link

Describe Your Changes

Fixes Issues

Self Checklist

  • Added relevant comments, esp in complex areas
  • Updated docs (for bug fixes / features)
  • Created issues for follow-up changes or refactoring needed

@Realmbird Realmbird marked this pull request as draft May 9, 2024 02:26
@Realmbird Realmbird mentioned this pull request May 9, 2024
Open
@hiro-v hiro-v requested a review from louis-jan May 9, 2024 02:41
@Realmbird
Copy link
Author

Currently has error with api so not 100% working
2024-05-08T20:29:43.223Z [SPECS]::OS Version: #21-Ubuntu SMP PREEMPT_DYNAMIC Wed Apr 3 14:54:32 UTC 2024
2024-05-08T20:29:43.223Z [SPECS]::OS Platform: linux
2024-05-08T20:29:43.224Z [SPECS]::OS Release: 6.5.0-1020-oem
2024-05-08T20:29:43.224Z [APP]::{"notify":true,"run_mode":"cpu","nvidia_driver":{"exist":false,"version":""},"cuda":{"exist":false,"version":""},"gpus":[],"gpu_highest_vram":"","gpus_in_use":[""],"is_initial":false,"vulkan":false}
2024-05-08T20:31:21.952Z [SPECS]::Version: 0.1.3
2024-05-08T20:31:21.954Z [SPECS]::Parallelism: 16
2024-05-08T20:31:21.954Z [SPECS]::Machine: x86_64
2024-05-08T20:31:21.955Z [SPECS]::Free Mem: 5712773120
2024-05-08T20:31:21.953Z [SPECS]::CPUs: [{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1554,"times":{"user":437700,"nice":210,"sys":144670,"idle":5604050,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1425,"times":{"user":442640,"nice":410,"sys":145430,"idle":5642180,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1393,"times":{"user":525600,"nice":440,"sys":174950,"idle":5498940,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1057,"times":{"user":344370,"nice":980,"sys":110540,"idle":5774290,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1572,"times":{"user":578500,"nice":270,"sys":212800,"idle":5373700,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1569,"times":{"user":349780,"nice":380,"sys":78640,"idle":5826690,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1106,"times":{"user":600050,"nice":2300,"sys":171860,"idle":5413480,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1565,"times":{"user":369180,"nice":80,"sys":77430,"idle":5806010,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1495,"times":{"user":381150,"nice":120,"sys":133730,"idle":5671520,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1491,"times":{"user":305630,"nice":230,"sys":101980,"idle":5820330,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1332,"times":{"user":253360,"nice":190,"sys":86110,"idle":5887660,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1610,"times":{"user":213020,"nice":140,"sys":78250,"idle":5804210,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1380,"times":{"user":183650,"nice":70,"sys":68400,"idle":5987400,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1303,"times":{"user":159840,"nice":110,"sys":60630,"idle":5985440,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":869,"times":{"user":135100,"nice":40,"sys":53880,"idle":6049800,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":400,"times":{"user":134800,"nice":50,"sys":82380,"idle":5989210,"irq":0}}]
2024-05-08T20:31:21.954Z [SPECS]::Endianness: LE
2024-05-08T20:31:21.955Z [SPECS]::Total Mem: 16453881856
2024-05-08T20:31:21.955Z [SPECS]::OS Version: #21-Ubuntu SMP PREEMPT_DYNAMIC Wed Apr 3 14:54:32 UTC 2024
2024-05-08T20:31:21.955Z [SPECS]::OS Platform: linux
2024-05-08T20:31:21.955Z [SPECS]::OS Release: 6.5.0-1020-oem
2024-05-08T20:31:21.956Z [APP]::{"notify":true,"run_mode":"cpu","nvidia_driver":{"exist":false,"version":""},"cuda":{"exist":false,"version":""},"gpus":[],"gpu_highest_vram":"","gpus_in_use":[""],"is_initial":false,"vulkan":false}
2024-05-08T20:33:18.601Z [SPECS]::Version: 0.1.3
2024-05-08T20:33:18.603Z [SPECS]::CPUs: [{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1200,"times":{"user":456710,"nice":210,"sys":147550,"idle":5685260,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1152,"times":{"user":457910,"nice":410,"sys":147460,"idle":5729540,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1043,"times":{"user":545950,"nice":440,"sys":177970,"idle":5579800,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":548,"times":{"user":361140,"nice":980,"sys":113890,"idle":5857830,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1199,"times":{"user":604740,"nice":280,"sys":216150,"idle":5446530,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1152,"times":{"user":363980,"nice":380,"sys":80460,"idle":5915900,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1152,"times":{"user":624850,"nice":2300,"sys":175120,"idle":5489580,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1191,"times":{"user":382580,"nice":80,"sys":79210,"idle":5895960,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1178,"times":{"user":398920,"nice":120,"sys":136300,"idle":5756060,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1152,"times":{"user":318760,"nice":230,"sys":104100,"idle":5910080,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1173,"times":{"user":264240,"nice":190,"sys":88090,"idle":5979450,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1300,"times":{"user":222250,"nice":140,"sys":79950,"idle":5894270,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1139,"times":{"user":191930,"nice":70,"sys":69950,"idle":6082250,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":981,"times":{"user":166800,"nice":110,"sys":62050,"idle":6081870,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1080,"times":{"user":140930,"nice":40,"sys":55320,"idle":6147460,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1200,"times":{"user":140130,"nice":50,"sys":84060,"idle":6086660,"irq":0}}]
2024-05-08T20:33:18.603Z [SPECS]::Machine: x86_64
2024-05-08T20:33:18.603Z [SPECS]::Endianness: LE
2024-05-08T20:33:18.604Z [SPECS]::Free Mem: 6521778176
2024-05-08T20:33:18.604Z [SPECS]::Parallelism: 16
2024-05-08T20:33:18.605Z [SPECS]::Total Mem: 16453881856
2024-05-08T20:33:18.605Z [SPECS]::OS Version: #21-Ubuntu SMP PREEMPT_DYNAMIC Wed Apr 3 14:54:32 UTC 2024
2024-05-08T20:33:18.605Z [SPECS]::OS Platform: linux
2024-05-08T20:33:18.605Z [SPECS]::OS Release: 6.5.0-1020-oem
2024-05-08T20:33:18.605Z [APP]::{"notify":true,"run_mode":"cpu","nvidia_driver":{"exist":false,"version":""},"cuda":{"exist":false,"version":""},"gpus":[],"gpu_highest_vram":"","gpus_in_use":[""],"is_initial":false,"vulkan":false}
2024-05-08T20:43:21.640Z [SPECS]::Version: 0.1.3
2024-05-08T20:43:21.643Z [SPECS]::Machine: x86_64
2024-05-08T20:43:21.642Z [SPECS]::CPUs: [{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1531,"times":{"user":584980,"nice":210,"sys":169520,"idle":6038470,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1516,"times":{"user":602250,"nice":410,"sys":164930,"idle":6075880,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1595,"times":{"user":683010,"nice":440,"sys":200950,"idle":5925530,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1585,"times":{"user":496600,"nice":980,"sys":133890,"idle":6210490,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1550,"times":{"user":753230,"nice":290,"sys":239340,"idle":5775130,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1591,"times":{"user":491780,"nice":380,"sys":96270,"idle":6283520,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1587,"times":{"user":780560,"nice":2300,"sys":198300,"idle":5815930,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":400,"times":{"user":523310,"nice":80,"sys":94390,"idle":6251480,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1502,"times":{"user":541080,"nice":120,"sys":160100,"idle":6097940,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1585,"times":{"user":449380,"nice":230,"sys":126300,"idle":6266100,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1537,"times":{"user":384940,"nice":190,"sys":109480,"idle":6345230,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1467,"times":{"user":335200,"nice":140,"sys":100300,"idle":6257740,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1437,"times":{"user":297710,"nice":90,"sys":89190,"idle":6467120,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1558,"times":{"user":267040,"nice":110,"sys":79440,"idle":6472750,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1213,"times":{"user":235600,"nice":60,"sys":72110,"idle":6545870,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":916,"times":{"user":228200,"nice":50,"sys":101510,"idle":6489390,"irq":0}}]
2024-05-08T20:43:21.643Z [SPECS]::Endianness: LE
2024-05-08T20:43:21.643Z [SPECS]::Parallelism: 16
2024-05-08T20:43:21.644Z [SPECS]::Total Mem: 16453881856
2024-05-08T20:43:21.644Z [SPECS]::Free Mem: 7386509312
2024-05-08T20:43:21.644Z [SPECS]::OS Version: #21-Ubuntu SMP PREEMPT_DYNAMIC Wed Apr 3 14:54:32 UTC 2024
2024-05-08T20:43:21.644Z [SPECS]::OS Platform: linux
2024-05-08T20:43:21.645Z [SPECS]::OS Release: 6.5.0-1020-oem
2024-05-08T20:43:21.645Z [APP]::{"notify":true,"run_mode":"cpu","nvidia_driver":{"exist":false,"version":""},"cuda":{"exist":false,"version":""},"gpus":[],"gpu_highest_vram":"","gpus_in_use":[""],"is_initial":false,"vulkan":false}
2024-05-08T20:52:45.693Z [NITRO]::Debug: Request to kill Nitro
2024-05-09T02:21:27.004Z [SPECS]::Version: 0.1.3
2024-05-09T02:21:27.007Z [SPECS]::Parallelism: 16
2024-05-09T02:21:27.007Z [SPECS]::Free Mem: 7490146304
2024-05-09T02:21:27.007Z [SPECS]::Total Mem: 16453873664
2024-05-09T02:21:27.007Z [SPECS]::OS Version: #21-Ubuntu SMP PREEMPT_DYNAMIC Wed Apr 3 14:54:32 UTC 2024
2024-05-09T02:21:27.005Z [SPECS]::CPUs: [{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1200,"times":{"user":220820,"nice":260,"sys":51320,"idle":1255730,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1200,"times":{"user":242720,"nice":160,"sys":44420,"idle":1250390,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1190,"times":{"user":246050,"nice":280,"sys":51440,"idle":1232990,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1200,"times":{"user":230600,"nice":190,"sys":39590,"idle":1270120,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1176,"times":{"user":227100,"nice":120,"sys":53070,"idle":1250470,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1195,"times":{"user":240450,"nice":160,"sys":40460,"idle":1260770,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1200,"times":{"user":257570,"nice":1690,"sys":70730,"idle":1198150,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":400,"times":{"user":190640,"nice":1830,"sys":40990,"idle":1309080,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1200,"times":{"user":241080,"nice":70,"sys":48160,"idle":1222510,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1200,"times":{"user":220180,"nice":90,"sys":43840,"idle":1275730,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1200,"times":{"user":201520,"nice":30,"sys":40660,"idle":1297770,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1200,"times":{"user":185680,"nice":90,"sys":40520,"idle":1262890,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1199,"times":{"user":172820,"nice":90,"sys":38320,"idle":1329700,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1215,"times":{"user":160550,"nice":50,"sys":36710,"idle":1341580,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1225,"times":{"user":146740,"nice":40,"sys":34590,"idle":1360470,"irq":0}},{"model":"13th Gen Intel(R) Core(TM) i5-1340P","speed":1181,"times":{"user":136090,"nice":50,"sys":37920,"idle":1362040,"irq":0}}]
2024-05-09T02:21:27.006Z [SPECS]::Endianness: LE
2024-05-09T02:21:27.006Z [SPECS]::Machine: x86_64
2024-05-09T02:21:27.008Z [SPECS]::OS Platform: linux
2024-05-09T02:21:27.008Z [SPECS]::OS Release: 6.5.0-1020-oem
2024-05-09T02:21:27.008Z [APP]::{"notify":true,"run_mode":"cpu","nvidia_driver":{"exist":false,"version":""},"cuda":{"exist":false,"version":""},"gpus":[],"gpu_highest_vram":"","gpus_in_use":[""],"is_initial":false,"vulkan":false}

@hiro-v
Copy link
Contributor

hiro-v commented May 9, 2024

Community user support thread in: https://discord.com/channels/1107178041848909847/1236435708881797211

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@louis-jan louis-jan Awaiting requested review from louis-jan

At least 1 approving review is required to merge this pull request.

Assignees

@Realmbird Realmbird

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

feat: support Nvidia NIMs remote API
2 participants
@Realmbird @hiro-v