Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

【XVERSE-7B-chat WebDemo 部署】报错 torch.cuda.OutOfMemoryError: CUDA out of memory. #113

Open
hgsw opened this issue May 13, 2024 · 2 comments
Open

【XVERSE-7B-chat WebDemo 部署】报错 torch.cuda.OutOfMemoryError: CUDA out of memory. #113

hgsw opened this issue May 13, 2024 · 2 comments

Comments

@hgsw
Copy link

hgsw commented May 13, 2024

你好，我在 XVERSE-7B-chat WebDemo 部署的模块中运行chatBot.py文件连续体验十几轮问答后，就开始报存显存溢出了，我不知这是为什么，chatBot.py代码里我没有明显看到和显存相关的设置。
报错信息如下：
图片

此外我发现随着是问答轮数的增加，显存也是在逐渐增加的，chatBot.py文件刚运行时候大概子啊14.6G左右，报错时的显存占用大概在18G，截图如下：
图片
@KMnO4-zx
Copy link
Contributor

可能是超出对话长度了

@hgsw
Copy link
Author

hgsw commented May 20, 2024

也就是说不是累积造成的？而是某一轮的对话，文本太长导致的吗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hgsw @KMnO4-zx