We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
2024-05-15T16:10:20.104786Z INFO text_generation_launcher: Runtime environment: Target: x86_64-unknown-linux-gnu Cargo version: 1.75.0 Commit sha: bf700e7eef4771f280c19dbc7270c8c7c20efbbc Docker label: N/A nvidia-smi: Wed May 15 16:10:19 2024 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 545.23.08 Driver Version: 545.23.08 CUDA Version: 12.3 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | | 0% 34C P0 60W / 300W | 19020MiB / 23028MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| | 0 N/A N/A 4235 C ...untu/miniconda3/envs/tgi/bin/python 19008MiB | +---------------------------------------------------------------------------------------+ ### Information - [ ] Docker - [ ] The CLI directly ### Tasks - [X] An officially supported command - [ ] My own modifications ### Reproduction 1. send a request to `chat_completions()` 2. increments request_total_count by 2 ### Expected behavior `chat_completions()` itself should increment the counter by 1. its subsequent func calls (within chat_completions()) shouldn't increment the counter `chat_completions()` calls functions `generate` or `generate_stream_internal` depending on if steam is enabled and both these functions further increments the counter, hence incrementing the counter by 2 for every request
The text was updated successfully, but these errors were encountered:
No branches or pull requests
System Info
The text was updated successfully, but these errors were encountered: