Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Listening thausands messages/sec #35

Open
rst630 opened this issue Dec 10, 2021 · 4 comments
Open

Listening thausands messages/sec #35

rst630 opened this issue Dec 10, 2021 · 4 comments
Labels

Comments

@rst630
Copy link

rst630 commented Dec 10, 2021

I listening binance socket and 100 pairs have about 14k/msg/s. When I doing blocking operations when recieving messages (insert to db, calculating indicators etc.) After 10-20min of listening queue of socket messages have huge backlog and server can't going faster.
Then socket client crashes w/o any errors.

How to deal with it?
How I know how much msgs pending to proceed?
I need to listen much more then 14k/s x10 more. I think better hardware does't resolve problem - because CPU is't 100% when client crashes

@rst630 rst630 changed the title Listening thausands messeages/sec Listening thausands messages/sec Dec 10, 2021
@kelunik
Copy link
Member

kelunik commented Dec 10, 2021

The best thing you can do is getting rid of blocking calls, e.g by using amphp/mysql or amphp/postgres instead of your current blocking DB client.

@kelunik kelunik closed this as completed Dec 10, 2021
@kelunik kelunik reopened this Dec 10, 2021
@kelunik
Copy link
Member

kelunik commented Dec 10, 2021

Sorry, I didn't intend to close this, yet.

@rst630
Copy link
Author

rst630 commented Dec 10, 2021

does Amp\call will do the same if mysql operation in it's callback?

@kelunik
Copy link
Member

kelunik commented Dec 10, 2021

No, Amp\call won't magically make a blocking I/O call non-blocking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

2 participants