Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

performance issue when using request_multiple #283

Open
FlyingBlazer opened this issue Jun 12, 2017 · 2 comments · May be fixed by #284
Open

performance issue when using request_multiple #283

FlyingBlazer opened this issue Jun 12, 2017 · 2 comments · May be fixed by #284

Comments

@FlyingBlazer
Copy link

The following code will cause serious performance problem:

do {
    $status = curl_multi_exec($multihandle, $active);
}
while ($status === CURLM_CALL_MULTI_PERFORM);

it will be much better if changed as follow

do {
    $status = curl_multi_exec($multihandle, $active);
    usleep(1000);
}
while ($status === CURLM_CALL_MULTI_PERFORM);
@TysonAndre
Copy link
Contributor

https://curl.haxx.se/libcurl/c/libcurl-errors.html#CURLMCALLMULTIPERFORM

CURLM_CALL_MULTI_PERFORM (-1)

This is not really an error. It means you should call curl_multi_perform again without doing select() or similar in between. Before version 7.20.0 this could be returned by curl_multi_perform, but in later versions this return code is never used.

From the PHP implementation of curl_multi_exec, it seems to call https://curl.haxx.se/libcurl/c/curl_multi_perform.html , which says if it returns CURLM_CALL_MULTI_PERFORM

Also seen:

Most applications will use curl_multi_fdset to get the multi_handle's file descriptors, and curl_multi_timeout to get a suitable timeout period, then it'll wait for action on the file descriptors using select(3). As soon as one or more file descriptor is ready, curl_multi_perform gets called.

Also seen

https://secure.php.net/manual/en/function.curl-multi-select.php

On success, returns the number of descriptors contained in the descriptor sets. This may be 0 if there was no activity on any of the descriptors. On failure, this function will return -1 on a select failure (from the underlying select system call).

TysonAndre added a commit to TysonAndre/Requests that referenced this issue Jun 13, 2017
@TysonAndre
Fixes WordPress#283 : Fixes CPU busy loop when using request_multiple.
46926f4 
This will require more testing across various libcurl versions.
I doubt that the timeout is necessary for curl_multi_select
(calls select() if it can),
but leaving in one just in case of bugs, so that it will end.
- Haven't thoroughly checked for relevant libcurl bugs.

Asynchronously wait for events with a short timeout if CURLM_CALL_MULTI_PERFORM
TysonAndre added a commit to TysonAndre/Requests that referenced this issue Jun 13, 2017
@TysonAndre
Fixes WordPress#283 : Fixes CPU busy loop when using request_multiple.
7234c79 
This will require more testing across various libcurl versions.
I doubt that the timeout is necessary for curl_multi_select
(calls select() if it can),
but leaving in one just in case of bugs, so that it will end.
- Haven't thoroughly checked for relevant libcurl bugs.

Asynchronously wait for events with a short timeout if CURLM_CALL_MULTI_PERFORM
TysonAndre added a commit to TysonAndre/Requests that referenced this issue Jun 14, 2017
@TysonAndre
Fixes WordPress#283 : Fixes CPU busy loop when using request_multiple.
c7d8ee4 
This may be the same as WordPress#110

This will require more testing across various libcurl versions.
I doubt that the timeout is necessary for curl_multi_select
(calls select() if it can),
but leaving in one just in case of bugs, so that it will end.
- Haven't thoroughly checked for relevant libcurl bugs.

Asynchronously wait for events with a short timeout if CURLM_CALL_MULTI_PERFORM
fails.
@soulseekah
Copy link
Contributor

Related #266

TysonAndre added a commit to TysonAndre/Requests that referenced this issue Oct 23, 2019
@TysonAndre
Fixes WordPress#283 : Fixes CPU busy loop when using request_multiple.
437cecf 
See WordPress#284 for background information

This will require more testing across various libcurl versions.
I doubt that the timeout is necessary for curl_multi_select
(calls select() if it can),
but leaving in one just in case of bugs, so that it will end.
- Haven't thoroughly checked for relevant libcurl bugs.

Asynchronously wait for events with a short timeout if CURLM_CALL_MULTI_PERFORM
fails. Use shorter timeouts when the total time elapsed so far is
shorter. Make the largest possible manual usleep 2ms.
@jrfnl jrfnl mentioned this issue Aug 13, 2021
Open
@jrfnl jrfnl linked a pull request Nov 6, 2023 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixes #283 : Fixes CPU busy loop(s) seen in request_multiple. TysonAndre/Requests
3 participants
@soulseekah @TysonAndre @FlyingBlazer