Replies: 2 comments
-
Nope, that's not possible, the problem is that crawlee is processing requests in parallel, that's why we always open new pages. Only way you could achieve this is by having a single request in the queue and changing the pages inside the handler probably (which sounds like a bad idea in the first place, you'd have to handle everything yourself and you'd loose parallel processing completely). May I ask what is that target site? |
Beta Was this translation helpful? Give feedback.
-
@B4nan unfortunately, cant mention the website, but I can say that the website has strict cloudflare restrictions, so when we open the first page, we solve the captcha manually then the website works for that tab, if a new tab is opened cloudflare shows up again. This is also the case, when we use a proxy, the first page opens without cloudflare but the new tab even with proxy shows CF page. I am thinking of creating a wrapper around |
Beta Was this translation helpful? Give feedback.
-
Some websites have advanced crawl protection and they show a captcha for a new tab open, however if the page is loaded on the same page, it doesnt complain. Is there a way in crawlee to reuse the same tab instead of opening a new tab for each request?
Beta Was this translation helpful? Give feedback.
All reactions