-
Hi all, I'm putting this here since there is no nodriver discussions section. I'm trying to get the browser setup with an authenticated proxy but chrome does not support credentials via browser args. I found this, where the answer from Arkadiy Bolotov is interesting. There are the same cdp methods as in his post available in Network though they are deprecated for those in Fetch. Here is my current attempt I put together this afternoon, which does intercept the requests however no authRequired events are fired as the documentation suggests.
|
Beta Was this translation helpful? Give feedback.
Answered by
WindBora
Mar 29, 2024
Replies: 1 comment 4 replies
-
Hi Here 2 problems arises:
Solutions:
Example of working code based of of yours: class Scraper:
main_tab: uc.Tab
def __init__(self):
uc.loop().run_until_complete(self.run())
async def run(self):
browser = await uc.start(
browser_args=[f"--proxy-server={PROXY}"],
)
self.main_tab = await browser.get("draft:,")
self.main_tab.add_handler(uc.cdp.fetch.RequestPaused, self.req_paused)
self.main_tab.add_handler(
uc.cdp.fetch.AuthRequired, self.auth_challenge_handler
)
await self.main_tab.send(uc.cdp.fetch.enable(handle_auth_requests=True))
page = await browser.get("https://www.whatismyip.com/")
await asyncio.sleep(6000)
async def auth_challenge_handler(self, event: uc.cdp.fetch.AuthRequired):
# Split the credentials
# Respond to the authentication challenge
asyncio.create_task(
self.main_tab.send(
uc.cdp.fetch.continue_with_auth(
request_id=event.request_id,
auth_challenge_response=uc.cdp.fetch.AuthChallengeResponse(
response="ProvideCredentials",
username=USERNAME,
password=PASSWORD,
),
)
)
)
async def req_paused(self, event: uc.cdp.fetch.RequestPaused):
asyncio.create_task(
self.main_tab.send(
uc.cdp.fetch.continue_request(request_id=event.request_id)
)
)
if __name__ == "__main__":
Scraper() |
Beta Was this translation helpful? Give feedback.
4 replies
Answer selected by
strukiii
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi
Here 2 problems arises:
Solutions:
fire-and-forget
method e.g. do not await for sending response onreq_paused
orauth_required
. Just wrap the call intoasyncio.create_task
Example of working code based of of yours: