Cannot run under Windows (path contains invalid characters)

[AXYZE9]

Windows 10 22H2, Node 18

Running my own config or sample ones always causes this issue:

yarn start -p simple.json
Sender::constructor
Sender::init
Webhook::constructor
Webhook not configured; if you want to use a webhook, set the WEBHOOK_URL environment variable or provide the webhook_url option in the config
DefaultScraper::constructor
Sender::updateSettings
C:\Users\AXYZE\Downloads\scrapix-0.1.9\scrapix-0.1.9\node_modules\fs-extra\lib\mkdirs\utils.js:16
      const error = new Error(`Path contains invalid characters: ${pth}`)
                    ^

Error: Path contains invalid characters: C:\Users\AXYZE\Downloads\scrapix-0.1.9\scrapix-0.1.9\storage\request_queues\["https:\platform.openai.com\docs"]
    at checkPath (C:\Users\AXYZE\Downloads\scrapix-0.1.9\scrapix-0.1.9\node_modules\fs-extra\lib\mkdirs\utils.js:16:21)
    at module.exports.makeDir (C:\Users\AXYZE\Downloads\scrapix-0.1.9\scrapix-0.1.9\node_modules\fs-extra\lib\mkdirs\make-dir.js:12:3)
    at Object.defineProperty.value (C:\Users\AXYZE\Downloads\scrapix-0.1.9\scrapix-0.1.9\node_modules\universalify\index.js:21:45)
    at RequestQueueFileSystemEntry.update (C:\Users\AXYZE\Downloads\scrapix-0.1.9\scrapix-0.1.9\node_modules\@crawlee\memory-storage\fs\request-queue\fs.js:61:44)
    at async RequestQueueClient.batchAddRequests (C:\Users\AXYZE\Downloads\scrapix-0.1.9\scrapix-0.1.9\node_modules\@crawlee\memory-storage\resource-clients\request-queue.js:322:13)
    at async RequestQueue.addRequests (C:\Users\AXYZE\Downloads\scrapix-0.1.9\scrapix-0.1.9\node_modules\@crawlee\core\storages\request_queue.js:371:28)
    at async Crawler.run (C:\Users\AXYZE\Downloads\scrapix-0.1.9\scrapix-0.1.9\dist\src\crawler.js:40:9)
    at async C:\Users\AXYZE\Downloads\scrapix-0.1.9\scrapix-0.1.9\dist\src\bin\index.js:56:5 {
  code: 'EINVAL'
}

As we can see it throws error because it wants to name folder like full URL "storage\request_queues["https:\platform.openai.com\docs"]" which of course won't work.
It looks like it is transforming "start_urls": ["https://platform.openai.com/docs"] from sample config to ["https:\platform.openai.com\docs"].

None of simple solutions like removing "https://", changing that to string by removing square brackets work.

Issue should be fixed by either transforming URL differently, hardcoding some kind of value or getting name from different parameter, maybe from "meilisearch_index_uid" parameter.

I'm trying to modify 40 line of "dist/src/crawler.js" file in order to transform that value, but so far I'm not able to
await requestQueue.addRequests(this.urls.map((url) => ({ url })));

[curquiza]

Hello @AXYZE9
thanks for the report and sorry for the trouble. Any PRs are welcome for this ❤️