Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sourcery plugin in VSCode uses too much process #405

Open
3 tasks done
PhunkyBob opened this issue Dec 2, 2023 · 5 comments
Open
3 tasks done

Sourcery plugin in VSCode uses too much process #405

PhunkyBob opened this issue Dec 2, 2023 · 5 comments
Labels
bug Something isn't working

Comments

@PhunkyBob
Copy link

Checklist

Description

In VSCode, Sourcery is always scanning my code, even if I don't touch it.
It uses a lot of processor time.
So much process that the others features of the IDE are not working well (Pylance, ...).
Sometimes I type a variable and my IDE takes 10 seconds to understand that the variable exists!

If I look in the VSCode notification, I can see Sourcery is scanning 10 times the same file.
OOrbNoABcz

Code snippet that reproduces issue

No particular code.
It do so on every Python script.

Debug Information

IDE Version:
VSCode
Version: 1.84.2 (user setup)
Commit: 1a5daa3a0231a0fbba4f14db7ec463cf99d7768e
Date: 2023-11-09T10:51:52.184Z
Electron: 25.9.2
ElectronBuildId: 24603566
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Windows_NT x64 10.0.19045

Sourcery Version:
v1.14.0

Operating system and Version:
Windows 10

@PhunkyBob PhunkyBob added the bug Something isn't working label Dec 2, 2023
@ruancomelli
Copy link
Contributor

Hello, @PhunkyBob! Thanks for reporting this issue.

This is definitely a strange bug. If I understand correctly:

  1. Sourcery does this for every Python script in your project
  2. You don't even need to open the file for Sourcery to start scanning it

Is this correct?

If so, can you please send your Sourcery log file to [email protected]?

@PhunkyBob
Copy link
Author

Sourcery does this for every Python script in your project

Not every script.
At this time, it scans in loop 2 of my files.

You don't even need to open the file for Sourcery to start scanning it

Yes. Sometimes it scans files that are not opened in my IDE.

Log file has been sent by email.

@ruancomelli
Copy link
Contributor

Thank you for your reply and for sending your log file, @PhunkyBob!
We will investigate this issue and fix it as soon as we can.

@blakeNaccarato
Copy link

blakeNaccarato commented Jan 3, 2024

I've been tracking this issue since it was closed by the original poster (in my opinion unresolved) over in sourcery-ai/sourcery-vscode#150. I believe that the identified root cause is correct, that Pylance and Sourcery are not playing nicely together.

However, the conclusion seems to be that it's not Sourcery's responsibility (or was understandably not top priority at the time), since these files are being opened by Pylance and thus triggering the re-scanning. I suspect that Pylance+Sourcery is not an entirely uncommon combination, so maybe this issue can be revisited (either here or in the VSCode extension repo)?

I think the conclusion reached in sourcery-ai/sourcery-vscode#150 is not quite right, because lots of these pending Sourcery scans get banished in a limbo state, never actually completing their scan because perhaps there is no file left to scan. I believe Pylance creates ad-hoc temporary files when scanning, especially with Jupyter notebooks it creates temporary Python scripts per-cell and scans those to report on notebook contents. I imagine that the temporary files are necessitated by some limitation I'm not aware of.

I suspect the solution options might look like the following:

  • Sourcery for VSCode will still respond to the same events as it always does, presumably file open/save/close or w/e
  • Sourcery should expect that some files may exist at trigger time, but may be deleted by another extension, leading to limbo/invalid scans
    • Do other extensions (e.g. Pylance) emit any standard signals that "lock" that file? Can Sourcery listen for those signals and wait until they resolve before checking for file existence again, and throwing the analysis away if the file is gone?
    • Alternatively, would a configurable delay (or unconfigurable good-enough default) suffice where Sourcery checks for file existence after x milliseconds post-event-trigger, interrupting/cancelling scanning if the file is gone?

Some contact with the Pylance team might help chase this problem down. Here are some quoted snippets from the earlier issue for some more context:

sourcery-ai/sourcery-vscode#150 (comment)
As you suggested, it seems that many files are being opened and closed, triggering Sourcery reviews multiple times. Just note that Sourcery does not open/close files for you - instead, a Sourcery review is triggered every time a file is opened via other means.

sourcery-ai/sourcery-vscode#150 (comment)
After a little big of extra digging, I've narrowed this down to an issue with Pyright/Pylance going crazy and opening and closing files en masse, so I'm fairly certain this is not an actual bug with Sourcery, so I'm closing this issue. I've since put a pyrightconfig.json file in my project with a lot of explicit exclusions and inclusions and it seems to have calmed down.

I've configured my pyrightconfig.json various ways, and have never been able to get Sourcery to calm down. For now, I just set my VSCode notifications to "silent mode" and ignore the dozen or so infinite-duration Sourcery notifications that pop up in my repos.

@PhunkyBob
Copy link
Author

lots of these pending Sourcery scans get banished in a limbo state, never actually completing their scan because perhaps there is no file left to scan

Sometimes, I create a new file ("Untitled 1") and I quickly save it as "[something].py". But I see in the Sourcery window a loop in scanning "Untitled 1".
This coroborates your statement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants