Use as python library

[Badg]

Hello!

There was some discussion in this post about potentially shipping python bindings to py-spy to support its use as a library. I'd like to formally request this feature :)

My use case is as follows: I'm starting to design the controlplane for an application I'm working on. The local controlplane service is written in python, is started by systemd, and manages launching, monitoring, etc, some number of identical application processes. Each of the application processes would be child processes to the controlplane service. Meanwhile, the controlplane service contacts the remote controlplane coordinator, which devs can connect to. I'd like to completely disable remote access to the individual app servers, and only allow remote operations through explicitly programmed RPC calls between the controlplane coordinator and the controlplane service.

"All well and good", you say, "but where does py-spy come into this?" Well, I'd like to expose py-spy as an RPC call. So a dev could, for example, log into the controlplane coordinator (via cloudflare zero-trust), and then get a dump from a particular process on a particular app server. Or start a profiling run. Etc.

"Okay, but you can do that already. Why the library use?" Simple: if I can use py-spy as a python library, then it'll run with the same PID as the controlplane service, which means that the application processes will all be child processes -- which in turn means that I won't need to run the controlplane as a sudoer, which is a huge win from a security standpoint.

[ankush]

By dump do you mean current stacktrace only right?

If so, you can probably implement it with signals fairly easily. Your master process can send signal to all child process (let's say SIGUSR1) and they can dump the stacktrace to stderr or something for you to extract. You don't even have to write a ton of code for this: https://docs.python.org/3/library/faulthandler.html#faulthandler.register

That being said, I still want to see library access. It can unlock a lot of new use cases.

[Badg]

Hooking into signals would certainly be one option for getting a stacktrace of all running threads, and it's actually even something I've done before (though it has been a looooooong time, so thanks for the reminder!). Though I seem to remember it having some resiliency issues with deadlocks in the main thread; I might be mis-remembering that though.

At any rate, as you say, library access would open up a lot of new use cases. And in fact, I'd really like to be able to start and stop remote profiling this way, so I can avoid something like new relic, which is both super expensive and super heavyweight. Help with troubleshooting is all well and good, but for my architecture it's of limited use (it's trivial for me to just destroy the underlying process/instance and start up a new one, all with zero downtime, because of my load balancing setup). But remote profiling... now that's something that gets super interesting.