ObservableModule

[jayvdb]

It would be very handy to have an ObservableModule, especially for sys in order to identify which code is making changes there.

[nocarryr]

Can you elaborate a little on this? Something like a subclass of Observable that tracks changes to a module's __dict__ attribute?

[jayvdb]

Ya, that is roughly what I had in mind. And if recursive, then also the containers within the module. And tracking __slots__ would be the other major source of change for a module.

[jayvdb]

Ideally something like

import sys, pydispatch
sys_change_emitter = pydispatch.wrap(sys)

[nocarryr]

In order to do that, the module's __dict__ would have to be replaced with an ObservableDict copy.

I'm not sure how that would impact the module's references to its own objects at that point though. For pure-python modules, it may be fine, but for modules relying on binary extensions (like most of Python's standard lib) there could be some crazy side effects.

If doing that recursively, other imported modules (since they're in the __dict__ as well) would either be wrapped or skipped, with some sort of tracking mechanism to avoid circular references (wrapping the same module over and over).

It sounds like fun, lol! But I think it'd be hard to keep things from breaking

[nocarryr]

According to https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarchy a module's __dict__ is read-only:

Modules
Modules are a basic organizational unit of Python code...
...
Special read-only attribute: __dict__ is the module’s namespace as a dictionary object.

[jayvdb]

module's dict is read-only

ya, the __dict__ contents are modifiable, but the __dict__ itself cant be replaced.

>>> import os
>>> len(os.__dict__.keys())
344
>>> os.__dict__['foo'] = 'bar'
>>> len(os.__dict__.keys())
345
>>> os.__dict__ = dict(os.__dict__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: readonly attribute

But the wrapper will need to create a new module object. Last time I did this, I used a normal class as the module wrapper (e.g. https://github.com/rinslow/fakeos/blob/master/fakeos.py#L13), and added module-like attributes so it acted like a module, like https://github.com/Akrog/modulefaker/blob/master/modulefaker/__init__.py#L35.

More modern attempts use the new import machinery voodoo to create real module objects which inherit from the correct classes.
https://github.com/rominf/module-wrapper looks interesting
iirc https://github.com/GrahamDumpleton/wrapt didnt have a module wrapper.

[jayvdb]

"module proxy" should be a good term, except it is full of http proxy stuff. Lazy module loaders may be useful to see how to do this, and search results tend to be more useful. https://github.com/cacilhas/ObjectProxy