add methods support. #30

jonykalavera · 2022-04-15T20:11:59Z

❗ WIP #11

jonykalavera · 2022-04-15T20:16:06Z

py2puml/inspection/inspectclass.py

@@ -103,19 +129,20 @@ def inspect_class_type(
 handle_inheritance_relation(class_type, class_type_fqn, root_module_name, domain_relations)

 def inspect_dataclass_type(
- class_type: Type[dataclass],
+ class_type: Type,


dataclass is just a decorator function.

jonykalavera · 2022-04-15T20:30:53Z

py2puml/inspection/inspectnamedtuple.py

@@ -16,5 +16,6 @@ def inspect_namedtuple_type(
 attributes=[
 UmlAttribute(tuple_field, 'Any', False)
 for tuple_field in namedtuple_type._fields
- ]
+ ],
+ methods=[],


technically namedtuples can have methods. should we support it?

from typing import NamedTuple class Employee(NamedTuple): """Represents an employee.""" name: str id: int = 3 def foo(self) -> str: return f'<Employee {self.name}, id={self.id}>' emp = Employee('Hello') print(emp) print(emp.foo()) print(emp._fields) ```

if it is easy to distinguish added methods (like the Employee.foo() one) from native methods of named tuples (like _asdict), let's do it. Otherwise, let's wait for the need to come and the feature to be requested 😃

let's see after I figure it out for regular classes and then we get back to this one to do it or split it.

jonykalavera · 2022-04-15T20:33:19Z

py2puml/inspection/inspectclass.py

+def inspect_methods(
+ definition_methods, class_type,
+):
+ no_dunder = lambda x: not (x[0].startswith('__') or x[0].endswith('__'))


just excluding magic methods

please, make no_dunder a proper function that must be unit-tested. And please, give more meaningful names to variables (x -> member, for instance)

😰 ups. lol I was really hacking it together. sure thing. also, I tested this with some other projects and it totally shows more than what we want to see. need to look a lot further into determining declared methods only.

lucsorel · 2022-04-20T21:40:09Z

py2puml/inspection/inspectclass.py

@@ -1,15 +1,14 @@

-from inspect import isabstract
+import inspect


guidelines favor named imports instead of module imports: named imports gives meaning to what is happening in the module, module imports clutters the module's memory footprint

lucsorel · 2022-04-20T21:43:55Z

py2puml/inspection/inspectclass.py

+def inspect_methods(
+ definition_methods, class_type,
+):
+ no_dunder = lambda x: not (x[0].startswith('__') or x[0].endswith('__'))


please, make no_dunder a proper function that must be unit-tested. And please, give more meaningful names to variables (x -> member, for instance)

lucsorel · 2022-04-20T21:44:25Z

py2puml/inspection/inspectclass.py

+ definition_methods, class_type,
+):
+ no_dunder = lambda x: not (x[0].startswith('__') or x[0].endswith('__'))
+ methods = filter(no_dunder, inspect.getmembers(class_type, callable))


from inspect import getmembers

lucsorel · 2022-04-20T21:44:42Z

py2puml/inspection/inspectclass.py

+ no_dunder = lambda x: not (x[0].startswith('__') or x[0].endswith('__'))
+ methods = filter(no_dunder, inspect.getmembers(class_type, callable))
+ for name, method in methods:
+ signature = inspect.signature(method)


from inspect import signature

lucsorel · 2022-04-20T21:48:57Z

py2puml/inspection/inspectclass.py

@@ -103,19 +129,20 @@ def inspect_class_type(
 handle_inheritance_relation(class_type, class_type_fqn, root_module_name, domain_relations)

 def inspect_dataclass_type(
- class_type: Type[dataclass],
+ class_type: Type,


lucsorel · 2022-04-20T21:52:43Z

py2puml/inspection/inspectnamedtuple.py

@@ -16,5 +16,6 @@ def inspect_namedtuple_type(
 attributes=[
 UmlAttribute(tuple_field, 'Any', False)
 for tuple_field in namedtuple_type._fields
- ]
+ ],
+ methods=[],


if it is easy to distinguish added methods (like the Employee.foo() one) from native methods of named tuples (like _asdict), let's do it. Otherwise, let's wait for the need to come and the feature to be requested 😃

lucsorel · 2022-04-20T21:54:24Z

py2puml/inspection/inspectclass.py

+ fqn=class_type_fqn,
+ attributes=definition_attrs,
+ methods=definition_methods,
+ is_abstract=inspect.isabstract(class_type)


from inspect import isabstract

lucsorel · 2022-04-20T21:56:21Z

py2puml/inspection/inspectclass.py

@@ -1,15 +1,14 @@

-from inspect import isabstract
+import inspect
 from typing import Type, List, Dict

 from re import compile
 from dataclasses import dataclass


dataclass seems not necessary anymore after your fix, is it?

I'll check.

lucsorel · 2022-04-20T22:04:55Z

py2puml/inspection/inspectclass.py

+ signature = inspect.signature(method)
+ uml_method = UmlMethod(
+ name=name,
+ signature=str(signature),


it is a convenient way to get the whole signature, but:

it does not output the signature in the PlantUML expected syntax see comments 2. in extract methods #11 (comment)

the way str(signature) behave is likely to change and break the feature

However, I don't like how I implemented the the way attributes detected by inspection (dataclasses, class attributes) are typed: builtins.date would be displayed instead of just date for example. I did a better job when detecting instance attributes in constructors, the whole would need harmonization.

it is a convenient way to get the whole signature, but:

* it does not output the signature in the PlantUML expected syntax see comments `2.` in [extract methods #11 (comment)](https://github.com/lucsorel/py2puml/issues/11#issuecomment-1104471337)

it does not guarantee it for sure. I should just iterate the structure we can rely on and build it from ther

* the way `str(signature)` behave is likely to change and break the feature

It might. yeah

However, I don't like how I implemented the the way attributes detected by inspection (dataclasses, class attributes) are typed: builtins.date would be displayed instead of just date for example. I did a better job when detecting instance attributes in constructors, the whole would need harmonization.

I'll make sure to take a closer look there.

for now, let's leave the signature format like that, it is not how UML users expect it, but it is readable to python users

lucsorel · 2022-04-20T22:06:22Z

py2puml/inspection/inspectclass.py

@@ -83,7 +74,37 @@ def inspect_static_attributes(
 uml_attr = UmlAttribute(attr_name, attr_type, static=True)
 definition_attrs.append(uml_attr)

- return definition_attrs
+
+def inspect_methods(


this function needs unit-testing, once we agree on the implementation details

lucsorel · 2022-04-20T22:08:05Z

thank you for the great job. We're nearly there 😃

Please also read my reply here: #11 (comment) and let me know what you think 🙏

lucsorel · 2022-04-20T22:09:22Z

thank you for the great job. We're nearly there 😃

Please also read my reply here: #11 (comment) and let me know what you think 🙏

jonykalavera · 2022-04-21T20:22:21Z

won't work on this today. will save it for the weekend.

lucsorel · 2022-05-24T07:21:02Z

hi @jonykalavera, how are you?

I would like to see your work being merged in master, surely so do you 😃

I listed a couple of things to be done:

add methods support. #30 (comment): you were considering removing methods parsing for namedtuples
add methods support. #30 (comment): propoer method for ignoring dunder methods + unit tests
add methods support. #30 (comment): restore named imports from inspect
add methods support. #30 (comment): methods signature extraction needs unit testing
update the library version: https://github.com/lucsorel/py2puml/blob/master/CONTRIBUTING.md#contributions-and-version-update

Do you need some help? No rush anyway, I just wanted to show some support now that I found a little bit of free time

IllustratedMan-code · 2022-12-14T16:49:05Z

@lucsorel @jonykalavera I would love to help with this! Let me know if there is anything I can do.

IllustratedMan-code · 2022-12-14T17:41:37Z

I would just like to add that I tested this on the add-method branch and received the following error (on a fairly complicated project):

ValueError: no signature found for builtin <method 'clear' of 'dict' objects>

Adding a try except to py2puml.py seemed to solve the issue for me, though it obviously isn't ideal:

    for _, name, is_pkg in walk_packages([domain_path], f'{domain_module}.'):
        if not is_pkg:
            try:
                domain_item_module: ModuleType = import_module(name)
                inspect_module(
                    domain_item_module,
                    domain_module,
                    domain_items_by_fqn,
                    domain_relations
                )
            except ValueError as e:
                pass

lucsorel · 2022-12-14T22:10:19Z

I would love to help with this! Let me know if there is anything I can do.

Hi @IllustratedMan-code, thank you for volunteering to help this issue move forward 😃

What needs to be done is listed in this message: #30 (comment)

One of the main issue is tackling #30 (comment), where too many unwanted methods show up (I believe methods from inherited classes and Python magic methods we don't want to show up in the PlantUML diagram). Using inspection (which is done here) is easy but makes it hard to distinguish coded methods from inherited ones / magic methods; the alternative it to parse the Abstract Syntax Tree (as done to detect instance attributes in __init__ constructors), but it is harder to code.

The other issue is to add unit tests along with the feature.

lucsorel · 2022-12-14T22:18:41Z

I would just like to add that I tested this on the add-method branch and received the following error (on a fairly complicated project):
ValueError: no signature found for builtin <method 'clear' of 'dict' objects>
Adding a try except to py2puml.py seemed to solve the issue for me, though it obviously isn't ideal:
    for _, name, is_pkg in walk_packages([domain_path], f'{domain_module}.'):
        if not is_pkg:
            try:
                ...
            except ValueError as e:
                pass

Could you provide the stacktrace of the error, please? I believe the project has a user-defined class which extends a dictionary and that the branch's feature attempts to extract information from a native method which has no signature. Can you confirm that such a class exists in the project?

IllustratedMan-code · 2022-12-14T22:22:37Z

It does indeed! I'll provide a full trace soon

draft implementation of methods support.

823a7ca

jonykalavera commented Apr 15, 2022

View reviewed changes

adjustments

1bb6ef3

jonykalavera commented Apr 15, 2022

View reviewed changes

jonykalavera added 2 commits April 15, 2022 22:55

more adjustments

cd23f06

example

ad7712c

lucsorel requested changes Apr 20, 2022

View reviewed changes

lucsorel added the waiting for feedback questions waiting to be answered label Feb 12, 2023

lucsorel mentioned this pull request Mar 29, 2023

extract methods #11

Open

grayfox88 mentioned this pull request Apr 9, 2023

Add methods #43

Open

lucsorel added the enhancement New feature or request label Aug 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add methods support. #30

add methods support. #30

jonykalavera commented Apr 15, 2022 •

edited

jonykalavera Apr 15, 2022 •

edited

lucsorel Apr 20, 2022

jonykalavera Apr 15, 2022 •

edited

lucsorel Apr 20, 2022

jonykalavera Apr 21, 2022

jonykalavera Apr 15, 2022

lucsorel Apr 20, 2022

jonykalavera Apr 21, 2022 •

edited

lucsorel Apr 20, 2022

lucsorel Apr 20, 2022

lucsorel Apr 20, 2022

lucsorel Apr 20, 2022

lucsorel Apr 20, 2022

lucsorel Apr 20, 2022

lucsorel Apr 20, 2022

lucsorel Apr 20, 2022

jonykalavera Apr 21, 2022

lucsorel Apr 20, 2022

jonykalavera Apr 21, 2022

lucsorel May 24, 2022

lucsorel Apr 20, 2022

lucsorel commented Apr 20, 2022

lucsorel commented Apr 20, 2022

jonykalavera commented Apr 21, 2022

lucsorel commented May 24, 2022

IllustratedMan-code commented Dec 14, 2022

IllustratedMan-code commented Dec 14, 2022 •

edited

lucsorel commented Dec 14, 2022 •

edited

lucsorel commented Dec 14, 2022

IllustratedMan-code commented Dec 14, 2022

		@@ -1,15 +1,14 @@

		from inspect import isabstract
		import inspect

add methods support. #30

Are you sure you want to change the base?

add methods support. #30

Conversation

jonykalavera commented Apr 15, 2022 • edited

jonykalavera Apr 15, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonykalavera Apr 15, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonykalavera Apr 21, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lucsorel commented Apr 20, 2022

lucsorel commented Apr 20, 2022

jonykalavera commented Apr 21, 2022

lucsorel commented May 24, 2022

IllustratedMan-code commented Dec 14, 2022

IllustratedMan-code commented Dec 14, 2022 • edited

lucsorel commented Dec 14, 2022 • edited

lucsorel commented Dec 14, 2022

IllustratedMan-code commented Dec 14, 2022

jonykalavera commented Apr 15, 2022 •

edited

jonykalavera Apr 15, 2022 •

edited

jonykalavera Apr 15, 2022 •

edited

jonykalavera Apr 21, 2022 •

edited

IllustratedMan-code commented Dec 14, 2022 •

edited

lucsorel commented Dec 14, 2022 •

edited