Upgrading from 3.6 to 3.7.x greatly decreased performance or raising a RecursionError in some situations

[tomagendium]

Hi,

We upgraded from 3.6 to 3.7 (and also tested all 3.7.x releases individually) and there seems to have been something introduced which has massively decreased performance in some situations, if not hanging altogether. I haven't managed to create an isolated test case, but from what I can tell it's happening to models which have a ForeignKey field to themselves, and that field is also one of the fields in order_with_respect_to, e.g.

class MyModel(django.db.models.Model):
    another_field = CharField()
    parent = ForeignKey('MyModel')

    order_with_respect_to = ('another_field','parent')

In one case I did manage to catch a RecursionError before the app crashed, here's a partial trace of that (the top 23 lines just repeated over and over)

  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/fields/related_descriptors.py", line 187, in __get__
    rel_obj = self.get_object(instance)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/fields/related_descriptors.py", line 154, in get_object
    return qs.get(self.field.get_reverse_related_filter(instance))
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/query.py", line 431, in get
    num = len(clone)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/query.py", line 262, in __len__
    self._fetch_all()
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/query.py", line 1324, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/query.py", line 69, in __iter__
    obj = model_cls.from_db(db, init_list, row[model_fields_start:model_fields_end])
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/base.py", line 515, in from_db
    new = cls(*values)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/ordered_model/models.py", line 126, in __init__
    self._original_wrt_map = self._wrt_map()
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/ordered_model/models.py", line 131, in _wrt_map
    d[a] = get_lookup_value(self, a)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/ordered_model/models.py", line 14, in get_lookup_value
    return reduce(lambda i, f: getattr(i, f), field.split(LOOKUP_SEP), obj)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/ordered_model/models.py", line 14, in <lambda>
    return reduce(lambda i, f: getattr(i, f), field.split(LOOKUP_SEP), obj)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/fields/related_descriptors.py", line 187, in __get__
    rel_obj = self.get_object(instance)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/fields/related_descriptors.py", line 154, in get_object
    return qs.get(self.field.get_reverse_related_filter(instance))
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/query.py", line 424, in get
    clone = self._chain() if self.query.combinator else self.filter(*args, **kwargs)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/query.py", line 941, in filter
    return self._filter_or_exclude(False, args, kwargs)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/query.py", line 961, in _filter_or_exclude
    clone._filter_or_exclude_inplace(negate, args, kwargs)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/query.py", line 968, in _filter_or_exclude_inplace
    self._query.add_q(Q(*args, **kwargs))
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/sql/query.py", line 1416, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/sql/query.py", line 1435, in _add_q
    child_clause, needed_inner = self.build_filter(
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/sql/query.py", line 1288, in build_filter
    return self._add_q(
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/sql/query.py", line 1433, in _add_q
    joinpromoter = JoinPromoter(q_object.connector, len(q_object.children), current_negated)
  File "/Users/user/.virtualenvs/api/lib/python3.9/site-packages/django/db/models/sql/query.py", line 2453, in __init__
    self.votes = Counter()
  File "/opt/homebrew/Cellar/[email protected]/3.9.16/Frameworks/Python.framework/Versions/3.9/lib/python3.9/collections/__init__.py", line 592, in __init__
    super().__init__()
  RecursionError: maximum recursion depth exceeded while calling a Python object

[rafammpp]

I think #295 resolves this issue aswell.

[shuckc]

How many levels deep are your child/parent groupings? Is this some sort of a Tree/TOC structure?
It's not one we have testing for, I will see if we can add it to the test suite and see what causes the recursive calls through _wrt_map.

[tomagendium]

Not very deep, perhaps 3 or 4 levels at most, but yep essentially a tree structure of the same model, where children are ordered relative to their immediate parent (plus some other unrelated field)