Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Impossible to run errbot test using official docs #1641

Open
maxeval opened this issue May 29, 2023 · 2 comments
Open

Impossible to run errbot test using official docs #1641

maxeval opened this issue May 29, 2023 · 2 comments

Comments

@maxeval
Copy link

maxeval commented May 29, 2023

Describe the bug
Followed the instructions per official docs for testing, both options with using pytest with function based tests / or unittest with FullStackTest class are failing (instruction for usage of FullStackTest is taken from the class docstring). By failing means test env runner doesn't quit after all the tests are run (apparently due to Threads are not closed properly, because postgresql sometimes throws an error that there are several connections to db that is not closed).

To Reproduce
Follow the instructions of the official docs

Expected behavior
Test env run all the tests and quits.

Screenshots
See traceback below.

Environment (please complete the following information):

  • Errbot version: 9.9.9
  • OS version: Ubuntu
  • Python version: 3.10
  • Using a virtual environment: yes
  • Using Docker: yes and no (locally is not used, but used in CI/CD pipeline)

Additional context
After running the test, test env runner is not closed automatically (it simply freezes), once you Ctrl+C, the following traceback is appeared:

^C
Exception ignored in at exit callback: <function _exit_function at 0x7f27cafcf520>
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/util.py", line 334, in _exit_function
    _run_finalizers(0)
  File "/usr/lib/python3.10/multiprocessing/util.py", line 300, in _run_finalizers
    finalizer()
  File "/usr/lib/python3.10/multiprocessing/util.py", line 224, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 720, in _terminate_pool
    task_handler.join()
  File "/usr/lib/python3.10/threading.py", line 1096, in join
    self._wait_for_tstate_lock()
  File "/usr/lib/python3.10/threading.py", line 1116, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
KeyboardInterrupt:

Here is a traceback that occurs in CI/CD pipeline (see below):

Exception in thread Thread-32 (_handle_workers):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
.    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 522, in _handle_workers
    cls._wait_for_updates(current_sentinels, change_notifier)
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 503, in _wait_for_updates
    while not change_notifier.empty():
  File "/usr/lib/python3.10/multiprocessing/queues.py", line 353, in empty
    return not self._poll()
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 257, in poll
    return self._poll(timeout)
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 424, in _poll
    r = wait([self], timeout)
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 931, in wait
    ready = selector.select(timeout)
  File "/usr/lib/python3.10/selectors.py", line 416, in select
    fd_event_list = self._selector.poll(timeout)
OverflowError: timeout is too large
Exception in thread Thread-53 (_handle_workers):
Traceback (most recent call last):
.  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 522, in _handle_workers
    cls._wait_for_updates(current_sentinels, change_notifier)
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 503, in _wait_for_updates
    while not change_notifier.empty():
  File "/usr/lib/python3.10/multiprocessing/queues.py", line 353, in empty
    return not self._poll()
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 257, in poll
    return self._poll(timeout)
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 424, in _poll
    r = wait([self], timeout)
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 931, in wait
    ready = selector.select(timeout)
  File "/usr/lib/python3.10/selectors.py", line 416, in select
    fd_event_list = self._selector.poll(timeout)
OverflowError: timeout is too large
Exception in thread Thread-74 (_handle_workers):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
.
Ran 4 tests in 2.868s

OK
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 522, in _handle_workers
    cls._wait_for_updates(current_sentinels, change_notifier)
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 503, in _wait_for_updates
    while not change_notifier.empty():
  File "/usr/lib/python3.10/multiprocessing/queues.py", line 353, in empty
    return not self._poll()
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 257, in poll
    return self._poll(timeout)
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 424, in _poll
    r = wait([self], timeout)
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 931, in wait
    ready = selector.select(timeout)
  File "/usr/lib/python3.10/selectors.py", line 416, in select
    fd_event_list = self._selector.poll(timeout)
OverflowError: timeout is too large

This problem occurs on both unittest (with usage of FullStackTest cls) and pytest (using func based tests) runner.

@keenborder786
Copy link
Contributor

okay let me have a look at it.

@maxeval
Copy link
Author

maxeval commented Jun 8, 2023

@keenborder786
Alright, described above problem occurs when I try to use a bot as a separate django app and ORM inside of it respectively: looks like django closes the thread if there is a call to db using standard django procedure - HTTP request; once it is something different (e.g. errbot separate runner with their own threads) it cannot close the thread.. Decided to move the errbot to a separate docker container to make communication with django BE via HTTP.
However, tests are still failing if I try to use the official errbot docs: the thing is, it cannot find the attribute:
AttributeError: 'TestClass' object has no attribute 'bot_thread'
The following plug helped to solve the issue:

from errbot.backends.test import FullStackTest

supportbot_plugins_dir = "plugins"

class TestBot(FullStackTest):
    def __init__(self, *args, **kwargs) -> None:
        self.bot_thread = None
        super().__init__(*args, **kwargs)

    def setUp(self, **kwargs) -> None:
        super().setUp(extra_plugin_dir=supportbot_plugins_dir, loglevel=logging.ERROR, **kwargs)

Thanks in advance! Kindest regards!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants