: TypeError: RawGitHubAPI._make_request() got an unexpected keyword argument 'extra_headers'

[clstaudt]

I am trying out octomachinery with a simple bot implemented as a class:

import re
from octomachinery.app.server.runner import run as run_app
from octomachinery.routing import process_event_actions
from octomachinery.routing.decorators import process_webhook_payload
from octomachinery.runtime.context import RUNTIME_CONTEXT

class EyeingBot:
    """
    A class representing a GitHub bot that reacts to comments on issues.

    This bot, when mentioned in a comment on an issue, responds by 
    reacting to the comment with "eyes" and posting a response stating 
    it has read the issue.
    """
    name = "eyeingbot"

    @process_event_actions("issue_comment", {"created"})
    @process_webhook_payload
    async def on_comment(
        self,
        *,
        action,
        issue,
        comment,
        repository=None,
        sender=None,
        installation=None,
        assignee=None,
        changes=None,
        organization=None,
    ):
        """  
        Asynchronously responds to comments on issues where the bot is mentioned.

        This method is triggered when a comment is created on an issue. If the bot is 
        mentioned in the comment, it reacts to the comment with "eyes" and posts a 
        response stating it has read the issue.
        """
        github_api = RUNTIME_CONTEXT.app_installation_client
        if re.search(r"\@{}\b".format(self.name), comment["body"]):
            comment_reactions_api_url = f"{comment['url']}/reactions"
            await github_api.post(
                comment_reactions_api_url,
                preview_api_version="squirrel-girl",
                data={"content": "eyes"},
            )

            issue_comments_api_url = f"{issue['url']}/comments"
            await github_api.post(
                issue_comments_api_url,
                data={"body": "I have read the issue."},
            )


bot = EyeingBot()

run_app(
    name=bot.name,
    version="0.0.1",
    url="https://github.com/apps/eyeingbot",
)

Traceback

2023-06-16T13:17:45.229604+00:00 heroku[web.1]: Starting process with command `python triage/eyeingbot.py`
2023-06-16T13:17:47.340600+00:00 app[web.1]: /app/.heroku/python/lib/python3.10/site-packages/envparse.py:195: UserWarning: Could not any envfile.
2023-06-16T13:17:47.340625+00:00 app[web.1]:   warnings.warn('Could not any envfile.')
2023-06-16T13:17:47.441986+00:00 app[web.1]: INFO:octomachinery.app.server.machinery:Webhook secret is [NOT SET]: SIGNED WEBHOOKS WILL BE REJECTED
2023-06-16T13:17:47.842291+00:00 app[web.1]: INFO:octomachinery.app.server.machinery:Starting the following GitHub App:
2023-06-16T13:17:47.842746+00:00 app[web.1]: INFO:octomachinery.app.server.machinery:* app id: 337171
2023-06-16T13:17:47.842800+00:00 app[web.1]: INFO:octomachinery.app.server.machinery:* private key SHA-1 fingerprint: cc:d2:28:01:9f:62:7a:39:85:ee:36:cc:15:0c:be:37:18:0e:67:4b
2023-06-16T13:17:47.842840+00:00 app[web.1]: INFO:octomachinery.app.server.machinery:* user agent: eyeingbot/0.0.1 (+https://github.com/apps/eyeingbot)
2023-06-16T13:17:47.845414+00:00 app[web.1]: Traceback (most recent call last):
2023-06-16T13:17:47.845415+00:00 app[web.1]:   File "/app/triage/eyeingbot.py", line 58, in <module>
2023-06-16T13:17:47.845475+00:00 app[web.1]:     run_app(
2023-06-16T13:17:47.845476+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/octomachinery/app/server/runner.py", line 70, in run
2023-06-16T13:17:47.845512+00:00 app[web.1]:     run_until_complete(run_server_forever, config, event_routers)
2023-06-16T13:17:47.845512+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/anyio/__init__.py", line 72, in run
2023-06-16T13:17:47.845555+00:00 app[web.1]:     return asynclib.run(func, *args, **backend_options)  # type: ignore
2023-06-16T13:17:47.845555+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 114, in run
2023-06-16T13:17:47.845592+00:00 app[web.1]:     raise exception
2023-06-16T13:17:47.845592+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 76, in wrapper
2023-06-16T13:17:47.845622+00:00 app[web.1]:     retval = await func(*args)
2023-06-16T13:17:47.845623+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/octomachinery/utils/asynctools.py", line 14, in async_func_wrapper
2023-06-16T13:17:47.845648+00:00 app[web.1]:     return await async_func(*args, **kwargs)
2023-06-16T13:17:47.845648+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/octomachinery/app/server/machinery.py", line 134, in run_forever
2023-06-16T13:17:47.845684+00:00 app[web.1]:     await _prepare_github_app(github_app)
2023-06-16T13:17:47.845684+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/octomachinery/app/server/machinery.py", line 64, in _prepare_github_app
2023-06-16T13:17:47.845713+00:00 app[web.1]:     await github_app.log_installs_list()
2023-06-16T13:17:47.845713+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/octomachinery/github/api/app_client.py", line 71, in log_installs_list
2023-06-16T13:17:47.845744+00:00 app[web.1]:     installations = await self.get_installations()
2023-06-16T13:17:47.845744+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/octomachinery/github/api/app_client.py", line 131, in get_installations
2023-06-16T13:17:47.845778+00:00 app[web.1]:     async for install in amap(
2023-06-16T13:17:47.845779+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/octomachinery/utils/asynctools.py", line 81, in amap
2023-06-16T13:17:47.845808+00:00 app[web.1]:     async for async_value in async_iterable:
2023-06-16T13:17:47.845808+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/octomachinery/github/api/utils.py", line 49, in async_generator_wrapper
2023-06-16T13:17:47.845836+00:00 app[web.1]:     async for result_item in coroutine_instance:
2023-06-16T13:17:47.845836+00:00 app[web.1]:   File "/app/.heroku/python/lib/python3.10/site-packages/gidgethub/abc.py", line 185, in getiter
2023-06-16T13:17:47.845876+00:00 app[web.1]:     data, more, _ = await self._make_request(
2023-06-16T13:17:47.845884+00:00 app[web.1]: TypeError: RawGitHubAPI._make_request() got an unexpected keyword argument 'extra_headers'

[clstaudt]

I was able to avoid this error by downgrading:

gidgethub<=5.2.0

[webknjaz]

Thanks for the report! I'll keep it open until it's fixed by either restricting our direct runtime requirements or addressing the API change...

P.S. Meanwhile, I recommend using something like pip-tools to pin the runtime deps. Obviously, this is not something that would've helped in with the demos but still, is a good practice for production deployments.

[webknjaz]

Looks like this PR gidgethub/gidgethub#192 is at fault. It was merged on May 27, 2023, and released in v5.3.0 on May 30, 2023. We probably just need to handle the new arg with an optional support of no-arg for the older versions (or just bump the lower bound of the dependency).

[clstaudt]

@webknjaz Not related to this specific error, but does octomachinery support an OOP approach like in the code example above, where the callbacks are methods of a class? I had also trouble getting this to run.

[webknjaz]

Oh, I never thought about this case. Technically, you could make it work by decorating the methods on the initialized instance. Like instance.method = decorator(instance.method).

I suppose it's worth filing a feature request separately. Though, I'd probably prioritize refactoring I have been having in mind for quite a while, before considering such features...

[clstaudt]

Does this mean that a bot should better be implemented in a stateless, functional way? Are there any examples of complex bots implemented with octomachinery?

[webknjaz]

1. GitHub Apps are kinda stateless in sense that the events are delivered as standalone entities. This basically means that you can't share state across events due to the nature of GitHub's design. You can't easily make an app that would wait for an event in a callback for another event. Besides, GitHub is eventually consistent and this means that some events would come out of order, some events would come before some of GitHub's own caches populate (as in you can get a 404 when querying the API for a just-created issue). Also, GitHub never attempts redelivering events automatically — you can only redeliver events by clicking a button while debugging, one by one. Essentially, this means that some events will be lost due to network problems or GitHub's own outages — they sometimes disable sending out webhooks when https://githubstatus.com is red. You can query the Events API manually, but there's no common identifier between the webhook-delivered events and the ones returned from this API endpoint ([TODO] Implement event queue augmentation to fill in gaps/undelivered events #1). In short — complex state may be painful.
   If you need to save data, add a database, save changes when you see events but also re-verify them later to make sure the state on GitHub actually matches what you have.

2. I know that folks over https://github.com/thoth-station/ and https://github.com/AICoE/ have some bots like https://github.com/AICoE/Sefkhet-Abwy. I never remember which ones, so I tend to just use GitHub Search to check which repos use octomachinery.. Some of my apps look like this: https://github.com/sanitizers/patchback-github-app.

3. When I get to [Research] Figure out how to migrate the API to a snekomatic-style shared-maintenance lib #31, it'll be easier to add typical web app stuff, but right now, you'll need to stick with simpler functions. The framework provides the context available. You could also try using contextvars for some shared stuff if you don't mind them.

[webknjaz]

I released https://pypi.org/project/octomachinery/0.3.6.post0/ with a temporary version restriction for gidgethub. The next step will be to actually patch the method with the broken signature and revert that restriction.

[clstaudt]

@webknjaz Thanks, then I can remove the restriction in https://github.com/trIAgelab/trIAge

[webknjaz]

Hopefully, https://pypi.org/project/octomachinery/0.3.7/ should address the issue fully.