Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Enhancement]: Implement timeout feature for GPT4 API #402

Closed
BytePhoenixCoding opened this issue Dec 22, 2023 · 7 comments · May be fixed by #520
Closed

[Enhancement]: Implement timeout feature for GPT4 API #402

BytePhoenixCoding opened this issue Dec 22, 2023 · 7 comments · May be fixed by #520
Labels
enhancement New feature or request

Comments

@BytePhoenixCoding
Copy link

Sometimes when running it, the API hangs and gets stuck on 'waiting'.

I would recommend a feature where if there are no new tokens generated from the API (ie no new response) in X amount of seconds, it re-completes the failed development step
@senko
Copy link
Collaborator

senko commented Dec 23, 2023

A simple way of achieving this would be to add timeout=X (where X is some value in seconds, eg. 120, ie. 2 minutes) to fail the request if it doesn't complete within a reasonable amount of time:

gpt-pilot/pilot/utils/llm_connection.py

Line 343 in a6bd846

response = requests.post(

This is not perfect but would probably be better than the current solution (wait forever).

@senko senko added the good first issue Good for newcomers label Dec 23, 2023
@dsitnik
Copy link

dsitnik commented Dec 23, 2023

@senko tried it with timeout=120. Still gets stuck on waiting.

@senko
Copy link
Collaborator

senko commented Dec 24, 2023

@senko tried it with timeout=120. Still gets stuck on waiting.

Could you post a screenshot or copy/paste (if there's anything to copy paste)? If you're using CLI, you can also hit Ctrl-C to stop it, which wil also give you a full exception stack trace (~50 lines of messages), if you can copy/paste that it would also help out in pinpointing the location of the wait.

@dsitnik
Copy link

dsitnik commented Dec 24, 2023

@senko Here it is:

^C? We would appreciate if you let us store your initial app prompt. If you are OK with that, please just press ENTER                                                                     
Traceback (most recent call last):
  File "/home/dario/gpt-pilot/pilot/main.py", line 88, in <module>
    exit_gpt_pilot(project, ask_feedback)
  File "/home/dario/gpt-pilot/pilot/utils/exit.py", line 102, in exit_gpt_pilot
    ask_to_store_prompt(project, path_id)
  File "/home/dario/gpt-pilot/pilot/utils/exit.py", line 63, in ask_to_store_prompt
    answer = styled_text(project, question, ignore_user_input_count=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/utils/questionary.py", line 35, in styled_text
    response = questionary.text(question, style=used_style).unsafe_ask()  # .ask() is included here
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/questionary/question.py", line 92, in unsafe_ask
    return self.application.run()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 973, in run
    return loop.run_until_complete(coro)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 640, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 607, in run_forever
    self._run_once()
  File "/usr/lib/python3.11/asyncio/base_events.py", line 1922, in _run_once
    handle._run()
  File "/usr/lib/python3.11/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 881, in run_async
    return await _run_async(f)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/prompt_toolkit/application/application.py", line 744, in _run_async
    result = await f
             ^^^^^^^
KeyboardInterrupt

@dsitnik
Copy link

dsitnik commented Dec 24, 2023

@senko and this is the output I get when I change the model to gpt-3.5-turbo in .env

^CTraceback (most recent call last):
  File "/home/dario/gpt-pilot/pilot/main.py", line 77, in <module>
    project.start()
  File "/home/dario/gpt-pilot/pilot/helpers/Project.py", line 141, in start
    self.developer.start_coding()
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 66, in start_coding
    self.implement_task(i, dev_task)
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 113, in implement_task
    result = self.execute_task(convo_dev_task,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 400, in execute_task
    result = self.step_command_run(convo, step, i, success_with_cli_response=need_to_see_output)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 195, in step_command_run
    return run_command_until_success(convo, data['command'],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/cli.py", line 509, in run_command_until_success
    success = convo.agent.debugger.debug(convo, {
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/Debugger.py", line 75, in debug
    result = self.agent.project.developer.execute_task(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 400, in execute_task
    result = self.step_command_run(convo, step, i, success_with_cli_response=need_to_see_output)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 195, in step_command_run
    return run_command_until_success(convo, data['command'],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/cli.py", line 509, in run_command_until_success
    success = convo.agent.debugger.debug(convo, {
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/Debugger.py", line 75, in debug
    result = self.agent.project.developer.execute_task(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 400, in execute_task
    result = self.step_command_run(convo, step, i, success_with_cli_response=need_to_see_output)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 195, in step_command_run
    return run_command_until_success(convo, data['command'],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/cli.py", line 509, in run_command_until_success
    success = convo.agent.debugger.debug(convo, {
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/Debugger.py", line 75, in debug
    result = self.agent.project.developer.execute_task(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 400, in execute_task
    result = self.step_command_run(convo, step, i, success_with_cli_response=need_to_see_output)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 195, in step_command_run
    return run_command_until_success(convo, data['command'],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/cli.py", line 509, in run_command_until_success
    success = convo.agent.debugger.debug(convo, {
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/Debugger.py", line 75, in debug
    result = self.agent.project.developer.execute_task(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 405, in execute_task
    result = self.step_code_change(convo, task_description, step, i, test_after_code_changes)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/Developer.py", line 163, in step_code_change
    updated_convo = code_monkey.implement_code_changes(convo, task_description, step['code_change_description'],
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/agents/CodeMonkey.py", line 22, in implement_code_changes
    llm_response = convo.send_message('development/implement_changes.prompt', {
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/helpers/AgentConvo.py", line 86, in send_message
    response = create_gpt_chat_completion(self.messages, self.high_level_step, self.agent.project,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/utils/llm_connection.py", line 106, in create_gpt_chat_completion
    response = stream_gpt_completion(gpt_data, req_type, project)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/utils/llm_connection.py", line 172, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot/utils/llm_connection.py", line 343, in stream_gpt_completion
    response = requests.post(
               ^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
                       ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 1378, in getresponse
    response.begin()
  File "/usr/lib/python3.11/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 706, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1311, in recv_into
    return self.read(nbytes, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1167, in read
    return self._sslobj.read(len, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dario/gpt-pilot/pilot/main.py", line 88, in <module>
    exit_gpt_pilot(project, ask_feedback)
  File "/home/dario/gpt-pilot/pilot/utils/exit.py", line 100, in exit_gpt_pilot
    send_telemetry(path_id)
  File "/home/dario/gpt-pilot/pilot/utils/exit.py", line 20, in send_telemetry
    response = requests.post("https://api.pythagora.io/telemetry", json=telemetry_data)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connection.py", line 358, in connect
    conn = self._new_conn()
           ^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dario/gpt-pilot/pilot-env/lib/python3.11/site-packages/urllib3/util/connection.py", line 86, in create_connection
    sock.connect(sa)
KeyboardInterrupt

@dsitnik
Copy link

dsitnik commented Dec 24, 2023 •
edited

I noticed that it often gets stuck on "Implementing code changes for". For example:

Implementing code changes for e `Create a Django data migration to add a default Site object with domain 'localhost'. You must create a new migration file in the 'migrations' directory of an app (e.g., 'secopi_app'). Use the 'Site' model from 'django.contrib.sites.models' to create the new object.

Example inside a migration file:


from django.db import migrations
from django.conf import settings


def create_site(apps, schema_editor):
    Site = apps.get_model('sites', 'Site')
    Site.objects.create(name='localhost', domain='localhost')


class Migration(migrations.Migration):

    dependencies = [
        ('your_app_name', 'previous_migration_file_name'),
    ]

    operations = [
        migrations.RunPython(create_site),
    ]

Run the migrations after creating this file using the command 'python manage.py migrate'.`

@Yugma2605 Yugma2605 mentioned this issue Jan 11, 2024
Open
@techjeylabs techjeylabs changed the title Implement timeout feature for GPT4 API [Enhancement]: Implement timeout feature for GPT4 API Apr 24, 2024
@techjeylabs techjeylabs added enhancement New feature or request and removed good first issue Good for newcomers labels Apr 24, 2024
@techjeylabs
Copy link
Collaborator

closing this issue since it got solved with 2 timeouts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Adding timeout of 120 seconds Yugma2605/gpt-pilot
4 participants
@senko @dsitnik @BytePhoenixCoding @techjeylabs