Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: refine job matrix and enable cache for poetry #1381

Merged
merged 11 commits into from Apr 29, 2024
Merged

ci: refine job matrix and enable cache for poetry #1381

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
ec0711e
ci: refine job matrix and enable cache for poetry
Umpire2018 Apr 26, 2024
8d391f6
Merge branch 'main' into feat/enable-poetry-cache
Umpire2018 Apr 26, 2024
42c34ae
ci: enable Homebrew caching in Actions
Umpire2018 Apr 27, 2024
2eac801
Merge branch 'main' into feat/enable-poetry-cache
Umpire2018 Apr 27, 2024
36648a3
ci: optimize Docker and Colima installation in GitHub Actions
Umpire2018 Apr 27, 2024
b7e61ad
ci: remove Homebrew cache.
Umpire2018 Apr 27, 2024
c0c6e2f
fix typo
Umpire2018 Apr 27, 2024
8cd6995
Merge branch 'main' into feat/enable-poetry-cache
rbren Apr 27, 2024
ea8b19f
fix: specified the Python version to avoid errors in actions.
Umpire2018 Apr 27, 2024
361803c
Merge branch 'main' into feat/enable-poetry-cache
Umpire2018 Apr 28, 2024
d1ebf68
Merge branch 'main' into feat/enable-poetry-cache
enyst Apr 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/lint.yml
View file
Open in desktop
Expand Up @@ -32,6 +32,7 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: 3.11
cache: 'pip'
- name: Install pre-commit
run: pip install pre-commit==3.7.0
- name: Run pre-commit hooks
Expand Down
72 changes: 25 additions & 47 deletions .github/workflows/run-integration-tests.yml
View file
Open in desktop
Expand Up @@ -6,64 +6,42 @@ jobs:
on-linux:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.11"]
agent: ["SWEAgent", "PlannerAgent", "MonologueAgent", "CodeActAgent"]
sandbox: ["ssh", "exec"]
include:
- name: SWEAgent-py311-ssh
python-version: "3.11"
agent: "SWEAgent"
embedding-model: "none"
sandbox: "ssh"
- name: PlannerAgent-py311-ssh
python-version: "3.11"
agent: "PlannerAgent"
embedding-model: "none"
sandbox: "ssh"
- name: MonologueAgent-py311-ssh
python-version: "3.11"
agent: "MonologueAgent"
- agent: "MonologueAgent"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we remove this line (and L17, L19, L21)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite follow.

Current combinations are as follows

{"python-version": "3.11", "agent": "SWEAgent", "sandbox": "ssh", "embedding-model": "none"}
{"python-version": "3.11", "agent": "SWEAgent", "sandbox": "exec", "embedding-model": "none"}
{"python-version": "3.11", "agent": "PlannerAgent", "sandbox": "ssh", "embedding-model": "none"}
{"python-version": "3.11", "agent": "PlannerAgent", "sandbox": "exec", "embedding-model": "none"}
{"python-version": "3.11", "agent": "MonologueAgent", "sandbox": "ssh", "embedding-model": "local"}
{"python-version": "3.11", "agent": "MonologueAgent", "sandbox": "exec", "embedding-model": "local"}
{"python-version": "3.11", "agent": "CodeActAgent", "sandbox": "ssh", "embedding-model": "none"}
{"python-version": "3.11", "agent": "CodeActAgent", "sandbox": "exec", "embedding-model": "none"}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, my bad; this looks good

embedding-model: "local"
sandbox: "ssh"
- name: CodeActAgent-py311-ssh
python-version: "3.11"
agent: "CodeActAgent"
embedding-model: "none"
sandbox: "ssh"
- name: SWEAgent-py311-exec
python-version: "3.11"
agent: "SWEAgent"
- agent: "MonologueAgent"
# sufficient to have one agent testing against local sandbox
sandbox: "local"
embedding-model: "local"
- agent: "SWEAgent"
embedding-model: "none"
sandbox: "exec"
- name: PlannerAgent-py311-exec
python-version: "3.11"
agent: "PlannerAgent"
- agent: "PlannerAgent"
embedding-model: "none"
sandbox: "exec"
- name: MonologueAgent-py311-exec
python-version: "3.11"
agent: "MonologueAgent"
embedding-model: "local"
sandbox: "exec"
- name: CodeActAgent-py311-exec
python-version: "3.11"
agent: "CodeActAgent"
- agent: "CodeActAgent"
embedding-model: "none"
sandbox: "exec"
- name: MonologueAgent-py311-local
python-version: "3.11"
agent: "MonologueAgent"
embedding-model: "local"
# sufficient to have one agent testing against local sandbox
sandbox: "local"
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2

- name: Install poetry via pipx
run: pipx install poetry

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install Poetry
run: curl -sSL https://install.python-poetry.org | python3 -
python-version: '3.11'
cache: 'poetry'

- name: Install Python dependencies using Poetry
run: poetry install

- name: Build Environment
run: make build

- name: Run Integration Tests
env:
SANDBOX_TYPE: ${{ matrix.sandbox }}
Expand Down
34 changes: 25 additions & 9 deletions .github/workflows/run-unit-tests.yml
View file
Open in desktop
Expand Up @@ -11,22 +11,30 @@ jobs:

steps:
- uses: actions/checkout@v4

- name: Install poetry via pipx
run: pipx install poetry

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'poetry'

- name: Install Python dependencies using Poetry
run: poetry install

- name: Install & Start Docker
run: |
brew install colima docker
colima start
- name: Install and configure Poetry
uses: snok/install-poetry@v1
with:
version: latest

- name: Build Environment
run: make build

- name: Run Tests
run: poetry run pytest ./tests/unit

on-linux:
runs-on: ubuntu-latest
strategy:
Expand All @@ -35,13 +43,21 @@ jobs:

steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2

- name: Install poetry via pipx
run: pipx install poetry

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install Poetry
run: curl -sSL https://install.python-poetry.org | python3 -
cache: 'poetry'

- name: Install Python dependencies using Poetry
run: poetry install --without evaluation
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is part of make build

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know which have been removed in previous commit. I think add poetry install here can speed up actions and looks like no other side effects so i added it here.

Copy link
Collaborator

@li-boxuan li-boxuan Apr 28, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logically it wouldn't make sense to speed up... but it wouldn't harm either. I feel like we could remove make build completely from run-integration-tests.yml though.


- name: Build Environment
run: make build

- name: Run Tests
run: poetry run pytest ./tests/unit