Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write RFC around performance/caching improvements #128

Closed
Tracked by #132
bpmct opened this issue Apr 18, 2024 · 3 comments
Closed
Tracked by #132

Write RFC around performance/caching improvements #128

bpmct opened this issue Apr 18, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@bpmct
Copy link
Member

bpmct commented Apr 18, 2024

We want to re-architect envbuilder in a way where it can run inside an existing devcontainer image (registry.company.com/envbuilder-images/python-project:latest) and pick up and rebuild any stale layers. This also includes a Terraform data source where it can check if there is an existing registry image for a repo/branch/Dockerfile or if the bare envbuilder image needs to be started. Envbuilder will also push the newly-built image

@mafredri
Copy link
Member

mafredri commented May 14, 2024

Status update: We've written an RFC (Notion: Improving Workspace Startup Performance with Envbuilder), but we're still investigating the best path forward.

For now, we've decided to create two PoCs to validate our approach:

If we can prove these two methods work, we will be able to implement our ideal solution, and have options for further improvement.

(Moving this issue to blocked until validation is complete.)

@mafredri
Copy link
Member

mafredri commented Jun 1, 2024

Both #185 and #186 have now been investigated and the approach for utilizing pre-built images has been validated. (See conclusions in aforementioned issues.)

See #213 for a reference implementation combining these PoCs.

The updated RFC is still pending approval, but in broad strokes we're looking to implement the following key components:

  1. New subcommands (or implemented as options via environment variables)
    • envbuilder get-cached-image
    • envbuilder build
  2. Embed the envbuilder binary in images
  3. Sanitize the final container image for envbuilder
  4. Two modes of operation
    1. Repo - use the repo/commit/hash as source of truth.
    2. Filesystem (current) - use the local files as source of truth.
  5. Implement terraform provider for envbuilder
  6. Future enhancements
    • Feed parsed devcontainer.json back into Terraform
    • Push Kaniko build cache layers as fully-fledged container images
    • Light-weight checkout for Repo-mode

(The details for each component have been omitted for brevity and pending approval.)

@mtojek
Copy link
Member

mtojek commented Jun 3, 2024

Thanks for the update, Mathias. I will make sure to include the relevant issues in the current sprint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants