Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduled Disk Cleanup #457

Closed
AnandChowdhary opened this issue Jul 2, 2019 · 16 comments · Fixed by #2050
Closed

Scheduled Disk Cleanup #457

AnandChowdhary opened this issue Jul 2, 2019 · 16 comments · Fixed by #2050

Comments

@AnandChowdhary
Copy link

Is your feature request related to a problem? Please describe.
I have a Netdata monitor set up which sends me a notification when it thinks "Out of space time = 7h", which means I've probably done a bunch of deploys in a short period of time, and I should use the "Disk Cleanup" feature to remove older images.

Describe the solution you'd like
It would be great if I can schedule Disk Cleanup, say once a day, week, or month. I can set up how many recent images I want to keep, and it — like a cron job — deletes unused images automatically.

Describe alternatives you've considered
I can use an API or set up a cron job myself which hits the endpoint to clean up the disk automatically, but it's not very secure since Caprover uses a password-based authentication. It would be super useful to schedule it from within the Caprover UI.

@githubsaturn
Copy link
Collaborator

It's a good suggestion, meanwhile, you can use a 3rd party solution like this one:
blog: https://marcopeg.com/2019/docker-vacuum
github project: https://github.com/marcopeg/docker-vacuum

@grantcodes
Copy link

In case someone ends up wanting to set up docker vacuum with caprover like I did:

  1. Create an app with persistent data
  2. Don't expose as a webapp
  3. Map /var/run/docker.sock to /var/run/docker.sock as a persistent directory
  4. Deploy captain-definition file:
{
  "schemaVersion": 2,
  "imageName": "marcopeg/docker-vacuum"
}

@redrockzee
Copy link

redrockzee commented Apr 17, 2020

In case someone ends up wanting to set up docker vacuum with caprover like I did:

  1. Create an app with persistent data
  2. Don't expose as a webapp
  3. Map /var/run/docker.sock to /var/run/docker.sock as a persistent directory
  4. Deploy captain-definition file:
{
  "schemaVersion": 2,
  "imageName": "marcopeg/docker-vacuum"
}

I had this working. Unfortunately, maybe with a docker update, the image started throwing errors.

Error: "Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

@negati-ve
Copy link

if an application is set to have 0 instances, would it be at risk to be cleaned up by Vaccum(with default settings)?

@cakeslice
Copy link

Yes please!

I think this is one of the last missing pieces to make CapRover a fully setup and forget system

@seiyria
Copy link

seiyria commented Nov 29, 2020

Not sure if it helps, but what I do is set this in my crontab:

0 3 * * * /usr/bin/docker system prune -af

@Turab
Copy link

Turab commented Dec 11, 2020

@seiyria, that might be quite dangerous. If for a reason your apps had stopped, that command will remove all your app containers too. Maybe you can redeploy but there is also a chance for Caprover to break.

@maietta
Copy link

maietta commented Feb 4, 2021

Not sure if it helps, but what I do is set this in my crontab:

0 3 * * * /usr/bin/docker system prune -af

This is exactly what I do. But I also use continuous deployment model so just in case something goes wrong, I'm only one click away from a quick re-build from last git commit. So far the docker system prune option has not let me down.

@maietta
Copy link

maietta commented Aug 26, 2023

Not sure if it helps, but what I do is set this in my crontab:

0 3 * * * /usr/bin/docker system prune -af

This is exactly what I do. But I also use continuous deployment model so just in case something goes wrong, I'm only one click away from a quick re-build from last git commit. So far the docker system prune option has not let me down.

Update: This bit me. I had turned off a service to back it up then it was gone when I tried to scale it from 0 back to 1. So, I recommend nobody does this.

@Turab
Copy link

Turab commented Sep 2, 2023

@seiyria, that might be quite dangerous. If for a reason your apps had stopped, that command will remove all your app containers too. Maybe you can redeploy but there is also a chance for Caprover to break.

As I said before, running that command is actually dangerous. As a temporary solution, I use Portainer. I log into it once in a while and remove unused images. That's a safe solution until Caprover develops a better way.

@s-kris
Copy link

s-kris commented Feb 29, 2024

In case someone ends up wanting to set up docker vacuum with caprover like I did:

  1. Create an app with persistent data
  2. Don't expose as a webapp
  3. Map /var/run/docker.sock to /var/run/docker.sock as a persistent directory
  4. Deploy captain-definition file:
{
  "schemaVersion": 2,
  "imageName": "marcopeg/docker-vacuum"
}

@grantcodes
getting this error: exec /bin/sh: exec format error

@githubsaturn
Copy link
Collaborator

Usually this error happens when the cpu architecture doesn't match. Are you running on an ARM server by any chance?

@s-kris
Copy link

s-kris commented Feb 29, 2024

Yes! I'm on Hetzner ARM64.

@githubsaturn
Copy link
Collaborator

You can try forking the project and use buildx, similar to how CapRover build works. We compile for all CPU architectures.

Here are the relevant lines:

docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
export DOCKER_CLI_EXPERIMENTAL=enabled
docker buildx ls
docker buildx create --name mybuilder
docker buildx use mybuilder
docker buildx build --platform linux/amd64,linux/arm64,linux/arm -t $IMAGE_NAME:$CAPROVER_VERSION -t $IMAGE_NAME:latest -f dockerfile-captain.release --push .

@s-kris
Copy link

s-kris commented Mar 4, 2024

Thank you!

However, I just forked the repo and added a captain-definition file to deploy using dockerfile. It seemed faster. :)

Screenshot 2024-03-04 at 12 40 39 PM

@githubsaturn
Copy link
Collaborator

tenor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants