Werf cleanup question

[bonza]

Hi,

We have a bit specific issue with our ad-hoc ECR registry cleanup tool and are very interested to understand if werf will help to solve it or not.

We have short-living Kubernetes jobs running as part of HELM deployment using hooks, so these jobs are disappearing from Kubernetes right after release is finished. We are using helm manifests to get a list of images that are in use, and apparently, these jobs are not listed there.
helm get manifest --namespace $NAMESPACE ${CHART_NAME}-${NAMESPACE} > release-manifest.yml

Does werf use a different approach to get the list of images that are in use?

[alexey-igrychev]

We simply take the images of all the resources from Kubernetes (https://github.com/werf/werf/blob/main/pkg/cleaning/allow_list/kubernetes.go).

[bonza]

@alexey-igrychev your documentation says: Created hooks resources will not be deleted after completion unless there is special annotation "helm.sh/hook-delete-policy": hook-succeeded,hook-failed.
So if the resource has this annotation, the resource will be deleted and werf will delete the corresponding image, right?

[alexey-igrychev]

werf will not ignore this image when cleaning if it does not find it in Kubernetes, but there are other policies. https://werf.io/documentation/v1.2/advanced/cleanup.html#cleaning-up-outdated-data

[alexey-igrychev]

Also, werf cleanup does not keep images for completed and failed jobs (

werf/pkg/cleaning/allow_list/kubernetes.go

Lines 220 to 231 in b7edaa3

	switch c.Type {
	case batchv1.JobComplete:
	if c.Status == corev1.ConditionTrue {
	logboek.Context(ctx).Debug().LogF("Ignore complete job/%s: images in this resource are not used anymore and can be safely removed\n", job.Name)
	continue FindActiveJobs
	}
	case batchv1.JobFailed:
	if c.Status == corev1.ConditionTrue {
	logboek.Context(ctx).Debug().LogF("Ignore failed job/%s: images in this resource are not used anymore and can be safely removed\n", job.Name)
	continue FindActiveJobs
	}
	}

).