Trivy-operator in air-gapped environment

[krishk8s]

Discussed in #1873

^{Originally posted by Chaitan007 February 23, 2024}
What steps did you take and what happened:

Intalled trivy-operator using helm chart (with buildInTrivyServer) on EKS cluster
Operator :v0.18.5
Trivy Version: 0.49.1

Uploaded Trivy and Java db to artifactory using oras
added dbregistry to our private repo and also added DbRepositoryUsername and password to allow pull (used curl to test and it can get manifest.json file)

But it fails to pull the DB:
This is from trivy-server-0 logs:

Error: Failed to download vulnerablity DB: database download error: OCI repository error: 1 error occurred:

• GET https://our_repo/trivy/trivy-db/manifests/2: UNAUTHORIZED: the client does not have permissions for manifest; map[manifest:trivy/trivy-db/2/manifest.json.

• ---- used curl and it worked so no issue with file or auth

I have now tried with ECR.. it's same... UNAUTHORIZED, tried with inbuildserver and without it's all same
2-24T15:10:18Z ERROR reconciler.scan job Scan job container

Without servermode

InitContainer:
Command:
trivy
Args:
--cache-dir
/tmp/trivy/.cache
image
--download-db-only
--db-repository
01212121.dkr.ecr.eu-west-1.amazonaws.com/trivy/trivy-db

{"job": "trivy-system/scan-vulnerabilityreport-7b59f85f4d", "container": "5b12f8a 482c-9a48-1b937c7db448", "status.reason": "Error", "status.message": "2024-02-24T15:10:12.557Z\t\u0e1b[34mINFO\u0e1b [em\tNeed to update DB\n2024-02-24T15:10

Z\t\u001b[34mINFO\u001b[@m\tDB Repository: 01212121.dkr.ecr.eu-west-1.amazonaws.com/trivy/trivy-db\n2024-02-24T15:16:12.557Z\t\u0@1b[34mINFO\u0e1b
wnloading DB...\n2024-02-24T15:10:16.078Z\t\u00lb[31mFATAL\u001b[@m\tinit error: DB error: failed to download vulnerability DB: database download error: OCI tory error: 1 error occurred:\n\t* GET https://121212.dkr.ecr.eu-west-1.amazonaws.com/v2/trivy/trivy-db/manifests/2: unexpected status code 401
rized: Not Authorized\n\n\n\n"}
b.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).processFailedScanJob /home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:346
b.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller. (*ScanJobController).SetupwithManager.(*ScanJobController).reconcileJobs.funcl
/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:81
k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113
k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile /home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119
k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316
k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266
k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2

**
used oras to push the db.tar.gz file to ecr.

oras push localhost:5000/trivy-db:2
db.tar.gz:application/vnd.aquasec.trivy.db.layer.v1.tar+gzip

What did you expect to happen:

Able to pull the db from private repo.

Thanks

[krishk8s]

Does this even supported ?

i have tried just running trivy to download the db using private ecr registry... same no luck.

1. Uploaded the db.tar.gz file using oras

trivy_db.tar.gz:application/vnd.aquasec.trivy.db.layer.v1.tar+gzip \
        --artifact-type application/vnd.aquasec.trivy.config.v1+json"

2. Created IRSA role and used to access ecr

3. Command used:
   args: ["image", "--db-repository", "ecr/your/private-db", "--download-db-only"]

Used debug flag:
-There is no valid metadata file: unable to open file: open /root/.cache/trivy/db/metadata.json: no such file or directory

• no metadata file
• FATAL inti error
• open /tmo/trivy232323/workspace/db.tar.gz: no such file or directory.

[chen-keinan]

@krishk8s Have you added credentials from private registry

[krishk8s]

@chen-keinan thanks for the response, yes i did tried that too, but our artifactory has blocked sending plan text username/password (artifactory config setup, which can't be modified), also found out that oras doesn't support artifactory https://oras.land/docs/compatible_oci_registries.
So switched to ECR using IAM-ServiceAccount (IRSA), but still it's the same, i shared the logs in my previous comments.

[chen-keinan]

@krishk8s unfortunately currently it is only support user/password , I'll have a look and see if it possible to add support for iam-servicesaccount

[krishk8s]

@chen-keinan is there any insure connection option ? when i use "curl" with -k it works (so wanted to try), so wondering if there are any such option here...
other wise looks my option is to use s3 with init-container to mount the db directly in cachdir..

[chen-keinan]

@chen-keinan is there any insure connection option ? when i use "curl" with -k it works (so wanted to try), so wondering if there are any such option here... other wise looks my option is to use s3 with init-container to mount the db directly in cachdir..

nope , no option like this

[krishk8s]

@chen-keinan thanks for the confirmation, do you have any other suggestions for me to get this working ?

[chen-keinan]

@chen-keinan thanks for the confirmation, do you have any other suggestions for me to get this working ?

currently no, if I'll think on something new I'l update you.

qq: you can generate credential (user/pass) on ecr ?

[krishk8s]

@chen-keinan thanks for the confirmation, do you have any other suggestions for me to get this working ?

currently no, if I'l think on something new I'l update you.

qq: you can generate credential (user/pass) on ecr ?

Thanks @chen-keinan, unfortunately no.

[chen-keinan]

@krishk8s have you looked at ECR setting

[krishk8s]

Hello @chen-keinan yes i did no luck.
btw.. that's how we can access most(all) of aws resources from eks pods - IRSA-OIDC.

[github-actions]

This issue is stale because it has been labeled with inactivity.