Distro tests fail within nixos #1822
Comments
|
Hi @JordanFaust, Would you mind providing more detail in your repro steps? I have never used Nix before, and I don't know how to get into a situation where I can run I tried the following:
This fails with so Thanks! |
|
Unfortunately it looks like this is a specific issue for NixOS and not nix the package manager. The above runs a container with a different OS and just has the nix CLI. I'm not sure the best way to test that in a container. I'm not sure what the specific issue is but something about these tests are failing when being built within NixOS. If you have any ideas on what could be the source I can try it out myself and try building from there. |
|
I wasn't able to reproduce this issue: FROM ubuntu:22.04
# install prerequisites
RUN apt-get update && apt-get install -y curl sudo xz-utils
# create a non-root user (recommended for running Nix)
RUN useradd -m nixuser && echo 'nixuser ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
# switch to the non-root user
USER nixuser
WORKDIR /home/nixuser
# install Nix
RUN curl -L https://nixos.org/nix/install | sh
# set up Nix environment
ENV USER nixuser
ENV HOME /home/nixuser
ENV PATH /home/nixuser/.nix-profile/bin:/home/nixuser/.nix-profile/sbin:/nix/var/nix/profiles/default/bin:/nix/var/nix/profiles/default/sbin:/run/current-system/sw/bin:/home/nixuser/.local/bin:$PATH
RUN . /home/nixuser/.nix-profile/etc/profile.d/nix.sh
# enable experimental features in Nix configuration
RUN mkdir -p /home/nixuser/.config/nix
RUN echo "experimental-features = nix-command flakes" >> /home/nixuser/.config/nix/nix.confThen: $ docker build -t localhost/nix:latest .
$ docker run --rm -it localhost/nix:latestAnd from within the container, everything seems to work: $ docker run --rm -it localhost/nix:latest
nixuser@78cbeeb35219:~$ nix profile install nixpkgs/92d295f588631b0db2da509f381b4fb1e74173c5#grype
nixuser@78cbeeb35219:~$ grype alpine:latest
✔ Vulnerability DB [updated]
✔ Parsed image sha256:ace17d5d883e9ea5a21138d0608d60aa2376c68f616c55b0b7e73fba6d8556a3
✔ Cataloged contents a0264d60f80df12bc1e6dd98bae6c43debe6667c0ba482711f0d806493467a46
├── ✔ Packages [15 packages]
├── ✔ File digests [80 files]
├── ✔ File metadata [80 locations]
└── ✔ Executables [17 executables]
✔ Scanned for vulnerabilities [14 vulnerability matches]
├── by severity: 0 critical, 0 high, 12 medium, 0 low, 0 negligible (2 unknown)
└── by status: 2 fixed, 12 not-fixed, 0 ignored
NAME INSTALLED FIXED-IN TYPE VULNERABILITY SEVERITY
busybox 1.36.1-r15 apk CVE-2023-42366 Medium
busybox 1.36.1-r15 apk CVE-2023-42365 Medium
busybox 1.36.1-r15 apk CVE-2023-42364 Medium
busybox 1.36.1-r15 apk CVE-2023-42363 Medium
busybox-binsh 1.36.1-r15 apk CVE-2023-42366 Medium
busybox-binsh 1.36.1-r15 apk CVE-2023-42365 Medium
busybox-binsh 1.36.1-r15 apk CVE-2023-42364 Medium
busybox-binsh 1.36.1-r15 apk CVE-2023-42363 Medium
libcrypto3 3.1.4-r5 3.1.4-r6 apk CVE-2024-2511 Unknown
libssl3 3.1.4-r5 3.1.4-r6 apk CVE-2024-2511 Unknown
ssl_client 1.36.1-r15 apk CVE-2023-42366 Medium
ssl_client 1.36.1-r15 apk CVE-2023-42365 Medium
ssl_client 1.36.1-r15 apk CVE-2023-42364 Medium
ssl_client 1.36.1-r15 apk CVE-2023-42363 Medium
A newer version of grype is available for download: 0.77.4 (installed version is 0.76.0)Are there other specific details that I'm missing from this setup? |
|
I don't believe there is an issue for nix the package manager. This specifically is an issue with NixOS and something specific about the nature of the distro tests that are failing within that hermetic build environment. I will need to dig into what the distro tests are actually doing to simulate the different distros. I'm assuming something in the test setup is causing issues within the nixos build environment. I can try and track this down if you can help me understand these tests a little better. |
|
Thanks @JordanFaust ! What the "distro" tests are doing is testing Grype's ability to detect what distro the image it's scanning is based on. The tests are defined here: grype/grype/distro/distro_test.go Line 109 in 8347931 My guess is that something in here is broken by the hermetic build environment. The way the tests essentially work is:
They're failing at Creating the file resolver is happening over in Syft code here: https://github.com/anchore/syft/blob/2d318cffaaca07a8b3e87d68f80f881153a305b8/syft/source/directorysource/directory_source.go#L135 I don't see anything in there that I think NixOS should be blocking - the Thanks for taking a look! Let me know if there's any other help I can be. |
|
Hello all, I help maintain the syft and grype packages in nixpkgs. I'm looking into this as part of our Zero Hydra (our ci system) Failures process for the next stable release. https://hydra.nixos.org/build/259554448/nixlog/3 I've tried just running Interestingly in a container I don't have the issue
You can then run And I'm not getting the issue inside the nixos/nix docker container. Is there an easy way to make the logs visible so I can check the errors during the |
|
Thanks for taking a look @06kellyjac! I haven't found an easy way to enable Grype's normal logging during a test run, so it might make sense to do a quick local change to return the error. |
|
Here's what's needed to log within tests (today... we can probably make this better): import (
"github.com/anchore/go-logger"
"github.com/anchore/go-logger/adapter/discard"
"github.com/anchore/go-logger/adapter/logrus"
"github.com/anchore/grype/internal/log"
"github.com/anchore/syft/syft"
...
)
...
for _, test := range tests {
t.Run(test.fixture, func(t *testing.T) {
l, err := logrus.New(
logrus.Config{
EnableConsole: true,
Level: logger.TraceLevel,
},
)
syft.SetLogger(l)
log.Set(l)
require.NoError(t, err)
require.NotNil(t, l)
// run your test...
syft.SetLogger(discard.New())
log.Set(discard.New())
})
}Another observation, the link to the logs doesn't seem to be running tests with |
|
Running locally with those changes: I don't see anything super useful here from my understanding. Unless maybe it sees "ignoring /" and that ignores everything? But I wouldn't expect that as the test fixture dir is meant to be the root from that point right? I don't see any of the error traces so unless the logging isn't set up right I guess it's not running into problems there Just running locally in my case I've got a tmpfs on root. Not 100% sure for the CI. https://grahamc.com/blog/erase-your-darlings/ |
ha! Agreed, that one seems a tad impactful (that will ignore the entire filesystem I believe). Syft doesn't catalog within certain filesystem mounts, so it seems like there is something peculiar with the system mounts (or something that needs to be corrected in syft). Can you confirm on a system that is exhibiting the bad behavior what the filesystem types for each mount? Would you be willing to post the output of edit: I do see a hint from the links you posted that we know the answer already...
|
|
I think this issue has the same root cause as anchore/syft#2894, which is more obviously a bug in Syft. |
What happened:
When building the latest version of grype the distro tests fail
What you expected to happen:
Successful build of grype
How to reproduce it (as minimally and precisely as possible):
nix profile install nixpkgs/92d295f588631b0db2da509f381b4fb1e74173c5#grype
Anything else we need to know?:
Test Failures:
Environment:
grype version:cat /etc/os-releaseor similar):The text was updated successfully, but these errors were encountered: