overrides: overriding a sequence node with an empty one doesn't work

[jmtd]

Consider

image.yaml:

name: testing
version: 1
from: anything
packages:
  repositories:
  - name: testing
    rpm: does-not-exist

Now say you wish to override this and remove the repositories key entirely. You could try this

packages:
  repositories: ~

Except cekit will now die, because it expects to be able to iterate over a list (TypeError: 'NoneType' object is not iterable). So instead I tried replacing the list of repositories with an empty list

packages:
  repositories: []

This does not result in any difference - the repositories key from image.yaml is processed.

Tested with 2.2.5, I will retry with devel soon.

[rcernich]

I think we should omit repositories from modules altogether, because these have a tendency to be related to the build environment and not necessarily the image definition itself.

[goldmann]

I agree with @rcernich, having repositories defined in the modules (as long as the idea may sound great) makes the implementation and maintenance very hard. Overriding of this is another level of complexity.

Something that is a bit related to this issue: #386.

Handling repositories for images that extend each other is tricky, but we have some ideas about it. If you maintain your community image then you can define whatever repositories you want in the image descriptor (not in modules). These will use most probably RPM or URL repository types. It may happen that nothing is required to be declared in case default repositories are sufficient.

When extending that image and building a product image you definitely should use content sets. This will disable any other repositories defined. This is very important. This means that content sets are the single truth about the source of the package content.

So, what we can do about the issue? I think the most obvious one is the hard cut: remove support for defining repositories in modules.

@jmtd Your example is a bit misleading and I don't know the use case so I cannot give you more detailed suggestion on how it should be refactored, but I hope information above will let you do it yourself.

[jmtd]

Hi - I'm not looking to handle repositories in modules. I didn't mention modules in my initial post. Specifically, I was wanting to override this repositories key in my image.yaml , deleting it entirely. This is part of RHEL8 investigative work. for the released product images, we define content_sets via overrides files in the same repository so the repositories setting in image.yaml is ignored anyway. But for RHEL8 I can't (currently) use content sets so I need to disable the repositories key from an override another way.

[goldmann]

Thanks for this clarification, now I understand what you mean. I'll try to comment on it soon.

[goldmann]

OK, if I get it correctly this is what you want to do:

1. Do not use the scl repository since it doesn't exists (yet).
2. Cannot use content sets for some reason and you want to use other types.

One (simple?) workaround for issue 1 I could think of is to use a URL type of repository of the name scl (you need to create that file) that would point to an empty repository created with createrepo_c and hosted somewhere (fedorapeople.org maybe?).

For the other issue, your override should point to generated or created by you .repo files to point to correct repositories.

But I have the feeling that everything above is not required. I'm pretty sure there are already content set for RHEL 8 that could be used (most probably have different names), so you could just use these.

Let me know if I'm closer to solving your issue.

[rcernich]

Perhaps the definitions could be placed in a local module, which gets swapped out depending on what the platform is. For example:

./platforms
./platforms/rhel
./platfomrs/rhel/7
./platfomrs/rhel/8
./platforms/centos
./platforms/centos/7

Then have something like this in the image.yaml:

  modules:
    repolsitories:
    - name: platform-repos
      path: platforms/centos/7

Overrides would simply change the path to whatever is required.

Just thinking out loud.

[goldmann]

I like very much this idea! Overriding the entire repositories set would be accomplished by overriding a path to module. Very clean.

I guess in most cases this will not be required (see the current CentOS/RHEL 7 OpenJDK image), but in cases where we need fine grained control over repositories then @rcernich suggestion is a great fit.

I think this deserves a blog post ;)

[goldmann]

Related: #193.

[jmtd]

I've hit this again (happy anniversary bug! Nearly ;)), this time the situation is: I have a test image that requires content_sets to build in OSBS, + some extra packages via ODCS compose, but I need to disable the content_sets in order to build locally. Attempting something like --overrides '{{"packages": { "content_sets_file": "~" }}' it seems that no changes are made after processing.

[jmtd]

Aha, in this case I was quoting "~" which made it a string, not a null.