fix additional disk path support #1784

cornfeedhobo · 2023-10-15T23:43:07Z

The original goal was to add support for uploading an image, if :path pointed to a local file, specifically to support the combustion use case. Once I began adding unit tests, it was easier to also break out the additional disk handling to a dedicated action. This had the unfortunate side effect of making this PR rather large.

I've broken up the commits to be logical units of change, each with working tests at the moment of commit. I hope that makes it a little easier to review.

Functional tests can be performed with the following steps:

Creating the following Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.box = "opensuse/MicroOS.x86_64"
  config.vm.box_check_update = false
  config.vm.synced_folder ".", "/vagrant", disabled: 'true'
  config.vm.provider :libvirt do |libvirt|
    libvirt.driver = "kvm"
    libvirt.cpus = 2
    libvirt.cpu_mode = "host-passthrough"
    libvirt.memory = 4096
    libvirt.storage :file,
      :type => "raw",
      :path => "ignition.img"
  end
end

Generating an appropriate ignition.img file, which can be done using this javascript based generator.
```
mv ~/Downloads/ignition-*.img ./ignition.img
```
Finally you'll be able to use bundle to vagrant and ensure this branch is in use
```
bundle exec vagrant up
```

Fixes #259
Fixes #1783

- add sleep to snapshot test - disable box update checks in all Vagrantfiles

cornfeedhobo · 2023-10-24T18:50:27Z

@purpleidea @sciurus @infernix @electrofelix

I've seen you comment or work on related issues. If you have time to review this, I would appreciate it. Thanks.

purpleidea · 2023-10-24T22:28:28Z

@cornfeedhobo

I've seen you comment or work on related issues. If you have time to review this, I would appreciate it. Thanks.

I haven't hacked on it recently, but until vagrant is under a real Free Software license, I'm definitely out. Sorry.

cornfeedhobo · 2023-10-25T00:00:03Z

@purpleidea I hear you, but you did say to ping you for a review for this one specific case 😅 🙏

purpleidea · 2023-10-25T00:25:13Z

haha! So true! Well it was 2014 so things have changed slightly. I can help you with an alternative idea for rapid virt provisioning if you are interested. Ping me elsewhere if you want to chat about it. Cheers

github-actions · 2023-10-29T19:30:02Z

A docker image containing the code from this plugin to allow testing locally without installing
can be pulled from: ghcr.io/vagrant-libvirt/vagrant-libvirt:pr-1784-slim

If you need the image with the full dev toolchain, you can instead pull: ghcr.io/vagrant-libvirt/vagrant-libvirt:pr-1784

cornfeedhobo · 2023-12-25T01:45:35Z

@sciurus @infernix @electrofelix Any hope?

electrofelix · 2023-12-25T09:13:03Z

@cornfeedhobo right now I'm pretty slammed with two young children under 2 years of age, and I don't expect to have time for any projects for at least another 3 months, though could be longer.

Given time constraints, as I hopefully start to get a little time in the future, smaller PRs will be picked up first just due most likely they are the ones I can review in the time available. I don't want to suggest splitting this PR into a load of separate PRs as that could be a lot of work and I may still not be able to review them in a timely manner. Though maybe there is one commit you can push up as a separate PR that is nice and small and I can try with that. Keep this one up to date and then we can try and iterate through the individual commits as PRs. I will try to look at just one or two of the early commits to do a first pass there first to see if I can manage it from my phone before you do a chunk of work.

I thought there was an option for triggering the remaining workflows to ensure that you'd have a gem published to the GitHub rubygem packages and could point users to the instructions to install from it for a while, not sure what is going on there.

I can give some general comments about what I prefer to see:.

Avoid adding more references to fog in variable names. At some point the backend should be abstracted to allow rewriting to use xmlrpc or possibly golang to allow a conversion. It makes sense to avoid saying something is a fog volume if what will create it in the future will change. Where just moving exisiting code around that already has these, don't worry about them, it's just new references.
Try to avoid moving code to send messages to the UI into until functions, best to keep these within the action calling. Not always possible, but generally preferred as then the actions can decide what to display. Typically I'd think a callback for those pieces that can't be moved where the default is to pass an empty object, test where it is UI type, and only write something like progress to it in that case.

electrofelix · 2023-12-25T09:55:36Z

spec/acceptance/snapshots_spec.rb

@@ -31,6 +31,10 @@
 status('Test: restore snapshot')
 expect(environment.execute('vagrant', 'snapshot', 'restore', 'test')).to exit_with(0)

+ # KVM needs a moment for IO to catch up.
+ # If anyone comes up with a better way to wait, please update this.
+ sleep(3)


I suspect an email to get libvirt devs would be needed. I assumed that revert was blocking, but perhaps that doesn't cover flushing or the state being fully switched. Hopefully they could enlighten on what would need to be done after the revert call to confirm the revert is fully completed.

This should be good enough for the tests for now though.

electrofelix · 2023-12-27T22:00:17Z

spec/acceptance/support-skeletons/additional_storage/Vagrantfile

@@ -5,6 +5,7 @@

 Vagrant.configure("2") do |config|
 config.vm.box = "infernix/tinycore"
+ config.vm.box_check_update = false


Meant to ask as well, how does this help with stability? Or is it just handy to disable when running a lot?

Generally I'm thinking that the CI here should be using the latest images rather than those that it was run with the first time. There should be caching in place to avoid this costing a lot of time by default, bit of it's useful to disable in certain cases I could see using an env var instead being better.

I was curious if this was intentional. Maybe I can switch to using an environment variable to toggle? Either way you hit the nail on the head - it was painful when running the test suite dozens of times in a day.

I'll put some thinking into this one.

cornfeedhobo force-pushed the add-disk-path-support branch 2 times, most recently from 9bed06f to 678547e Compare October 16, 2023 12:25

cornfeedhobo added 3 commits October 23, 2023 21:48

make acceptance tests more stable

c67d69d

- add sleep to snapshot test - disable box update checks in all Vagrantfiles

move storage fuctions to storage_util

18f1780

move additional disk handling to dedicated action

456f029

cornfeedhobo force-pushed the add-disk-path-support branch from 678547e to dd7bfce Compare October 24, 2023 17:37

cornfeedhobo marked this pull request as ready for review October 24, 2023 17:44

cornfeedhobo force-pushed the add-disk-path-support branch from dd7bfce to 27fe383 Compare October 24, 2023 18:46

add support for uploading an existing disk image

726083d

cornfeedhobo force-pushed the add-disk-path-support branch from 27fe383 to 726083d Compare October 29, 2023 19:27

electrofelix reviewed Dec 25, 2023

View reviewed changes

electrofelix reviewed Dec 27, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix additional disk path support #1784

fix additional disk path support #1784

cornfeedhobo commented Oct 15, 2023 •

edited

cornfeedhobo commented Oct 24, 2023 •

edited

purpleidea commented Oct 24, 2023

cornfeedhobo commented Oct 25, 2023 •

edited

purpleidea commented Oct 25, 2023

github-actions bot commented Oct 29, 2023

cornfeedhobo commented Dec 25, 2023

electrofelix commented Dec 25, 2023

electrofelix Dec 25, 2023

electrofelix Dec 27, 2023

cornfeedhobo Dec 29, 2023

fix additional disk path support #1784

Are you sure you want to change the base?

fix additional disk path support #1784

Conversation

cornfeedhobo commented Oct 15, 2023 • edited

cornfeedhobo commented Oct 24, 2023 • edited

purpleidea commented Oct 24, 2023

cornfeedhobo commented Oct 25, 2023 • edited

purpleidea commented Oct 25, 2023

github-actions bot commented Oct 29, 2023

cornfeedhobo commented Dec 25, 2023

electrofelix commented Dec 25, 2023

electrofelix Dec 25, 2023

Choose a reason for hiding this comment

electrofelix Dec 27, 2023

Choose a reason for hiding this comment

cornfeedhobo Dec 29, 2023

Choose a reason for hiding this comment

cornfeedhobo commented Oct 15, 2023 •

edited

cornfeedhobo commented Oct 24, 2023 •

edited

cornfeedhobo commented Oct 25, 2023 •

edited