feat: install dependencies in .tool-versions order #1723

chrisjpalmer · 2024-02-22T05:18:01Z

UPDATE

Summary

This PR changes the behaviour of asdf install to install tools in the order they are listed in .tool-versions.

The purpose is to prevent installation failures due to dependencies between plugins. An example is the poetry python plugins. When running asdf install for these, the poetry plugin will typically fail because python is not available (see here). This requires the user to manually run asdf install python xxx first and then continue with the rest of the install.

Rather than maintain the correct plugin order as part of the tool, this responsibility can be forwarded to users of the tool via .tool-versions file. asdf install will respect the order listed in .tool-versions when installing the tools.

Other Information

Tests have been written to verify the new behaviour works. All the old tests are passing locally except for two, but they seem unrelated.
- asdf update --head should checkout the master branch
- plugin_list_all should sync repo when check_duration set to 0
The behaviour of the tool is to start in the deepest directory (which is the cwd) and recurse upwards to shallower directories.
- install tools in the .tool-versions file one by one
- install any tools from legacy file versions if a plugin is installed for one, and it didn't appear in .tool-versions file
- continue to recurse to shallower directories until a tool for every plugin has been installed

Example

Given the following .tool-versions:

nodejs 18.16.1
python 3.10.13
poetry 1.4.2

before

asdf install

kubectl 1.24.16 is already installed
nodejs 18.16.1 is already installed
No preset version installed for command python3
Please install a version by running one of the following:

asdf install python 3.10.13

or add one of the following versions in your config file at /Users/xxx/workspace/service-developer-portal/.tool-versions
python 3.10.2

Cleanup: Something went wrong!

48 /Users/xxx/.asdf/plugins/poetry/bin/install: POETRY_HOME=$install_path python3 - --version "$version" $flags

after

asdf install

nodejs 18.16.1 is already installed
python-build 3.10.13 /Users/xxx/.asdf/installs/python/3.10.13
python-build: use openssl from homebrew
python-build: use readline from homebrew
Downloading Python-3.10.13.tar.xz...
-> https://www.python.org/ftp/python/3.10.13/Python-3.10.13.tar.xz
Installing Python-3.10.13...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
Installed Python-3.10.13 to /Users/xxx/.asdf/installs/python/3.10.13
asdf: Warn: You have configured asdf to preserve downloaded files (with always_keep_download=yes or --keep-download). But
asdf: Warn: the current plugin (python) does not support that. Downloaded files will not be preserved.
Retrieving Poetry metadata

# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.

It will add the `poetry` command to Poetry's bin directory, located at:

/Users/xxx/.asdf/installs/poetry/1.4.2/bin

You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.

Installing Poetry (1.4.2): Done

Poetry (1.4.2) is installed now. Great!

To get started you need Poetry's bin directory (/Users/xxx/.asdf/installs/poetry/1.4.2/bin) in your `PATH`
environment variable.

Add `export PATH="/Users/xxx/.asdf/installs/poetry/1.4.2/bin:$PATH"` to your shell configuration file.

Alternatively, you can call Poetry explicitly with `/Users/xxx/.asdf/installs/poetry/1.4.2/bin/poetry`.

You can test that everything is set up by executing:

`poetry --version`

Configuring poetry to behave properly with asdf ...
Running: "poetry config virtualenvs.prefer-active-python true".

('Configuration file exists at /Users/xxx/Library/Application Support/pypoetry, reusing this directory.\n\nConsider moving TOML configuration files to /Users/xxx/Library/Preferences/pypoetry, as support for the legacy directory will be removed in an upcoming release.',)
asdf: Warn: You have configured asdf to preserve downloaded files (with always_keep_download=yes or --keep-download). But
asdf: Warn: the current plugin (poetry) does not support that. Downloaded files will not be preserved.
Either specify a tool & version in the command
OR add .tool-versions file in this directory
or in a parent directory

... old PR description

Summary

This PR allows asdf to install tools in the order they are listed in .tool-versions. It works by the command asdf install --ordered-install to be provided.

The purpose is to prevent installation failures due to dependencies between plugins. An example is the poetry python plugins. When running asdf install for these, the poetry plugin will typically fail because python is not available (see here). This requires the user to manually run asdf install python xxx first and then continue with the rest of the install.

The PR takes a simple approach to resolve this problem. Rather than maintain the correct plugin order as part of the tool, which is much too difficult, allow developers to maintain the order in.tool-versions file. Then make the asdf tool follow that order when --ordered-install is provided.

Other Information

I have not written any tests, nor have I done extensive testing to verify this doesn't break any old functionality.
I also have not addressed this change for multiple levels of .tool-versions files in higher directories.
I can look into these things, if people like this change and we think its worth pursuing. Do let me know.

Our company uses asdf extensively for streamlining developer workflows so we are interested in improving the tool if possible.

Thanks!

jfly

Excited to see this pr! Why even add the option? I'd vote for just making this the unconditional behavior.

chrisjpalmer · 2024-02-23T01:33:48Z

Excited to see this pr! Why even add the option? I'd vote for just making this the unconditional behavior.

I'm down for that, just thought its better to leave the option in there in case it breaks anything for people using the tool now.

HashNuke · 2024-02-23T02:28:44Z

@chrisjpalmer Thanks for this very useful PR + the notes on the tests and the scenarios not handled.

Ordered install can be the default behaviour.

But I have not given any thought to how we could handle order across .tool-versions files (especially $HOME/.tool-versions and ${pwd}/.tool-versions). Open to ideas.

HashNuke

Added notes in the comments but repeating here.

Very useful PR ❤️

Ordered install can be the default behaviour
Open to ideas to handle scenarios for situations where there is a .tool-versions in the $HOME and the current directory.
Tests can be added once we do the above.

HashNuke · 2024-02-23T02:33:31Z

Changed this to a draft PR since this is work in progress.

chrisjpalmer · 2024-02-27T03:21:55Z

Thanks Akash. This sounds good.

I have some initial thoughts on how we can support multiple .tool-versions files in higher directories.
First I wanted to confirm the current behvaviour.

Current Process

Based on some testing and reverse engineering the code, I can see that ASDF does the following steps:

Check $PWD/.tool-versions to see if there are any versions who don't have a corresponding plugin installed on the system. If there are any versions without a corresponding plugin, a message is issued and the command exits early.
- Note: Higher level directories with .tool-versions are not searched.
Iterate plugins in alpha order. For each plugin:
- Recursively search .tool-versions files for a corresponding version - starting with $PWD/.tool-versions, going up a directory each time.
- As soon as a match is found, install that version and move on to the next plugin

Let me know if you agree with the above analysis. Assuming I'm right, then the key takeaway is that the .tool-versions files are recursively searched and the default .tool-versions file is considered last.

New Process

To emulate the same behaviour whilst now considering the version order in .tool-versions I believe we need the following process:

Check $PWD/.tool-versions to see if there are any versions who don't have a corresponding plugin installed... (keep this the same)
Recursively scan .tool-versions files starting with $PWD/.tool-versions, going up a directory each time. For each .tool-versions file:
- Install versions in the order they appear in the file.
- Store the plugins we have installed in a variable to keep track of what we have already covered off - this allows us to skip version installs for same plugin versions as we traverse higher directories.
Search ends when one of two conditions is true:
- there are no more directories to search
- we have satisfied every plugin that is installed on the system

Just for sanity checking its important to note the following about this new process:

versions in earlier .tool-versions take precedence over ones in higher directories - this is the same as the previous process which resolves the version for each plugin in earlier .tool-versions files before searching higher
versions without a corresponding plugin will not contribute to the installation process. This is the same as the previous process because, the previous installation process is driven by the list of plugins that do exist and hence will also ignore versions that have no corresponding plugin.

Let me know what you think :) Chris

lib/functions/installs.bash

chrisjpalmer · 2024-02-28T06:04:10Z

Hi there. I have actually gone ahead and implemented this. Current status is:

basic code is in place
replaced old install process with ordered install process
no unit tests written
hand tested on my machine with .tool-version files
have not tested legacy install files

chrisjpalmer · 2024-03-03T21:56:14Z

Hey @HashNuke just checking in to see if you are happy with the progress on this PR ?

chrisjpalmer · 2024-03-06T04:52:02Z

Update

I ran the unit test suite... majority are passing!

I added a new unit test which verifies the ordered install behaviour:

install_command installs plugins in order

Note: these are failing (on my local) but I believe they are unrelated:

asdf update --head should checkout the master branch
plugin_list_all should sync repo when check_duration set to 0

For consideration

I was chatting with a colleague of mine, and we were thinking that in addition to having tools be installed in .tool-version order, it might be good to consider the directory structure as well when installing tools. Suppose you have the following structure:

/
  - .tool-versions: python xxx
       /subdir
          - .tool-versions: poetry yyy

Currently in both the previous and new install process, poetry will be installed before python because asdf install installs tools in the deepest directory and the recurses upwards to shallower directories. However, there is an argument that it might be more useful to do this the opposite way. In a monorepo for example a user may specify a general tool in the root directory, and then sub directories could be free to declare additional tools. It may be possible that those additional tools have a dependency on tools declared in the root directory. Therefore it could make sense to actually modify asdf install's behaviour to recurse from the shallower directory downwards.

Thoughts on this ?

Thanks

chrisjpalmer · 2024-03-07T23:01:34Z

Hi there guys... I think this is ready for review now :)

chrisjpalmer · 2024-03-07T23:05:27Z

lib/functions/installs.bash

+ tools_installed=$(_install_directory_tools "" "$ASDF_DEFAULT_TOOL_VERSIONS_FILENAME" "$plugins_installed" "$tools_installed")
+ display_debug "install_directory_tools_recursive '$ASDF_DEFAULT_TOOL_VERSIONS_FILENAME': install_directory_tools returned tools_installed='$tools_installed'"
+ fi
+


^ regarding the absolute path stuff, there is no test for this, however this is behaviour that was in asdf already. I know because people at my work were using it like this.

The docs are clear this is only the filename of the file - https://asdf-vm.com/manage/configuration.html#asdf-default-tool-versions-filename

It is not an absolute path to a file and shouldn't be treated like it is here (lines 180-181 should be all that's needed). Was there any existing code that treated it like this?

I have removed this

when I tested asdf on of our company repos I noticed that we were setting ASDF_DEFAULT_TOOL_VERSIONS_FILENAME to an absolute path and it was working with asdf. Then I tested my new change and it obviously stopped working. It seems like it could have been one of those "accident by design" things.

chrisjpalmer · 2024-03-10T23:56:11Z

@HashNuke or @jfly would you mind taking a look to confirm you are happy with progress ?

Stratus3D · 2024-03-14T12:55:08Z

Check $PWD/.tool-versions to see if there are any versions who don't have a corresponding plugin installed on the system. If there are any versions without a corresponding plugin, a message is issued and the command exits early.
Note: Higher level directories with .tool-versions are not searched.

If this is the case I think this might be a bug in the old process (probably out of scope for this PR).

Recursively scan .tool-versions files starting with $PWD/.tool-versions, going up a directory each time. For each .tool-versions file:

Install versions in the order they appear in the file.
Store the plugins we have installed in a variable to keep track of what we have already covered off - this allows us to skip version installs for same plugin versions as we traverse higher directories.

For the new process we might actually want to start at the uppermost .tool-versions file, installing versions in order, and work our way down to the current directory. I feel like the home/root dir is typically used for system wide stuff that the user just happens to be using asdf for (ie version doesn't vary between dirs). This is definitely a nitpick. No need to implement right now.

Currently in both the previous and new install process, poetry will be installed before python because asdf install installs tools in the deepest directory and the recurses upwards to shallower directories. However, there is an argument that it might be more useful to do this the opposite way. In a monorepo for example a user may specify a general tool in the root directory, and then sub directories could be free to declare additional tools. It may be possible that those additional tools have a dependency on tools declared in the root directory. Therefore it could make sense to actually modify asdf install's behaviour to recurse from the shallower directory downwards.

Yep, exactly!

versions without a corresponding plugin will not contribute to the installation process. This is the same as the previous process because, the previous installation process is driven by the list of plugins that do exist and hence will also ignore versions that have no corresponding plugin.

Interesting, I wonder if that is best. I don't think we should be doing anything until all required plugins (ie all those referenced in all .tool-version files at and above the current dir) are installed. Again, definitely out of scope for this PR but I think it may have ended up this way by accident rather than by design. Any idea why we might want to silently ignore missing plugins at this step?

Stratus3D · 2024-03-14T13:28:12Z

lib/functions/installs.bash

+ # install tools from .tool-versions
+ # install order is the order listed in .tool-versions
+ file_name=$(asdf_tool_versions_filename)
+ tools_installed=$(_install_directory_tools "$search_path" "$file_name" "$plugins_installed" "$tools_installed")


I'm used to wrapping everything in quotes. Is there a reason the $() on these two lines aren't quoted? Like this:

file_name="$(asdf_tool_versions_filename)"

Stratus3D · 2024-03-14T13:31:10Z

lib/functions/installs.bash

+ return 0
+ fi
+
+ tool_versions=$(strip_tool_version_comments "$search_path/$file_name" | awk '{$1=$1};1')


Probably want this $() to be quoted right?

I am not sure to be honest. I noticed that most of the assignments in the installs.bash file do not put quotes around the $(). When I ran the linter, the linter pulled up times where I had passed a $() to a bash function if I didn't wrap it in quotes. It would give the following warning: SC2046 (warning): Quote this to prevent word splitting.. This makes me think that variable assignment is okay but potentially passing as arguments without wrapping in quotes is the one that isn't right. Not sure though, no expert on bash myself.

Typically if the value produced by the expression contains spaces and you want it to stay as a string you should quote - string_with_spaces="$(echo "string with spaces")". I think that is the case here, but I may be wrong. Generally, quoting is safer in Bash (unless you are dealing with an array, in which case it won't work).

lib/functions/installs.bash

Stratus3D · 2024-03-14T13:45:39Z

lib/functions/installs.bash

+display_debug() {
+ if [[ $DEBUG = "true" ]]; then
+ printf "debug: %s\n" "$1" >&2
+ fi
+}
+
+display_none() {
+ printf ""
+}


Might be good to put these in utils.sh.

Stratus3D

Thanks for the PR @chrisjpalmer ! Overall I think this is a good change.

Having been away from Bash for a little while I'm a little paranoid of unquoted variables and don't remember exactly when things should be quoted or unquoted. I'll test out this PR branch on my machine today and see how it goes.

…e path

chrisjpalmer · 2024-03-18T03:07:11Z

Thanks @Stratus3D for your review. Appreciate that time you put into it. I have tried to address all your comments and respond to anything that I still wasn't sure about.

Check $PWD/.tool-versions to see if there are any versions who don't have a corresponding plugin installed on the system. If there are any versions without a corresponding plugin, a message is issued and the command exits early.
Note: Higher level directories with .tool-versions are not searched.

If this is the case I think this might be a bug in the old process (probably out of scope for this PR).

Quite possibly. We can always follow this PR up with another one which hardens this behaviour.

Recursively scan .tool-versions files starting with $PWD/.tool-versions, going up a directory each time. For each .tool-versions file:
Install versions in the order they appear in the file.
Store the plugins we have installed in a variable to keep track of what we have already covered off - this allows us to skip version installs for same plugin versions as we traverse higher directories.

For the new process we might actually want to start at the uppermost .tool-versions file, installing versions in order, and work our way down to the current directory. I feel like the home/root dir is typically used for system wide stuff that the user just happens to be using asdf for (ie version doesn't vary between dirs). This is definitely a nitpick. No need to implement right now.

Currently in both the previous and new install process, poetry will be installed before python because asdf install installs tools in the deepest directory and the recurses upwards to shallower directories. However, there is an argument that it might be more useful to do this the opposite way. In a monorepo for example a user may specify a general tool in the root directory, and then sub directories could be free to declare additional tools. It may be possible that those additional tools have a dependency on tools declared in the root directory. Therefore it could make sense to actually modify asdf install's behaviour to recurse from the shallower directory downwards.

Yep, exactly!

I would be happy to chase this up in a follow up PR.

versions without a corresponding plugin will not contribute to the installation process. This is the same as the previous process because, the previous installation process is driven by the list of plugins that do exist and hence will also ignore versions that have no corresponding plugin.

Interesting, I wonder if that is best. I don't think we should be doing anything until all required plugins (ie all those referenced in all .tool-version files at and above the current dir) are installed. Again, definitely out of scope for this PR but I think it may have ended up this way by accident rather than by design. Any idea why we might want to silently ignore missing plugins at this step?

Ah I think when I said that "a version without a corresponding plugin will not contribute to the installation process" what I meant was, ignoring the sanity check that runs before installation kicks off, the main installation loop ignores unknown plugins. However there is a sanity check that runs before the main installation which lets the user know if it detects an unknown plugin. This is only in place for the .tool-versions file in the current directory. Again suggests to me a follow up PR to harden this behavior.

Stratus3D · 2024-03-26T01:41:49Z

lib/functions/installs.bash

- done
+ plugins_installed=$(printf "%s" "$plugins_installed" | tr "\n" " " | awk '{$1=$1};1')
+ display_debug "install_local_tool_versions: plugins_installed='$plugins_installed'"
+ tools_installed=$(install_directory_tools_recursive "$search_path" "$plugins_installed")


I think the $() on lines 122 and 124 should be quoted.

Stratus3D · 2024-03-26T01:43:45Z

lib/functions/installs.bash

+ while [ "$search_path" != "/" ]; do
+ # install tools from files in current directory
+ display_debug_hr
+ tools_installed=$(install_directory_tools "$search_path" "$plugins_installed" "$tools_installed")


quote this one too I think.

Stratus3D · 2024-03-26T01:44:19Z

lib/functions/installs.bash

+ fi
+
+ # go up a directory
+ search_path=$(dirname "$search_path")


This one as well. paths can contain spaces but they should be treated like a single string.

Stratus3D · 2024-03-27T01:05:41Z

lib/functions/installs.bash

+ local legacy_config
+ legacy_config=$(get_asdf_config_value "legacy_version_file")
+ if [ "$legacy_config" = "yes" ]; then
+ tools_installed=$(_install_directory_tools_legacy "$search_path" "$plugins_installed" "$tools_installed")


Quote $() here too I think.

Stratus3D · 2024-03-27T01:06:26Z

lib/functions/installs.bash

+ for plugin_version in "${parts[@]:1}"; do
+ # install the version
+ display_none "$(install_tool_version "$plugin_name" "$plugin_version")"
+ tools_installed=$(printf "%s %s" "$tools_installed" "$plugin_name" | awk '{$1=$1};1')


Quote $() here too I think.

Stratus3D · 2024-03-27T01:06:54Z

lib/functions/installs.bash

+
+ # install the version
+ display_none "$(install_tool_version "$plugin_name" "$plugin_version")"
+ tools_installed=$(printf "%s %s" "$tools_installed" "$plugin_name" | awk '{$1=$1};1')


Quote here.

Stratus3D · 2024-03-27T01:07:17Z

lib/functions/installs.bash

+
+ # extract plugin legacy information
+ local plugin_path
+ plugin_path=$(get_plugin_path "$plugin_name")


Quote here. Path may contain whitespace.

Stratus3D · 2024-03-27T01:08:17Z

lib/functions/installs.bash

+
+ # extract plugin legacy filenames
+ local legacy_filenames=""
+ legacy_filenames=$("$legacy_list_filenames_script")


For good measure quote here. I can't imagine any legacy file names would contain whitespace since these are typically filenames chosen for software developers, but it won't hurt.

Stratus3D · 2024-03-27T01:09:37Z

lib/utils.bash

+
+display_none() {
+ printf ""
+}


What is display_none for?

Stratus3D · 2024-03-27T01:10:25Z

lib/utils.bash

@@ -156,6 +156,20 @@ display_error() {
 printf "%s\n" "$1" >&2
 }

+display_debug_hr() {
+ display_debug "--------------------------------------------------------------------------------------------------------------"
+}


It looks like we only use this function once. Are we certain we need it?

Stratus3D · 2024-03-27T01:15:02Z

Thanks @chrisjpalmer! just scanned through the code and left some comments.

I see our build is broken here, but it isn't due to your changes. I'll see about fixing it on master.

feat: support ordered install mode

7c7232c

chrisjpalmer requested a review from a team as a code owner February 22, 2024 05:18

chrisjpalmer mentioned this pull request Feb 22, 2024

install order should follow the order in .tool-versions #929

Open

jfly reviewed Feb 22, 2024

View reviewed changes

HashNuke requested changes Feb 23, 2024

View reviewed changes

HashNuke marked this pull request as draft February 23, 2024 02:33

feat: refine ordered install mode

05718d4

chrisjpalmer commented Feb 28, 2024

View reviewed changes

lib/functions/installs.bash Outdated Show resolved Hide resolved

chrisjpalmer commented Feb 28, 2024

View reviewed changes

lib/functions/installs.bash Outdated Show resolved Hide resolved

chrisjpalmer commented Feb 28, 2024

View reviewed changes

lib/functions/installs.bash Outdated Show resolved Hide resolved

Christopher Palmer added 3 commits February 29, 2024 15:15

chore: small refactor + bug fix

cf718b2

fix: put back old behavior of def tools filename

1bd6cad

chore: remove unused vars

6693274

Christopher Palmer added 7 commits March 6, 2024 09:47

fix: include environment overrides

b4d8306

chore: shorten install_directory_tools fn names

675dc7f

chore: check installed tools has all plugins

cea75a6

chore: remove todo about env var resolution

f6fd11a

fix: avoid use of banned commands

54379a5

chore: add unit test to verified ordered install

596a1ea

chore: fix installs.bash linting

0acb232

chrisjpalmer requested a review from HashNuke March 6, 2024 05:01

Christopher Palmer added 2 commits March 8, 2024 09:52

fix: install multiple versions of same plugin in tool versions

1e2d698

chore: remove unused some_tools_installed

f1a268e

chrisjpalmer commented Mar 7, 2024

View reviewed changes

chrisjpalmer changed the title ~~feat: support ordered install mode~~ feature: install dependencies in .tool-versions order Mar 14, 2024

chrisjpalmer changed the title ~~feature: install dependencies in .tool-versions order~~ feat: install dependencies in .tool-versions order Mar 14, 2024

Stratus3D reviewed Mar 14, 2024

View reviewed changes

lib/functions/installs.bash Outdated Show resolved Hide resolved

Stratus3D reviewed Mar 14, 2024

View reviewed changes

lib/functions/installs.bash Outdated Show resolved Hide resolved

Stratus3D reviewed Mar 14, 2024

View reviewed changes

Christopher Palmer added 5 commits March 18, 2024 13:23

fix: stringlist_contains use correct var name

c684c1a

chore: create display_debug_hr for easy hr output

3d97f31

chore: move display_ functions to utils.sh

53998e2

fix: avoid treating ASDF_DEFAULT_TOOL_VERSIONS_FILENAME as an absolut…

257fff6

…e path

chore: fix linting issues

8d0fc4f

chrisjpalmer requested a review from Stratus3D March 18, 2024 02:57

Stratus3D reviewed Mar 26, 2024

View reviewed changes

Stratus3D reviewed Mar 27, 2024

View reviewed changes

lib/utils.bash

display_none() {

printf ""

}

Copy link

Member

Stratus3D Mar 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is display_none for?

Stratus3D reviewed Mar 27, 2024

View reviewed changes

feat: install dependencies in .tool-versions order #1723

Are you sure you want to change the base?

feat: install dependencies in .tool-versions order #1723

Conversation

chrisjpalmer commented Feb 22, 2024 • edited

UPDATE

Summary

Other Information

Example

... old PR description

Summary

Other Information

jfly left a comment

Choose a reason for hiding this comment

chrisjpalmer commented Feb 23, 2024

HashNuke commented Feb 23, 2024

HashNuke left a comment

Choose a reason for hiding this comment

HashNuke commented Feb 23, 2024

chrisjpalmer commented Feb 27, 2024 • edited

Current Process

New Process

chrisjpalmer commented Feb 28, 2024

chrisjpalmer commented Mar 3, 2024

chrisjpalmer commented Mar 6, 2024 • edited

Update

For consideration

chrisjpalmer commented Mar 7, 2024

chrisjpalmer Mar 7, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chrisjpalmer Mar 18, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chrisjpalmer commented Mar 10, 2024

Stratus3D commented Mar 14, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chrisjpalmer Mar 18, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Stratus3D left a comment

Choose a reason for hiding this comment

chrisjpalmer commented Mar 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Stratus3D commented Mar 27, 2024 • edited

chrisjpalmer commented Feb 22, 2024 •

edited

chrisjpalmer commented Feb 27, 2024 •

edited

chrisjpalmer commented Mar 6, 2024 •

edited

chrisjpalmer Mar 7, 2024 •

edited

chrisjpalmer Mar 18, 2024 •

edited

Stratus3D commented Mar 14, 2024 •

edited

chrisjpalmer Mar 18, 2024 •

edited

Stratus3D commented Mar 27, 2024 •

edited