Continuous integration

[greenled]

The script should be tested on each push in an easy, reproducible and auditable (public) way. A bonus benefit would be the ability to run tests against several Portainer versions to ensure compatibility.

[greenled]

See Portainer docs on setting admin password from the command line

[tortuetorche]

I'm planning to open source some code next month, for testing each push/commit through GitLab CI.
But there is a lot of work to do before achieving this goal, stay tuned 😉

[greenled]

That's great news! Looking forward for your code. BTW, I am currently using psu as part of a GitLab CI job (on premises instance). I am a big fan of GitLab.

[tortuetorche]

Hi @greenled,

I made some good progress on Continuous integration via GitLab CI.
In summary, it generates a Docker Swarm cluster with Traefik and Portainer, then we can run psu commands.
I'll open source my work in few days/weeks.

Have a good day,
Tortue Torche

[greenled]

That's great @tortuetorche. Let me know when it's out. I think we could use the GitLab's GitHub integration feature. Or maybe just move to gitlab.com?🤔

[tortuetorche]

Or maybe just move to gitlab.com?

💯

[tortuetorche]

Hi @greenled,

The continuous integration with GitLab CI is almost done!

Can you add me as a contributor collaborator of this project, then I'll import it in GitLab (with issues and pull requests) to the psuapp organization, please?

Have a good day,
Tortue Torche

[greenled]

Hi @tortuetorche. I just added you as a collaborator. Welcome onboard! Please import the project into the psuapp organization with issues, pull requests and everything else that can be imported. We are #movingToGitLab. Let's rock that CI pipelines!

I have some big news to share too. I have been working on a complete rewrite of the codebase...in Go. It's a radical change but I believe it's for the good, it will lean the way in the long run. Bash is great for scripts (the smaller, the better), but when you want something more complex (long AND short flags, commands and subcommands for context like in docker container ls, global and local flags, and so on) it's not so great anymore: you need to reinvent the wheel here and there and you end up with a hard to maintain codebase (maybe not for you, as I see you are fluent in Bash, but it it's not my case :). At that point you know it's time to find another tool that fits your needs. Go, for example, puts everything it needs in a single binary, so no more external dependencies, and cross compiles to most platforms and architectures, so no more "it works on my computer" (see #16). It also has great support for building CLI apps (see the Cobra project), with mentioned requirements and more fulfilled out of the box.

I feel very happy with the result. I brought most features from the bash version, implemented some of the ones you added to the next branch and I'm looking forward to implement the rest. Of course, if you happen to speak Go lang I encourage you to implement them by yourself (sorry for the double work 😅, I see you have been working hard in the next branch).

[tortuetorche]

Hi @greenled,

This is great news!
I understand the need to get a more solid code, but maybe we can release my next branch as the 1.0 version and your Go version as the 2.0 version, isn't it? Or simply create a new project for the Go version...
I spent a lot of times and effort on this.
You don't see all my work: CI, tests, online documentation... I'm going to push it to GitLab (private repository) so you could review it.
And of course, all this stuff can be used with the 2.0 version 👍

PS: I never coded in Go

Have a good day,
Tortue Torche

[greenled]

I like the first approach: releasing your *next * branch as a 1.x version and releasing the Go code as a 2.x version. This way all the contributions you made and the ones to come will stay in the project.

This is what I am thinking to do:

• Create a 1.x branch from the latest 0.x branch release and merge the next branch into it. You will be in charge of maintaining this 1.x branch and leading it to a stable state as you have been doing so far. There you can place the CI script, which I will probably reuse for the Go version.
• Make it clear in the docs there is a 1.x version in Bash and a 2.x version in Go, so everyone can pick the one which serves them better. As a user I would prefer the Go version but others may have valid reasons to prefer the Bash version, like using features which are not part of the Go version or tweaking the script.

What do you think?

BTW, if you have the time and interest I would highly recommend you learning Go. It's a beautiful and powerful language. I had never coded in go either, until I summed up the issues I described before.

Have a good day,
Juan Carlos

[tortuetorche]

Alright, but we should follow GitLab Flow and name stable branches like 1-0-stable instead of 1.x.

I can't import this repository to GitLab with issues and pull requests, because I'm not the owner...
So can you do it, please?
Go to https://gitlab.com/import/github/status, then select the psuapp organization and I suggest to name the GitLab repository psu instead of portainer-stack-utils.
Because with this new name, it's much easier to configure CI and Docker registry and also for legal reason I guess (Portainer is certainly a trademark)...

[tortuetorche]

Hi @greenled,

Any news to import this repository in GitLab with issues and pull request?
See my previous (and updated) messsage for more information...

Have a good day,
Tortue Torche

[greenled]

Hi @tortuetorche. I have got another idea:

• Revert the Go-related commits in master back to 3326a0.
• Merge your next branch into master.
• Cherry-pick Go-related commits into a new next branch.
• Keep working on each version.
• Eventually, release master as version 1.0.0, create branch 1-0-stable from master and merge next into master.

What do you think?

I have been thinking too about renaming the project (not only the namespace) to avoid possible trademarks issues. I opened #17 to address that. Does any name come to your mind? I think this should be done before migrating to GitLab, as group and repository name should reflect this change. Meanwhile you can still work on the CI script.

Regards,
Juan Carlos

[tortuetorche]

Hi @greenled,

OK, but all the links of my next branch are related to https://gitlab.com/psuapp/psu (which don't exist yet)

So I don't think it's a good idea to merge my branch before GitLab migration...
By the way, here a showcase of the GitLab project with all the merge and branches you asked in your last message: https://gitlab.com/psuapp/psuone (this is a private GitLab project, but I gave you full access)

Have a good day,
Tortue Torche

[greenled]

OK, I just started the import into psuapp/psu. It should not take long.

[greenled]

It's been importing the project for eight hours. I removed the project and reimported it, just in case.

[greenled]

Please let's keep the discussion about migration to GitLab on #19, this issue's topic is CI.

[greenled]

@tortuetorche I started reviewing in depth your latest changes to the next branch here in GitHub and the ones on CI and documentation in GitLab. To be honest, I have encountered feelings about them: I'm happy because you improved the project a lot, but I also have concerns about the way you reflect those improvements in the commit history.

Sometimes you batch several unrelated changes (feature additions, bug fixes, typo fixes, version number bumps...) into a single commit and name commits against a version number instead of a short description of what has been done, which is instead described on next lines of the commit message. This makes it hard for me to understand your work, and will make it hard too for anyone else who wants to read the project's history.

Please could you rewrite your next branch history taking this into account? See this commit guidelines for an example. After that please make a PR from tortuetorche/portainer-stack-utils:next to greenled/portainer-stack-utils:master, so any further discussion takes place there and not in this ci-related issue. Also leave the latest commits in the GitLab repo (d2994100..81b413fa) for other PRs, so we can iterate faster.

I apologize for this back, forward, back again. This is the first time I deal with large contributions and I have a lot to learn about the process, like adding a contribution guide.

[greenled]

As migration to GitLab #19 is not going as expected I am considering using another of the available CI solutions like Travis CI, Circle CI, Drone.io. Whatever is chosen, it should work like GitLab CI as much as possible, so future migration can be easier.