make Openstack security group configurable

[MoritzKeppler]

What do you think about making the security group of a runner instance configurable analogously to OPENSTACK_PRIVATE_NETWORK in the OpenStack provider?

[gabriel-samfira]

I think that's a great idea. The sample external providers in the garm repo are mostly meant as examples of how to build a provider. External providers can be extended and can implement whatever functionality they wish. They can even have their own separate state if they want. As long as they implement the expected interface, the sky is the limit.

When I get some time, I was thinking of adding something like "Extra Specs" to the pools. An opaque blob, which means nothing to garm but which may be interpreted by the external provider. This can be used as a way to pass per-pool (non-sensitive - ie: no credentials/secrets of any kind) configuration options to the provider in whatever format makes sense to the provider itself (json, base64, etc).

[MoritzKeppler]

Makes sense, thanks!
As an interim solution I added the following lines to the Openstack provider, not elegant but effective for the time being:

   SECURITY_GROUP=""
    if [ -n "${OPENSTACK_SECURITY_GROUP}" ];then
        SECURITY_GROUP="--security-group=${OPENSTACK_SECURITY_GROUP}"
    fi
    SRV_DETAILS=$(openstack server create --os-compute-api-version 2.52 ${SOURCE_ARGS} ${TAGS} ${PROPERTIES} --flavor "${FLAVOR}" --user-data="${CC_FILE}" --network="${NET}" ${SECURITY_GROUP} "${INSTANCE_NAME}")

[gabriel-samfira]

Hi @MoritzKeppler !

I created a PR here: #71

It adds a new option to pools called "extra specs". This is an opaque json you can attach to a pool and which will be sent via instance bootstrap parameters to the external providers like so:

{
  "name": "testing-vH3UP62C1vwj",
  "tools": [
      "... removed for clarity ..."
  ],
  "repo_url": "https://github.com/gsamfira",
  "callback-url": "https://garm.example.com/api/v1/callbacks/status",
  "metadata-url": "https://garm.example.com/api/v1/metadata",
  "instance-token": "super secret",
  "extra_specs": {
    "security_groups": [
      "default",
      "allow_web"
    ]
  },
  "ca-cert-bundle": null,
  "arch": "amd64",
  "flavor": "m1.small",
  "image": "focal-amd64",
  "labels": [
    "ubuntu",
    "self-hosted",
    "x64",
    "linux",
    "org",
    "runner-controller-id:f9286791-1589-4f39-a106-5b68c2a18af4",
    "runner-pool-id:dd4b40a7-eab5-479b-a4f9-789db092afbe"
  ],
  "pool_id": "dd4b40a7-eab5-479b-a4f9-789db092afbe"
}

To add extra specs to a pool you can simply:

ubuntu@garm:~/garm$ garm-cli pool update \
        --extra-specs='{"security_groups": ["default", "allow_web"]}'  \
        dd4b40a7-eab5-479b-a4f9-789db092afbe

of if you prefer to use a file:

ubuntu@garm:~/garm$ garm-cli pool update \
        --extra-specs-file=$HOME/extra-specs.json  \
        dd4b40a7-eab5-479b-a4f9-789db092afbe

This will allow you to modify your external provider to take into account the possibility of some config options passed in on a per-pool basis.

Note: this is not meant for secrets. Secrets should stay in the external provider config file.

[MoritzKeppler]

great, thanks a lot! Didn't have the time to read every line of code so far, but we will try it out soon and rewrite our external provider.

[gabriel-samfira]

Sounds good. After you set extra specs like in the above example, it should be as easy as:

function getSecurityGroupOptions() {
    OPTS=""
    DEFAULT_SEC="--security-group default"
    EXTRA_SPECS=$(echo "$INPUT" | jq -c -r '.extra_specs')
    checkValNotNull ${EXTRA_SPECS} "" > /dev/null 2>&1
    if [ $? -ne 0 ]; then
        echo $DEFAULT_SEC
        return 0
    fi
    SEC_GROUPS=$(echo "$EXTRA_SPECS" | jq -c -r '.security_groups')
    checkValNotNull ${SEC_GROUPS} "" >/dev/null 2>&1
    if [ $? -ne 0 ]; then
        echo $DEFAULT_SEC
        return 0
    fi

    for i in $(echo $SEC_GROUPS | jq -c -r '.[]');do
        OPTS+="--security-group $i "
    done
    echo "${OPTS}"
}

which should give you:

--security-group default --security-group allow_web

if set and:

--security-group default

if no extra specs or security_groups was set in the extra specs. Of course, as this is an opaque blob, you can add any extra specs with whichever name fits your needs.

Let me know how it works out 😄

[gabriel-samfira]

We have a new external OpenStack provider here: https://github.com/cloudbase/garm-provider-openstack

This provider supports customizing security groups via extra_specs.