Passing configurations from runLocally to main process

[boesing]

Hey there,

I am currently working on some kind of SSH hardening during deployment.
We do require TOTP for deployments and therefore, I thought this flow would be nice:

The task-tree for deploy:
├── remote:auth-verify  // before bundle:remote:auth
├── remote:auth  // before bundle:deploy
├── remote:auth-verify  // before bundle:remote:test-connections
├── remote:test-connections  // after bundle:remote:auth
├── deploy
│   ├── info
│   ├── upload
│   ├── remote:prepare
│   ├── remote:activate:start  // before bundle:remote:activate
│   └──remote:activate
│   ├── remote:activate:end  // after bundle:remote:activate
├── remote:cleanup  // after bundle:deploy
├── annotate  // after bundle:deploy
│   └── annotate:grafana

So before deployment is being started, I do have remote:auth-verify which does:

try {
    $detectedProjectUser = run('whoami', [
        'tty' => false,
        'timeout' => 1,
    ]);
} catch (Throwable) {
     $detectedProjectUser = null;
}

// All good
if ($projectUser === $detectedProjectUser) {
    set('remote-authenticated', get('remote-authenticated', true)); // only restore value in case something is already persisted, this should keep `false` once set
    return;
}

// Detected user seems to vary from what we expect
if ($detectedProjectUser !== null) {
    throw new RuntimeException(sprintf(
        'SSH connection established but invalid user detected: %s',
        $detectedProjectUser,
    ));
}

set('remote-authenticated', false);

The "detected project user" is then compare against our expectation and if the value is null we do expect authentication failed (passing that information via remote-authenticated.

Then there is remote:auth:

should be skipped once remote-authenticated is true. Not 100% sure if that config variable will be available as it was set in one or more "hosts"

$token = retrieve_token();
set('TOTP_TOKEN', $token);

After remote:auth, we do want to verify if connections are actually available with bundle:remote:test-connections:

if (get('remote-authenticated') === true) {
    writeln('Connection(s) successfully initiaized.');
    return;
}

throw new RuntimeException('Unable to initialize connection(s).');

---

However, my main problem is, that I am unable to pass the retrieved TOTP_TOKEN from remote:auth to the main process or to the servers environment as it seems that it is already executed in a subshell.

With v6, there were local tasks which were executed on the host and thus passing these kind of variables worked quite well.

Is there any way how I can actually pass my variable from one of the tasks to another task or how I can make the whole "auth*" stuff running only on local machine?

[antonmedv]

You can use host configuration for this. Use set() and get() functions.

[Tristan-MyAnaPro]

If I understand correctly, you want to set a config from within a task once and have this config shared to all selected hosts ?

If this is the case, I had the same problem recently: a task which, depending on the selected hosts (which is why it need to be a task) return or ask user some informations shared to all selected hosts.

Here is a simplified example of how I solved it (@antonmedv maybe there is a better way ?):

task('taskname', function () {
    // Custom logic...
    [...]

    // Interesting part:
    $data = askChoice("Make a choice :", array_values($shellOutput));
    foreach (selectedHosts() as $host) {
        // Ask the master to call the `set` method on all the selected hosts 
        Deployer::proxyCallToMaster(Deployer::get()->hosts->get($host), '\Deployer\set', 'config-on-the-fly', $data);
    }
})->hidden()->once();

[antonmedv]

on(selectedHosts(), function () {
    $hostData = get('some-config');
});

[boesing]

not sure if I understand correct...
So the example of @Tristan-MyAnaPro was to set config variables.
The example with on(selectedHosts() is to retrieve a config variable?

How does this replace the example above? I dont get it but maybe I am just not deep enough in this tool...

[Tristan-MyAnaPro]

Just do

on(selectedHosts(), function () {
    set('some-config', 'some-value');
});

// Or with arrow function
on(selectedHosts(), fn() => set('some-config', $some-value));

The callback is executed in the context of each hosts so in this example the set is applied to all selected hosts.

[boesing]

I see. That seems to work now. Awesome.
What I am still unable to manage is how to pass environment variables to the "master" process so that subshells (started via Symfony\Process can access those values.

I do want to provide an environment variable so that the ssh "binary" (custom script) can pick that up so that it can decide whether to call sshpass or usual ssh binary (from OpenSSH).

I most probably spent 16 hours in try/error - most of the time was blocked by the server as it blocked me after 3 invalid tries. I was able to handle that the server is now accepting the password for an infinite amount of retries (until I finished proper handling).

I do set a variable using putenv (and also use on(selectedHosts(), fn () => putenv('FOO=bar')) (and I tried the proxy to master call) but whatever I do, the env var is not available when running runLocally('env | grep FOO') and therefore it is not available when executing the ssh command.

Pretty sure that was more easy with the local tasks in v6 as these were simply executed by the main process rather than from a subshell or whatever is actually blocking me. I almost gave up but the latest hints in this discussion at least gives me some ideas on how to proceed...

[antonmedv]

runLocally('echo $KEY', env: ['key' => 'value']);

https://deployer.org/docs/7.x/api#runlocally

[boesing]

I need to pass that environment key to the ssh process which does not pass any environment variables, at least as per code.
It only runs ssh <arguments> <command>.
I either need <env> ssh <arguments> <command> or a way to set these variables to the main process which invokes the ssh command.

When I use runLocally, a new subshell is spawned which then would execute export key=value. After closing that subshell, that environment variable is gone as well.
Thats why I need to run putenv in the master process from within a worker which seems not to work. I also tried to use on(selectedHosts(), fn () => putenv('foo=bar')); but that does not work either.

Most probably because that putenv is not called in the main process which is started once calling deployer.phar deploy all from CLI.

[antonmedv]

I see. Let me think.