CPU client job links format needs to be specified and enforced

[rvencu]

As for the string format sent by CPU workers, we need to specify how to use it. Otherwise, GPU nodes cannot understand all formats people invent to create those links.

Weblinks are fine and universal, however, it means we need to add some webserver to the workers and create a REST API to be able to send commands to the worker (like delete the job I just downloaded from you)

Going rsync or other ssh derivative is too risky in this scenario

So I would say we need to impose this and make the CAH client refuse malformed strings

[TheoCoombes]

I believe the webserver is our best bet in this case. We can use the same URL for all requests, just changing the method:

Download = GET
Upload = POST / PUT
Delete = DELETE

What do you think? :)

[TheoCoombes]

Also I forgot to mention, there is another method for GPU jobs, which flags a URL as invalid: client.invalidURL(). This deletes the job from the server, moving it back into open jobs.

[rvencu]

ref invalidURL() please allow sending the actual string that is invalid. it might be useful for clients requesting multiple jobs at once

[rvencu]

After setting up the rsync targets my opinion is that we should use that for partial work transfer from CPU to GPU because it appears to be the fastest method by far

in such case I am generating an uuid in hex format like this 1b9437eb5f5144b9aeeae13d85909bb2 and I can pass this string to jobComplete function.

sending and receiving files via rsync it trivial we just need to append the above string as folder identification to the dedicated rsync target

also rsync can take care of deleting source files for successful transfers