RFC: evaluation of arbitrary scripts

[jdidion]

This proposal is for an alternative to user-defined functions. Instead of creating a separate top-level element for UDFs, we instead add an eval function that evaluates scripts in any language supported by the runtime environment.

Signature:

Any eval(String script, [String interpreter[, Object context]])

The eval function accepts, at a minimum, a script in an arbitrary language. The caller may specify the interpreter in the second argument; if the interpreter is not specified, the default interpreter is used. We may define the default interpreter in the spec (bash is the obvious choice), or it may be discovered from a runtime attribute (e.g. #408). The optional third argument is an object value (e.g. an object or struct literal) that provides a context for evaluating the expression. For example:

task expr {
  input {
    String firstName
  }
  String intro = eval('f"my name is {name}"', "python3.6", object { name: firstName })
  command <<< echo Hi ~{name}! >>>
  output {
    String greeting = read_string(stdout()) 
    String convo = "${intro}\n${greeting}"
  }
}

Similar to as in the UDFs proposal, the implementer can decide how to evaluate expressions. For example, all interpreters used could be required to be present in the user-provided container, or the runtime could evaluate every expression in a different subprocess in a different container that may be provided the runtime.

[illusional]

I don't know if this fits here, I think it would be more useful to run a task, without having to declare a whole step to run it. Because this is sort of just syntactic sugar. I can build a task that runs a script in Python, and by doing that ensure I'm portable across all engines that accept WDL.

For example, if I wanted to upper case all letters of a string, I'd want something like eval task to avoid :

import { toUpper } from './utils.wdl'

workflow foo {
  inputs {
    String myString
  }
  call SomeTask {
    myInput=toUpper(myString)
  }
  # ...
}

Then I could declare some task that does this.

task expr {
  input {
    String inp
  }
  command <<< python -c 'print("~{inp}".upper())'>>>
  output {
    String out = read_string(stdout())
  }
}

[jdidion]

I think what you're describing is more like what I proposed here: #407. I think there are pros and cons to both approaches. And there's nothing to say we couldn't do both.