Welcome to wdl Discussions!

[patmagee]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[llangit-broad]

Hello - I did a project (contract) for The Broad - 'learn-WDL' - the idea was to create an open source Repo of WDL learning resources. I welcome feedback and/or PRs - https://github.com/openwdl/learn-wdl

I run my own consultancy, doing work as a Cloud Architect building high volume data pipelines for bioinformatics and other domains (finance...) on GCP, AWS or Azure.

[rhpvorderman]

Hello.

At our institute (Leiden University Medical Center) we transitioned to WDL in 2018. We had been using a framework based on GATK queue before. But we decided to move on to something that was more community-based and actively mantained. There were other contenders, most notably snakemake, but WDL won out because it allowed modularity in an easy way.

My colleagues and I set up BioWDL to provide reusable tasks for workflows as well as providing reusable sub-workflows and full-scale production workflows. In doing so we found some annoyances in the language and engines and we became active in the community.

Also we set up a test framework, pytest-workflow, which is workflow language agnostic to make sure all workflows and tasks in BioWDL are well-tested. Since it is workflow-agnostic, it can be used to promote the switch to WDL, as tests are easily portable across workflow systems.

Most of my contributions to the WDL community have been to make Cromwell easier to use on HPC backends. I also made a small utility wdl-packager to make it easier to redistribute wdl workflows without relying on git.

[illusional]

Hi!

I'm a software engineer at the University of Melbourne and Peter MacCallum Cancer Centre on a collaborative project to produce pipelines that can be run portably, and reproducibly across our institute's HPCs and cloud environments.

We investigated a number of potential workflow systems (including CWL, WDL, Nextflow, Snakemake), but were cautious to devote all our effort to one framework. Instead we produced Janis, a python framework for building pipelines that get translated to CWL, WDL, and soon, Nextflow. For us it provides a good abstraction for a literal "write once, run everywhere", and we've done some cool things like automatically generating documentation and CWL / WDL tool wrappers for tools in Janis (eg: BWA MEM) or automatically generating tool wrappers for GATK.

I like finding cool ways people run workflows, and share those ideas between communities to make workflows easier to configure and run.

[patmagee]

@illusional Thats pretty cool! I didn't know you guys were working on an abtraction layer! Is it possible to take a WDL or CWL and then translate between them, or do you need to start with your abstract model?

[illusional]

Thanks @patmagee! I’m pretty sure I can get part of the way there with a CWL or WDL. CWL has a less flexible structure but the JS expressions are trickier; where WDLs expressions are less flexible, but the tool definitions are harder to parse.