path: root/doc/ccwl.texi

\input texinfo
@settitle Concise Common Workflow Language

@copying
Copyright @copyright{} 2021 Arun Isaac@*

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.  A
copy of the license is included in the section entitled ``GNU Free
Documentation License''.
@end copying

@ifnottex
@node Top
@top Concise Common Workflow Language
@end ifnottex

This manual documents the development version of @abbr{ccwl, Concise
Common Workflow Language}, a concise syntax to express @abbr{CWL, Common
Workflow Language} workflows.

@menu
* Introduction:: What is ccwl?
* Tutorial:: A quick tutorial to get started with ccwl
* Contributing:: Contributing
@end menu

@node Introduction
@chapter Introduction

@abbr{CWL, Common Workflow Language} is an open standard for describing
analysis workflows and tools in a way that makes them portable and
scalable across a variety of software and hardware environments.

@abbr{ccwl, Concise Common Workflow Language} is a concise syntax to
express CWL workflows. It is implemented as an @abbr{EDSL, Embedded
Domain Specific Language} in the Scheme programming language, a
minimalist dialect of the Lisp family of programming languages.

ccwl is a compiler to generate CWL workflows from concise descriptions
in ccwl. In the future, ccwl will also have a runtime whereby users can
interactively execute workflows while developing them.

@node Tutorial
@chapter Tutorial

This tutorial will introduce you to writing workflows in ccwl. Some
knowledge of CWL is assumed. To learn about CWL, please see the
@url{https://www.commonwl.org/user_guide/, Common Workflow Language User
Guide}.

@menu
* Important concepts:: Static typing, multiple named inputs and outputs
* First example:: Our first ccwl workflow
* Capturing stdout:: Capturing the standard output stream of a command
@end menu

@node Important concepts
@section The CWL and ccwl workflow languages

The CWL and ccwl workflow languages are statically typed programming
languages where functions accept multiple named inputs and return
multiple named outputs. Let's break down what that means.

@subsection Static typing

In CWL, the type of arguments accepted by a function and the type of
outputs returned by that function are specified explicitly by the
programmer, and are known at compile time even before the code has been
run. Hence, we say that it is statically typed.

@subsection Positional arguments and named arguments

In many languages, the order of arguments passed to a function is
significant. The position of each argument determines which formal
argument it gets mapped to. For example, passing positional arguments in
Scheme looks like

@lisp
(foo 1 2)
@end lisp

In a language that supports named arguments, the order of arguments is
not significant. Each argument explicitly names the formal argument it
gets mapped to. For example, in Scheme, passing named arguments may look
like

@lisp
(foo #:bar 1 #:baz 2)
@end lisp

@subsection Multiple function arguments and return values

In most languages, functions accept multiple input arguments but only
return a single output value. However, in CWL, a function can return
multiple output values as well. These multiple outputs are unordered and
are each addressed by a unique name.

@node First example
@section First example

As is tradition, let us start with a simple ``Hello World'' workflow in
ccwl. This workflow accepts a string input and prints that string.

@lisp
(define print
  (command #:run "echo" (input 'message #:type 'string)))

(workflow ((message #:type string))
  (print #:message message))
@end lisp

The first form in this code defines the @code{print} command. This form
is the equivalent of defining a @code{CommandLineTool} class workflow in
CWL. All arguments after @code{#:run} specify the command that will be
run. One of those arguments @code{(input 'message #:type 'string)}
refers to a @code{string} type input named @code{message}. Notice how
the command definition is very close to a shell command, only that it is
slightly annotated with inputs and their types.

The second form describes the actual workflow and is the equivalent of
defining a @code{Workflow} class workflow in CWL. The form
@code{((message #:type string))} specifies the inputs of the
workflow. In this case, there is only one input---@code{message} of type
@code{string}. The body of the workflow specifies the commands that will
be executed. The body of this workflow executes only a single
command---the @code{print} command---passing the @code{message} input of
the workflow as the @code{message} input to the @code{print} command.

If this workflow is written to a file @file{hello-world.scm}, we may
compile it to CWL by running

@example
$ ccwl compile hello-world.scm
@end example

This prints a big chunk of generated CWL to standard output. We have
achieved quite a lot of concision already! We write the generated CWL to
a file and execute it using @code{cwltool} as follows. The expected
output is also shown.

@example
$ ccwl compile hello-world.scm > hello-world.cwl
$ cwltool hello-world.cwl --message "Hello World!"
[workflow ] start
[workflow ] starting step echo
[step echo] start
[job echo] /tmp/zprgn3x0$ echo \
    'Hello World!'
Hello World!
[job echo] completed success
[step echo] completed success
[workflow ] completed success
@{@}
Final process status is success
@end example

@node Capturing stdout
@section Capturing the standard output stream of a command

Let us return to the ``Hello World'' example in the previous
section. But now, let us capture the standard output of the @code{print}
command in an output object. The ccwl code is the same as earlier with
only the addition of an @code{stdout} type output object to the command
definition.

@lisp
(define print
  (command #:run "echo" (input 'message #:type 'string)
           #:outputs (output 'printed-message #:type 'stdout)))

(workflow ((message #:type string))
  (print #:message message))
@end lisp

Let's write this code to a file @file{capture-stdout.scm}, generate CWL,
write the generated CWL to @file{capture-stdout.cwl}, and run it using
@code{cwltool}. We might expect something like the output below. Notice
how the standard output of the @code{print} command has been captured in
the file @file{51fe79d15e7790a9ded795304220d7a44aa84b48}.

@example
$ ccwl compile capture-stdout.scm > capture-stdout.cwl
$ cwltool capture-stdout.cwl --message "Hello World!"
[workflow ] start
[workflow ] starting step print
[step print] start
[job print] /tmp/7zksx3xm$ echo \
    'Hello World!' > /tmp/7zksx3xm/51fe79d15e7790a9ded795304220d7a44aa84b48
[job print] completed success
[step print] completed success
[workflow ] completed success
@{
    "printed-message": @{
        "location": "file:///home/manimekalai/51fe79d15e7790a9ded795304220d7a44aa84b48",
        "basename": "51fe79d15e7790a9ded795304220d7a44aa84b48",
        "class": "File",
        "checksum": "sha1$a0b65939670bc2c010f4d5d6a0b3e4e4590fb92b",
        "size": 13,
        "path": "/home/manimekalai/51fe79d15e7790a9ded795304220d7a44aa84b48"
    @}
@}
Final process status is success
@end example

@node Contributing
@chapter Contributing

ccwl is developed on GitHub at
@url{https://github.com/arunisaac/ccwl}. Feedback, suggestions, feature
requests, bug reports and pull requests are all welcome. Unclear and
unspecific error messages are considered a bug. Do report them!

@bye