aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-11-18scripts: When ccwl crashes, say so explicitly.Arun Isaac
* scripts/ccwl (main): When ccwl crashes, say so explicitly.
2023-11-18ccwl: Never set other and binding fields of inputs/outputs to #f.Arun Isaac
Setting the other and binding fields to #f is a type violation. It messes with code that expects these fields to always contain lists. * ccwl/ccwl.scm (cwl-workflow): Set other and binding fields of inputs and outputs to the empty list.
2023-11-18ccwl: Report non-existent external CWL workflow files.Arun Isaac
cwl-workflow needs to be a macro so we have access to syntax objects necessary to raise a &cwl-violation condition. * ccwl/ccwl.scm (cwl-workflow): Rewrite as macro. Report error if CWL file does not exist.
2023-11-18ccwl: Allow nested workflows in workflow body.Arun Isaac
* ccwl/ccwl.scm (function-inputs, function-outputs, function-object): Support <workflow> objects as functions. (<workflow>): Move above function-outputs. * ccwl/cwl.scm (workflow->cwl-scm): Allow nested workflows.
2023-11-18ccwl: Restrict #:binding parameter to YAML serializable trees.Arun Isaac
* ccwl/ccwl.scm (ensure-yaml-serializable): Accept parameter name for &formatted-message condition. (input): Explicitly pass parameter name #:other. Restrict #:binding parameter to YAML serializable trees. * doc/capture-output-file-with-parameter-reference.scm (extract-specific-file), doc/capture-output-file.scm (extract), doc/decompress-compile-run.scm (compile): Pass an YAML serializable tree, not an expression, as the #:binding parameter.
2023-11-17ccwl: Report errors in rename mappings.Arun Isaac
* ccwl/ccwl.scm (collect-steps): Report errors in rename mappings.
2023-11-17utils: Define pairify behaviour on odd number of elements.Arun Isaac
* ccwl/utils.scm (pairify): Ignore extra elements when list has an odd number of elements. * tests/utils.scm ("pairify must ignore extra elements when list has an odd number of elements"): New test.
2023-11-17ccwl: Implement cross product scattering operations.Arun Isaac
* ccwl/ccwl.scm (collect-steps): Implement scatter-cross and scatter-nested-cross. (key->output): Nest output array types for nested cross product scatters.
2023-11-17ccwl: Recreate types using syntax.Arun Isaac
* ccwl/ccwl.scm (type->syntax): New function. (key->output): Use type->syntax.
2023-11-17ui: Colorize format specifiers not format arguments.Arun Isaac
* ccwl/ui.scm (report-formatted-message): Colorize format specifiers not format arguments. * tests/ui.scm ("report-formatted-message must not fail on arguments that are not strings"): New test.
2023-11-17yaml: Serialize atomic dictionary values on the same line.Arun Isaac
* ccwl/yaml.scm (display-dictionary-entry): Serialize atomic values on the same line. * tests/yaml.scm ("dictionary keys that map to an atomic value must be serialized on the same line"): New test.
2023-11-17doc: Add chapter and section identifiers.Arun Isaac
Identifiers are important so the generated HTML documentation can have permanent URIs to all chapters and sections. We are not too concerned with adding identifiers to subsections since we don't list them in the table of contents. * doc/ccwl.skb: Add chapter and section identifiers to all chapters and sections.
2023-11-16yaml: Comment on various cases handled by scm->yaml.Arun Isaac
* ccwl/yaml.scm (scm->yaml): Comment on cases handled.
2023-11-16yaml: Display arrays with a single atomic element on the same line.Arun Isaac
* ccwl/yaml.scm (atom?): New function. (display-dictionary-entry, scm->yaml): Display arrays with a single atomic element on the same line. * tests/yaml.scm ("strings with hyphen characters should be escaped", "strings with asterisk characters should be escaped"): Update tests with new behaviour. ("single element vectors must be serialized on the same line"): New test.
2023-11-16ccwl: Support nested arrays.Arun Isaac
* ccwl/ccwl.scm (construct-type-syntax): Construct types recursively to support nested arrays. (key->output): Recursively convert stdout types to File types. * ccwl/cwl.scm (type->cwl): New function. (input->cwl-scm, output->cwl-scm): Use type->cwl. * tests/ccwl.scm (make-array-type): New function. (construct-type-syntax-wrapper): New syntax. ("construct-type-syntax on primitive types", "construct-type-syntax on array types", "construct-type-syntax on nested array types"): New tests. * tests/cwl.scm, doc/array-types.scm, doc/nested-array-types.scm: New files. * doc/ccwl.skb (Cookbook)[Array types]: New section.
2023-11-14doc: Add scatter-gather snippet.Arun Isaac
I forgot to add this file in my previous commit. * doc/scatter-gather.scm: New file.
2023-11-14ccwl: Implement scatter.Arun Isaac
* ccwl/ccwl.scm (<step>): Wrap constructor make-step. [scattered-inputs, scatter-method]: New fields. (collect-scatter-step): New function. (collect-steps): Implement scatter. (key->output): If step scatters, convert output to an array type. (workflow): Add syntax to initialize scattered-inputs and scatter-method fields of <step> object. * ccwl/cwl.scm (workflow->cwl-scm): Add ScatterFeatureRequirement if workflow contains scattering steps. Serialize scattered-inputs and scatter-method. * doc/ccwl.skb (Cookbook)[Scatter/gather]: New section.
2023-11-14ccwl: Clarify that key->output is meant for workflow outputs only.Arun Isaac
* ccwl/ccwl.scm (key->output): Clarify in the docstring that this is meant for workflow outputs only.
2023-11-14ccwl: Convert stdout type outputs to File type in ccwl.scm.Arun Isaac
This conversation should be in ccwl.scm, not in cwl.scm. cwl.scm should only deal with CWL serialization concerns. * ccwl/cwl.scm (output->cwl-scm): Move conversion to ... * ccwl/ccwl.scm (key->output): ... here. (<output>)[type]: Add setter.
2023-11-14ccwl: Build output from key outside of syntax.Arun Isaac
It is easier (no need to juggle complex with-syntax constructs) to do more complicated operations outside of the scope of syntax. * ccwl/ccwl.scm (key->output): Build output from key outside of syntax.
2023-11-14ccwl: Introduce syntax for array types.Arun Isaac
* ccwl/ccwl.scm (<array-type>): New type. (memoize, make-array-type, construct-type-syntax): New functions. (input, output): Use construct-type-syntax. * ccwl/cwl.scm (output->cwl-scm, input->cwl-scm): Render array types.
2023-11-07cwl: Unify rendering of command and workflow outputs.Arun Isaac
* ccwl/cwl.scm (output->cwl-scm): Handle both command and workflow outputs. Accept #:workflow? argument to indicate workflow outputs. (workflow->cwl-scm): Use output->cwl-scm.
2023-11-07cwl: Deduplicate input to CWL rendering.Arun Isaac
* ccwl/cwl.scm (input->cwl-scm): New function. (workflow->cwl-scm, command->cwl-scm): Use input->cwl-scm.
2023-11-07ccwl: Delete unused input=? function.Arun Isaac
* ccwl/ccwl.scm (input=?): Delete function.
2023-11-06ui: Do not attempt to display the line preceding the 0th.Arun Isaac
* ccwl/ui.scm (source-in-context): Do not attempt to display the line preceding the 0th. * tests/ui.scm ("display source in context on 0th line"): New line.
2023-11-06tests: Add tests for ui.scm.Arun Isaac
* tests/ui.scm: New file.
2023-11-06Use (term ansi-color) for colors.Arun Isaac
* build-aux/test-driver.scm (color): Use colorize-string from (term ansi-color). * ccwl/ui.scm (report-formatted-message, source-in-context, report-ccwl-violation): Use colorize-string from (term ansi-color). (color, bold, red, magenta): Delete functions.
2023-11-06ui: Pass port, not filename, to source-in-context.Arun Isaac
* ccwl/ui.scm (source-in-context): Accept port, not filename. (report-ccwl-violation): Pass port, not filename.
2023-11-06ccwl: Error out if #:other parameters are not serializable to YAML.Arun Isaac
* ccwl/ccwl.scm (ensure-yaml-serializable): New function. (input, output, command): Use ensure-yaml-serializable. * tests/ccwl.scm ("inputs with #:other parameters that fail to evaluate must raise a &ccwl-violation condition", "outputs with #:other parameters that fail to evaluate must raise a &ccwl-violation condition", "commands with #:other parameters that fail to evaluate must raise a &ccwl-violation condition"): New tests.
2023-10-28ccwl: Restrict #:other parameter to YAML serializable trees.Arun Isaac
Perhaps it fits in better with the spirit of ccwl to not evaluate #:other parameters and instead allow only YAML serializable trees. This saves users from quoting #:other parameters, of course. But it also permits more thorough analysis and clearer error messages. Besides, there is not much reason to allow general expressions in #:other parameters. Such dynamism is not commonly required in ccwl or CWL. * ccwl/ccwl.scm (input, output, command): Restrict #:other parameter to YAML serializable trees that are not evaluated.
2023-10-17doc: Add staging-input-files.scm snippet source file.Arun Isaac
This file was required by the previous commit, but I forgot to commit it! * doc/staging-input-files.scm: New file.
2023-10-17ccwl: Add "Invalid parameter" to #:stderr and #:stdout error messages.Arun Isaac
* ccwl/ccwl.scm (command): Add "Invalid parameter" to #:stderr and #:stdout error messages. * tests/ccwl.scm ("commands with non-string #:stderr parameters must raise a &ccwl-violation condition", "commands with non-string #:stdout parameters must raise a &ccwl-violation condition"): Adjust tests.
2023-10-17ccwl: Support staging input files.Arun Isaac
* ccwl/ccwl.scm (<input>)[stage?]: New field. * ccwl/ccwl.scm (input): Add #:stage argument. * ccwl/cwl.scm (command->cwl-scm): Serialize requirements for staged inputs. * doc/ccwl.skb (Cookbook)[Stage input files]: New section. * tests/ccwl.scm ("inputs with an invalid #:stage? parameter must raise a &ccwl-violation condition"): New test.
2023-10-16ccwl: Add #:requirements argument to command.Arun Isaac
* ccwl/ccwl.scm (<command>)[requirements]: New field. (command): Add #:requirements argument. * ccwl/cwl.scm (command->cwl-scm): Serialize requirements.
2023-10-16ccwl: Support arbitrary expressions in #:other arguments.Arun Isaac
Support arbitrary expressions, that are actually evaluated, in #:other arguments. This means, among other things, that users will have to quote them correctly. While this complicates matters for novice users who may not be familiar with scheme, it is the most general and most flexible solution. In this particular case, we value flexibility over ease of use since #:other is an escape hatch and if users have to use it, ease of use has already been lost. * ccwl/ccwl.scm (input, output, command): Make #:other an actually evaluated unary argument.
2023-10-16ccwl: Error out on prefixes that are not strings.Arun Isaac
* ccwl/ccwl.scm (run-args): Error out on prefixes that are not strings. * tests/ccwl.scm ("command definitions with non-string prefixes in prefixed inputs must raise a &ccwl-violation condition"): New test.
2023-10-16ccwl: Flatten prefixed string arguments in command definitions.Arun Isaac
* ccwl/ccwl.scm (run-args): Flatten prefixed string arguments in command definitions. * tests/ccwl.scm ("tolerate prefixed string arguments in command definitions"): New test.
2023-10-16ccwl: Move run arguments construction to separate function.Arun Isaac
* ccwl/ccwl.scm (command): Move run arguments construction to ... (run-args): ... new function.
2023-10-16ccwl: Raise &ccwl-violation on invalid command #:run arguments.Arun Isaac
* ccwl/ccwl.scm (command): Raise &ccwl-violation on invalid command #:run arguments. * tests/ccwl.scm ("command definitions with invalid #:run arguments must raise a &ccwl-violation condition"): New test.
2023-10-16ccwl: Check if inputs in command definitions are defined.Arun Isaac
* ccwl/ccwl.scm (command): Check if inputs used in #:run arguments of command definitions are actually defined. * tests/ccwl.scm ("command definitions with undefined inputs in their #:run arguments must raise a &ccwl-violation condition", "command definitions with undefined prefix inputs in their #:run arguments must raise a &ccwl-violation condition"): New tests.
2023-10-16tests: Allow literal ints as arguments.Arun Isaac
* tests/ccwl.scm ("allow literal as arguments"): Rename to "allow literal strings as arguments". (print-int): New variable. ("allow literal ints as arguments"): New test.
2023-10-15ccwl: Make #:stderr and #:stdout first class parameters.Arun Isaac
#:stderr and #:stdout, especially #:stdout, are commonly required. They ought to be first class parameters and not tucked away into #:other. * ccwl/ccwl.scm (<command>)[stderr, stdout]: New fields. * ccwl/ccwl.scm (command): Accept #:stderr and #:stdout as first class parameters. * ccwl/cwl.scm (command->cwl-scm): Serialize stderr and stdout fields. * doc/capture-stdout.scm (print), doc/decompress-compile-run.scm (run), doc/checksum.scm (md5sum, sha1sum, sha256sum), doc/spell-check.scm (find-misspellings): Capture stdout in file. * doc/checksum.scm, doc/decompress-compile-run.scm: * doc/ccwl.skb (Tutorial)[Capturing the standard output stream of a command]: Document #:stdout first class parameter. * doc/ccwl.skb (Tutorial)[Workflow with multiple steps]: Capture stdout in explicitly named files. * tests/ccwl.scm ("commands with non-string #:stderr parameters must raise a &ccwl-violation condition", "commands with non-string #:stdout parameters must raise a &ccwl-violation condition"): New tests.
2023-10-14ccwl: Detect multiple expressions in workflow body.Arun Isaac
* ccwl/ccwl.scm (workflow): Detect multiple expressions in workflow body and raise an exception. * tests/ccwl.scm ("unrecognized workflow syntaxes must raise a &ccwl-violation condition"): Test for the formatted message too. ("multiple expressions in workflow body must raise a &ccwl-violation condition"): New test.
2023-10-14ccwl: Raise &ccwl-violation when workflow syntax is not recognized.Arun Isaac
* ccwl/ccwl.scm (workflow): Raise &ccwl-violation when workflow syntax is not recognized. * tests/ccwl.scm ("unrecognized workflow syntaxes must raise a &ccwl-violation condition"): New test.
2023-10-13tests: Raise a &ccwl-violation on unknown keys.Arun Isaac
* tests/ccwl.scm ("step supplied with an unknown key must raise a &ccwl-violation condition"): New test.
2023-10-13ccwl: Convert syntax to datum before checking if it is a symbol.Arun Isaac
* ccwl/ccwl.scm (collect-steps): Convert syntax to datum before checking if it is a symbol.
2023-10-09ccwl: Deprecate auto-connection of single input commands.Arun Isaac
This seems like functionality that will be hard to support and will almost never be used in a real-world workflow. This feature is not even advertised in the manual. It is best to silently deprecate this feature now. * ccwl/ccwl.scm (collect-steps): Deprecate auto-connection of single input commands.
2023-10-09ccwl: Allow literals as arguments.Arun Isaac
* ccwl/ccwl.scm (<input>)[set-input-default]: New setter. * ccwl/ccwl.scm (<command>)[set-command-inputs]: Add setter. * ccwl/ccwl.scm (apply-partially): New function. (collect-steps): Support literal strings as arguments. * tests/ccwl.scm ("allow literal strings as arguments"): New test. * .dir-locals.el (scheme-mode): Indent set-command-inputs and set-input-default.
2023-10-09tests: Catch expressions that don't raise exceptions.Arun Isaac
Many tests that use guard to catch and ensure that certain conditions are raised actually leak and let through cases where the expressions result in a truthy value. * tests/ccwl.scm ("input, when passed more than one positional argument, must raise a &ccwl-violation condition", "input, when passed an unrecognized keyword, must raise a &ccwl-violation condition", "input, when passed multiple arguments to a unary keyword, must raise a &ccwl-violation condition", "output, when passed more than one positional argument, must raise a &ccwl-violation condition", "output, when passed an unrecognized keyword, must raise a &ccwl-violation condition", "output, when passed multiple arguments to a unary keyword, must raise a &ccwl-violation condition", "command, when passed positional arguments, must raise a &ccwl-violation condition", "command, when passed an unrecognized keyword, must raise a &ccwl-violation condition", "command, when passed multiple arguments to a unary keyword, must raise a &ccwl-violation condition"): Catch expressions that don't raise exceptions.
2023-10-09ccwl: Error out if steps with expression commands have no identifier.Arun Isaac
* ccwl/ccwl.scm (collect-steps): Error out if steps with expressions that evaluate to commands have no identifier. * tests/ccwl.scm ("step with expression that evaluates to a command but without a step identifier must raise a &ccwl-violation condition"): New test.
generated by cgit v1.2.3 (git 2.45.2) at 2024-11-14 16:27:05 +0000