aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
3 daysworkflow: Build out propagator network eagerly.Arun Isaac
Build out propagator network eagerly descending into the lowest subworkflows. This will come in handy later on to build the G-exp scripts ahead of time before the workflow is run. * ravanan/workflow.scm (<scheduler-proc>)[cwl]: Rename to cwl-or-propnet. [formal-inputs, formal-outputs]: New fields. * ravanan/workflow.scm (workflow->scheduler-proc): New function. (workflow-class->propnet): Build out propagator network eagerly. (workflow-scheduler)[schedule]: Handle eagerly built-out propagator network <schedule-proc> objects. (run-workflow): Use workflow->scheduler-proc to convert workflow to <scheduler-proc> object.
3 daysworkflow: Eliminate command-line-tool->propagator.Arun Isaac
command-line-tool->propagator was packaging up both CommandLineTool and Workflow class workflows into <scheduler-proc> objects. It was named erroneously. We now merge it into step->propagator. * ravanan/workflow.scm (command-line-tool->propagator): Merge into ... (workflow-class->propnet)[step->propagator]: ... here.
3 dayscommand-line-tool: Build script without inputs.Arun Isaac
We build the script using only the CommandLineTool workflow itself, and not its inputs. We pass the inputs in through a WORKFLOW_INPUTS environment variable. To maintain bullet-proof caching, we construct ravanan store paths by hashing both the script and the inputs together. This change paves the way to pre-building all scripts ahead of time. * ravanan/command-line-tool.scm (<command-line-binding>, command-line-binding->args): Move to (ravanan work command-line-tool). (run-command-line-tool): Do not pass inputs to build-command-line-tool-script. Call step-store-files-directory instead of script->store-files-directory, step-store-data-file instead of script->store-data-file, step-store-stdout-file instead of script->store-stdout-file and step-store-stderr-file instead of script->store-stderr-file. Put inputs into job state objects. Pass inputs to the script through the WORKFLOW_INPUTS environment variable. (capture-command-line-tool-output): Accept inputs. Call step-store-data-file instead of script->store-data-file and step-store-files-directory instead of script->store-files-directory. (copy-input-files-gexp): Delete function. (build-command-line-tool-script): Do not accept inputs; get them from the WORKFLOW_INPUTS environment variable. [coerce-argument]: New function. [run-command-gexp]: Build command inside the G-expression. [gexp]{copy-input-files}: New function. * ravanan/job-state.scm (<single-machine-job-state>)[inputs]: New field. * ravanan/job-state.scm (<slurm-job-state>)[inputs]: New field. * ravanan/job-state.scm (job-state-inputs): New public function. * ravanan/store.scm: Import (srfi srfi-71), (gcrypt hash) and (guix build utils). (sha1-hash-sexp): New function. (script->store-files-directory, script->store-data-file, script->store-stdout-file, script->store-stderr-file): Delete functions. (step-store-files-directory, step-store-data-file, step-store-stdout-file, step-store-stderr-file): New public functions. * ravanan/work/command-line-tool.scm: Import (srfi srfi-9 gnu) and (ravanan work ui). * ravanan/workflow.scm (&job-failure)[inputs]: New field. * ravanan/workflow.scm (workflow-scheduler)[poll]: Put inputs into &job-failure condition. [capture-output]: Call step-store-stdout-file instead of script->store-stdout-file and step-store-stderr-file instead of script->store-stderr-file. Pass inputs to capture-command-line-tool-output. (run-workflow): Call step-store-stdout-file instead of script->store-stdout-file and step-store-stderr-file instead of script->store-stderr-file.
3 dayscommand-line-tool: Define environment only once.Arun Isaac
* ravanan/command-line-tool.scm (run-command-line-tool): Define environment only once.
3 daysstore: Encode store paths using base32 instead of base16.Arun Isaac
* ravanan/store.scm: Import (guix base16) and (guix base32). (intern-file): Encode store paths using base32 instead of base16.
3 dayswork/command-line-tool: Add sha1-hash-bytes.Arun Isaac
* ravanan/work/command-line-tool.scm (sha1-hash-bytes): New public function. (sha1-hash): Use sha1-hash-bytes.
3 daysworkflow: Bind class variable with reduced scope.Arun Isaac
* ravanan/workflow.scm (workflow-scheduler): Bind class variable only in the else arm of the if condition.
3 daysbin: Use user-error instead of errors to report user errors.Arun Isaac
User errors are caused by incorrect input from the user, rather than by actual bugs in the code. user-error is named after the similarly named elisp function. * bin/ravanan: Import (ravanan work ui). (%options, invalid-option, main): Use user-error instead of error.
3 dayspropnet: Add record printers.Arun Isaac
* ravanan/propnet.scm (<propagator>, <propnet-state>): Add record printers.
3 dayspropnet: Export predicates.Arun Isaac
* ravanan/propnet.scm (propnet?, propagator?, scheduler?, state+status?): Export.
13 daysbin: Print usage when arguments are not matched.Arun Isaac
* bin/ravanan (print-short-usage): New function. (main): Print usage when arguments are not matched.
2025-05-29command-line-tool: Re-indent.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script):[cores]: Re-indent.
2025-05-16command-line-tool: Make inputs subdirectory only if it does not exist.Arun Isaac
* ravanan/command-line-tool.scm (copy-input-files-gexp): Make inputs subdirectory only if it does not exist.
2025-05-16command-line-tool: Copy input files into their own subdirectories.Arun Isaac
* ravanan/command-line-tool.scm (copy-input-files-gexp): Copy input files into their own subdirectories.
2025-05-06bin: Import (ravanan work vectors) for vector-map.Arun Isaac
* bin/ravanan: Import (ravanan work vectors).
2025-04-23store: Intern files into a directory of their own.Arun Isaac
This way, user-supplied files and intermediate files generated by steps are stored the same way. Also, basename is preserved and there is no need to strip store hashes from filenames. * ravanan/command-line-tool.scm (copy-input-files-gexp): Use basename instead of store-item-name. * ravanan/store.scm (intern-file): Intern files into a directory of their own. (%store-hash-length, store-item-name): Delete variables. * ravanan/workflow.scm (resolve-inputs): Use basename instead of store-item-name.
2025-04-23command-line-tool: Strip store hash when copying out of the store.Arun Isaac
Without stripping the hashes, the primary and secondary files have different hash prefixes. This interferes with successful command execution. * ravanan/command-line-tool.scm (copy-input-files-gexp): Strip store hash when copying files out of the store into the temporary inputs directory. Update basename, nameroot and nameext accordingly.
2025-04-23store: Update basename, nameroot and nameext when interning files.Arun Isaac
* ravanan/store.scm (intern-file): Update basename, nameroot and nameext.
2025-04-23workflow: Use secondary-path.Arun Isaac
* ravanan/workflow.scm (resolve-inputs)[match-secondary-file-pattern]: Accept secondary-file. Call secondary-path. [check-secondary-files]: Pass secondary-file, instead of just the pattern, to match-secondary-file-pattern.
2025-04-23command-line-tool: Move secondary-path to work module.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script): Move to ... * ravanan/work/command-line-tool.scm (secondary-path): ... here.
2025-04-22command-line-tool: Pass secondary-file to secondary-path.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script): Pass secondary-file, not pattern, to secondary-path.
2025-04-22command-line-tool: Stage secondary files too.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script): Stage secondary files too.
2025-04-22reader: Use canonicalize-file-value.Arun Isaac
* ravanan/reader.scm (normalize-input): Use canonicalize-file-value.
2025-04-22work/command-line-tool: Comment on trusting the checksum.Arun Isaac
* ravanan/work/command-line-tool.scm (canonicalize-file-value): Comment on trusting the checksum.
2025-04-22work/command-line-tool: Canonicalize secondary files too.Arun Isaac
* ravanan/work/command-line-tool.scm (canonicalize-file-value): Canonicalize secondary files too.
2025-04-22work/command-line-tool: Handle absent path and location correctly.Arun Isaac
* ravanan/work/command-line-tool.scm (canonicalize-file-value): Handle absent path and location correctly.
2025-04-22reader: Move location->path to work module.Arun Isaac
* ravanan/reader.scm (location->path): Move to ... * ravanan/work/command-line-tool.scm (location->path): ... here.
2025-04-22command-line-tool: Move canonicalize-file-value to work module.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script): Move canonicalize-file-value to ... * ravanan/work/command-line-tool.scm (canonicalize-file-value): ... here.
2025-04-17command-line-tool: Abstract out handling of SecondaryFile pattern.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script)[secondary-path]: New function. [capture-secondary-file]: Use secondary-path.
2025-04-17command-line-tool: Abstract out file staging into its own function.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script)[stage-file]: New function. [stage-files]: Use stage-file.
2025-04-14store: Intern secondary files.Arun Isaac
* ravanan/store.scm: Import (srfi srfi-26) and (ravanan work vectors). (intern-file): Intern secondary files. (%store-hash-length): New variable. (store-item-name): New public function. * ravanan/workflow.scm (resolve-inputs)[match-secondary-file-pattern]: Strip store hashes before matching secondary file patterns.
2025-04-14dir-locals: Indent call-with-atomic-output-file correctly.Arun Isaac
* .dir-locals.el (scheme-mode): Indent call-with-atomic-output-file correctly.
2025-03-18tests: Fix typo in test.Arun Isaac
* tests/reader.scm ("Normalize array of File arrays type formal output"): Replace normalize-formal-input with normalize-formal-output.
2025-03-18tests: Tolerate only path or location in File type inputs.Arun Isaac
* tests/reader.scm: Import (ice-9 filesystem), (web uri), (ravanan work command-line-tool) and (ravanan work utils). (normalize-input): New variable. ("Normalize inputs with only location", "Normalize inputs with only path"): New tests.
2025-03-18reader: Canonicalize File inputs completely in reader.Arun Isaac
It is the reader's job to completely canonicalize File type inputs. This should not be done later when the workflow is executing. * ravanan/reader.scm: Import (web uri). (normalize-formal-input): Normalize default value. (location->path): New function. (normalize-input): Canonicalize File type input completely. * ravanan/workflow.scm (workflow-scheduler): Update comment about partially specified File objects. (location->path): Delete function. (resolve-inputs): Do not canonicalize File type inputs; they already are.
2025-03-18reader: Spin off input normalization into separate function.Arun Isaac
Separate functions are easier to test. * ravanan/reader.scm (read-inputs): Spin off input normalization into ... (normalize-input): ... new function.
2025-02-28README: Add Contributing section.Arun Isaac
* README.md (Contributing): New section.
2025-02-24command-line-tool: Pass package versions separately to inferior.Arun Isaac
* ravanan/command-line-tool.scm (software-packages->environment): Pass package versions as separate argument to lookup-inferior-packages.
2025-02-24command-line-tool: Handle SoftwarePackage in helper function.Arun Isaac
* ravanan/command-line-tool.scm (specifications->environment): Rename to software-packages->environment. (software-packages->environment): Accept vector of SoftwarePackage association lists. (build-command-line-tool-script): Move into software-packages->environment. Pass vector of SoftwarePackage association lists to software-packages->environment.
2025-02-24guix: Inherit ravanan package from Guix upstream.Arun Isaac
* .guix/ravanan-package.scm: Do no import (gnu packages package-management), (gnu packages gnupg), (gnu packages guile), (gnu packages guile-xyz), (gnu packages node), (guix build-system gnu), (guix licenses) and (guix profiles). Import ravanan from (gnu packages bioinformatics) with a guix: prefix. * .guix/ravanan-package.scm (ravanan): Inherit from guix:ravanan.
2025-02-01Revert "bin: Remove unused (ravanan command-line-tool) import."Arun Isaac
This reverts commit 17b88bd2ca1411fbf2799e0e194d7052c059a36b. (ravanan command-line-tool) is required for manifest-file-error?.
2025-01-28README: Mention use of caching for developing workflows.Arun Isaac
* README.md: Mention use of caching for developing workflows.
2025-01-28images: Crop image of Ravanan.Arun Isaac
We crop the image to remove Siva and Parvathi. * images/ravanan-king-of-lanka.jpg: Crop.
2025-01-28releases: Add v0.1.0 release tarball.Arun Isaac
* releases/ravanan-0.1.0.tar.lz, releases/ravanan-0.1.0.tar.lz.asc: New files.
2025-01-28bin: Add newline after usage message.v0.1.0Arun Isaac
* bin/ravanan (print-usage): Add newline at the end.
2025-01-28Makefile: Add release tarball generation rules.Arun Isaac
* Makefile (GIT, GPG, LZIP, config_file_template, distribute_files, dist_archive): New variables. (dist, distsign, $(dist_archive), %.asc): New rules.
2025-01-28Makefile: Treat config.scm distinctly from sources.Arun Isaac
config.scm is a generated source file. Treat it the same as a hand-written source file creates problems when generating release tarballs. * Makefile (sources): Remove $(config_file). (objects): Add $(config_file:.scm=.go). (install): Depend on $(config_file) and install it.
2025-01-27README: Add table of contents.Arun Isaac
* README.md: Add table of contents.
2025-01-27README: Add image of Ravanan.Arun Isaac
* README.md: Add image of Ravanan. (License): Mention license and attribution for image. * images/ravanan-king-of-lanka.jpg, images/LICENSE.md: New files.
2025-01-27README: Fix grammar.Arun Isaac
* README.md (Building from source): Change "do" to "do so".
generated by cgit v1.2.3 (git 2.49.0) at 2025-06-29 22:37:34 +0000