aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2024-10-01command-line-tool: Move scheduler to (ravanan workflow).Arun Isaac
* ravanan/command-line-tool.scm: Do not import (rnrs conditions), (rnrs exceptions) and (ravanan propnet). (command-line-tool-scheduler): Move to (ravanan workflow) as workflow-scheduler. (&job-failure, <scheduler-proc>): Move to (ravanan workflow). (script->store-stdout-file, script->store-stderr-file, capture-command-line-tool-output): Export functions. * ravanan/workflow.scm: Import define-condition-type from (rnrs conditions), (rnrs exceptions), (srfi srfi-9 gnu) and (ravanan job-state). (run-workflow): Call workflow-scheduler instead of command-line-tool-scheduler.
2024-10-01propnet: Allow propagators to update their state.Arun Isaac
This will come in handy later when we implement compound propagators (propagators that are themselves a propagator network). * ravanan/command-line-tool.scm (command-line-tool-scheduler)[poll]: Return two values---the status and the updated state. * ravanan/job-state.scm (job-state-status): Return two values---the status and the updated state. * ravanan/propnet.scm (partition-map): New function. (poll-propnet): Update state of a propagator in flight.
2024-10-01propnet: Present a polling interface to callers.Arun Isaac
(ravanan propnet) now presents a polling interface to the outside world. Instead of computing the entire propagator network in one function call, it needs to be polled repeatedly until it is done. Hence, the polling logic (such as the polling interval) needs to be outside (ravanan propnet). * ravanan/propnet.scm (<scheduler>)[poll-interval]: Delete field. (<propnet-state>): New record type. (schedule-propnet, capture-propnet-output): New public functions. (run-propnet): Rename to poll-propnet; return status and state values instead of fulling computing the propnet. * ravanan/command-line-tool.scm (%job-poll-interval): Move to (ravanan workflow). (command-line-tool-scheduler): Do not initialize poll-interval in scheduler. * ravanan/workflow.scm: Import (srfi srfi-71). (run-workflow): Use the new polling interface to the propnet.
2024-09-23command-line-tool: Update path to staged input files.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script)[stage-files, set-staged-path]: New functions. After staging, update path to staged input files.
2024-09-23command-line-tool: Defer access of input values to runtime.Arun Isaac
* ravanan/command-line-tool.scm (build-command): Pass id argument to collect-bindings. Defer access of input values to runtime. [collect-bindings]: Accept id argument. [input+type-tree+binding->command-line-binding]: Rename to id+input+type-tree+binding->command-line-binding. Accept id argument and return G-expressions for command-line-binding values. (command-line-binding->args): Defer access of input values to runtime.
2024-09-23command-line-tool: Separate file copying from canonization.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script)[canonicalize-file-value]: Separate out file copying into ... [copy-file-value]: ... new function. [capture-outputs-gexp, capture-secondary-file, path->value]: Call copy-file-value after canonicalize-file-value.
2024-09-23utils: Move json-ref to (ravanan work utils).Arun Isaac
* ravanan/utils.scm: Move to ... * ravanan/work/utils.scm (json-ref): ... here. * ravanan/slurm-api.scm: Import (ravanan work utils) instead of (ravanan utils).
2024-09-20command-line-tool: Add missing argument to canonicalize-file-value.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script): Add missing workflow-output-directory argument to canonicalize-file-value.
2024-09-19work/utils: Add alist=?.Arun Isaac
* ravanan/work/utils.scm (alist=?): New public function.
2024-09-19propnet: Describe state machine implementation.Arun Isaac
* ravanan/propnet.scm (run-propnet): Add comment describing state machine implementation.
2024-09-16bin: Explain why we support the SLURM_JWT=token format.Arun Isaac
* bin/ravanan (read-jwt): Explain why we support the SLURM_JWT=token format.
2024-09-13bin: Support SLURM_JWT=token format in JWT files.Arun Isaac
* bin/ravanan: Import (srfi srfi-26) and (ravanan utils). (read-jwt): Support SLURM_JWT=token format.
2024-09-13bin: Refactor JWT reading into separate function.Arun Isaac
* bin/ravanan (read-jwt): New function. (main): Call read-jwt.
2024-09-13utils: Add string-trim-prefix.Arun Isaac
* ravanan/utils.scm (string-trim-prefix): New public function.
2024-09-13command-line-tool: Add nameroot and nameext fields.Arun Isaac
* ravanan/command-line-tool.scm (resolve-inputs)[canonicalize-file-value], (build-command-line-tool-script)[canonicalize-file-value]: Add nameroot and nameext fields.
2024-09-13command-line-tool: Remove unused collect-input-files function.Arun Isaac
* ravanan/command-line-tool.scm (collect-input-files): Delete function.
2024-09-13command-line-tool: Do not embed stdout-filename outside main body.Arun Isaac
stdout-output->value cannot access the inputs variable, and thus will fail when stdout-filename is a parameter reference that needs to be evaluated. To avoid this, pass it in as an argument. * ravanan/command-line-tool.scm (build-command-line-tool-script)[stdout-output->value]: Accept stdout-filename as argument instead of G-expression embedding. Pass stdout-filename as argument to stdout-output->value.
2024-09-13command-line-tool: Create vectors of secondary files for recursion.Arun Isaac
* ravanan/command-line-tool.scm (resolve-inputs)[resolve]: When recursing, create a vector of secondary files.
2024-09-13job-state: Pass on slurm API endpoint and jwt recursively.Arun Isaac
* ravanan/job-state.scm (job-state-status): Pass on slurm API endpoint and jwt recursively.
2024-09-13config: Remove.Arun Isaac
* Makefile (config_file): Delete variable. (sources): Remove $(config_file). (all, %.go): Do not depend on $(config_file). (clean): Do not delete $(config_file). * ravanan/config.scm, ravanan/config.scm.in: Delete files. * ravanan/command-line-tool.scm: Do not import (ravanan config).
2024-09-13command-line-tool: Support secondary files.Arun Isaac
* ravanan/command-line-tool.scm (resolve-inputs)[canonicalize-file-input, match-secondary-file-pattern, check-secondary-files]: New functions. [resolve]: Accept maybe-secondary-files argument. Check that secondary files are provided with File type inputs. (copy-input-files-gexp)[copy-input-files]: New function. Copy secondary files to inputs directory. (build-command-line-tool-script)[capture-outputs-gexp]: Pass maybe-secondary-files argument to other-output->value. [capture-secondary-file]: New function. [path->value]: Call capture-secondary-file. Accept maybe-secondary-files argument. [stdout-output->value]: Pass maybe-secondary-files argument to path->value. [other-output->value]: Accept maybe-secondary-files argument and pass it on to path->value. Add TODO note about updating paths of staged files in inputs objects.
2024-09-13vectors: Add vector-filter-map.Arun Isaac
* ravanan/work/vectors.scm (vector-filter-map): New public function.
2024-09-13work/command-line-tool: Comment on match-type clauses.Arun Isaac
* ravanan/work/command-line-tool.scm (match-type): Comment on cond ladder clauses.
2024-09-13work/command-line-tool: Comment on formal-parameter-type clauses.Arun Isaac
* ravanan/work/command-line-tool.scm (formal-parameter-type): Comment on cond ladder clauses.
2024-09-13vectors: Rename module to (ravanan work vectors).Arun Isaac
* ravanan/vectors.scm: Move to ravanan/work/vectors.scm. * ravanan/command-line-tool.scm, ravanan/job-state.scm, ravanan/reader.scm, ravanan/workflow.scm: Import (ravanan work vectors) instead of (ravanan vectors).
2024-09-13monads: Rename module to (ravanan work monads).Arun Isaac
* ravanan/monads.scm: Move to ravanan/work/monads.scm. * ravanan/command-line-tool.scm, ravanan/propnet.scm, ravanan/reader.scm, ravanan/workflow.scm: Import (ravanan work vectors) instead of (ravanan vectors).
2024-09-13utils: Move assoc-ref* to (ravanan work utils).Arun Isaac
* ravanan/utils.scm: Do not import (rnrs base). Move assoc-ref* to ... * ravanan/work/utils.scm: ... here. Import assertion-violation from (rnrs base). * ravanan/command-line-tool.scm, ravanan/workflow.scm: Do not import (ravanan utils).
2024-09-13command-line-tool: Do not re-intern already interned files.Arun Isaac
* ravanan/command-line-tool.scm (intern-file): Intern file only if it is not already in the store.
2024-09-13command-line-tool: Improve grammar in docstring.Arun Isaac
* ravanan/command-line-tool.scm (copy-input-files-gexp): Make G-expression singular.
2024-09-13command-line-tool: Coerce expressions in glob patterns.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script): Coerce expressions in glob patterns.
2024-09-13command-line-tool: Import (ravanan work ui) into G-expression.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script): Import (ravanan work ui) into G-expression.
2024-09-13ui: Rename module to (ravanan work ui).Arun Isaac
* ravanan/ui.scm: Move to ravanan/work/ui.scm. * ravanan/command-line-tool.scm, ravanan/workflow.scm: Import (ravanan work ui) instead of (ravanan ui).
2024-09-13reader: Normalize secondaryFiles in formals.Arun Isaac
* ravanan/reader.scm (normalize-workflow)[normalize-secondary-files]: New function. [normalize-formal-input]: Use normalize-secondary-files. [normalize-formal-output]: New function. Call normalie-formal-output from main body.
2024-09-12vectors: Add vector-append-map.Arun Isaac
* ravanan/vectors.scm (vector-append-map): New public function.
2024-09-12reader: Refactor type normalization into separate function.Arun Isaac
* ravanan/reader.scm (normalize-formals): Refactor type normalization into ... [normalize-type]: ... new function.
2024-09-12command-line-tool: Import guile-filesystem as an extension.Arun Isaac
This resolves the warning about importing (ice-9 filesystem) from the host. * ravanan/command-line-tool.scm: Import guile-filesystem from (gnu packages guile-xyz). (build-command-line-tool-script): Use with-extensions, not with-imported-modules, to import guile-filesystem.
2024-09-12command-line-tool: Support File type staging entries.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script): Support File type staging entries.
2024-09-12command-line-tool: Support interpolation with parameter references.Arun Isaac
* ravanan/command-line-tool.scm (javascript-expression?): Return #t even for parameter references mixed with literals. (interpolate-parameter-references): New function. (strip-javascript-expression): Delete function. (coerce-expression): Use interpolate-parameter-references instead of strip-javascript-expression.
2024-09-11command-line-tool: Import (rnrs io ports) into G-exp for put-string.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script): Import (rnrs io ports).
2024-09-11command-line-tool: Write, not read from, literal staged file.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script): Replace call-with-output-file with call-with-input-file.
2024-09-11command-line-tool: Use coerce-expression with staging files.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script)[files-to-stage]: Use coerce-expression. Adapt calling expression.
2024-09-11command-line-tool: Return maybe monad from maybe-bind.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script): Wrap result of files-to-stage in a maybe monad.
2024-09-11command-line-tool: Use vector-map->list.Arun Isaac
* ravanan/command-line-tool.scm (build-command-line-tool-script)[files-to-stage]: Use vector-map->list.
2024-09-11ui: Add docstrings.Arun Isaac
* ravanan/ui.scm (warning, user-error): Add docstrings.
2024-09-11workflow: Ensure required inputs are specified.Arun Isaac
* ravanan/workflow.scm: Import (ravanan ui). (run-workflow): Error out when required inputs are not specified.
2024-09-11workflow: Move optional input predicate to separate function.Arun Isaac
* ravanan/workflow.scm (optional-input?): New function. (command-line-tool->propagator): Use optional-input?.
2024-09-11ui: Rename error to user-error.Arun Isaac
error collides with a core binding. * ravanan/ui.scm (error): Rename to user-error. * ravanan/command-line-tool.scm: Import error without renaming to raise-error. Replace calls to error with user-error, and calls to raise-error with error.
2024-09-11ui: Move warning and error reporting functions to separate module.Arun Isaac
* ravanan/command-line-tool.scm: Import (ravanan ui). (warning, error): Move to (ravanan ui). * ravanan/ui.scm: New file.
2024-09-10job-state: Move job-state records into separate module.Arun Isaac
* ravanan/job-state.scm: New file. * ravanan/command-line-tool.scm: Import (ravanan job-state). (<single-machine-job-state>, <slurm-job-state>): Move to (ravanan job-state). (command-line-tool-scheduler): Use job-state-status and job-state-script.
2024-09-10slurm-api: Set name of job.Arun Isaac
* ravanan/slurm-api.scm (submit-job): Accept name argument and set it as the job name. * ravanan/command-line-tool.scm (run-command-line-tool): Pass name to submit-job.
generated by cgit v1.2.3 (git 2.45.2) at 2024-10-16 20:52:24 +0000