| Age | Commit message (Expand) | Author |
|---|
| 2024-10-02 | utils: Remove unused (ice-9 match) import.•••* ravanan/utils.scm: Do not import (ice-9 match).
| Arun Isaac |
| 2024-10-02 | command-line-tool: Remove unused imports.•••* ravanan/command-line-tool.scm: Do not import (srfi srfi-171), (ice-9
popen), (gcrypt base16), (gcrypt hash), (guix
describe), (yaml), (ravanan glob) and (ravanan reader).
| Arun Isaac |
| 2024-10-02 | propnet: Remove unused (srfi srfi-26) import.•••* ravanan/propnet.scm: Do not import (srfi srfi-26).
| Arun Isaac |
| 2024-10-02 | workflow: Remove unused (ravanan reader) import.•••* ravanan/workflow.scm: Do not import (ravanan reader).
| Arun Isaac |
| 2024-10-02 | command-line-tool: Remove redundant newline after warning.•••* ravanan/command-line-tool.scm (check-requirements): Remove redundant
newline after warning.
| Arun Isaac |
| 2024-10-02 | command-line-tool: Canonicalize inputs before embedding in G-exp.•••* ravanan/command-line-tool.scm (canonicalize-json): New function.
(build-command-line-tool-script): Use canonicalize-json.
| Arun Isaac |
| 2024-10-02 | workflow: Support subworkflows recursively.•••Implementing subworkflows recursively as sub-propnets is the correct
and elegant ouroboros way. Earlier, we were flattening workflows into
their constituent CommandLineTools steps. Among many other issues,
this approach ran into problems when scattering over subworkflows.
* ravanan/workflow.scm: Import assertion-violation from (rnrs base).
(<command-line-tool-state>, <workflow-state>): New record types.
(workflow-class->propnet): New function.
(workflow-class->propagators, workflow->command-line-tool-steps):
Delete functions.
(workflow-scheduler): Represent state using <command-line-tool-state>
and <workflow-state>. Recursively evaluate compound propagators as
propnets.
(run-workflow): Use capture-output and poll from scheduler.
| Arun Isaac |
| 2024-10-01 | workflow: Pass a list, not two vectors, to fold.•••* ravanan/workflow.scm (inherit-requirements-and-hints): Pass a list,
not two vectors, to fold.
| Arun Isaac |
| 2024-10-01 | propnet: Pass on scheduler to schedule calls.•••This enables sub-propnets (or compound propagators) to share the same
scheduler.
* ravanan/propnet.scm (activate-propagator): Accept scheduler as
argument instead of the schedule function.
(poll-propnet): Pass scheduler as argument instead of its schedule
function.
| Arun Isaac |
| 2024-10-01 | command-line-tool: Move inputs resolution to (ravanan workflow).•••We resolve inputs in run-workflow before doing anything else. We thus
avoid bugs due to partially or insufficiently resolved inputs.
* ravanan/command-line-tool.scm: Do not import (web uri).
(%store-files-directory, %store-data-directory,
%store-logs-directory): Export.
(location->path, resolve-inputs, intern-file): Move to (ravanan
workflow).
(build-command-line-tool-script): Do not call resolve-inputs.
* ravanan/workflow.scm: Import (web uri) and (ravanan work types).
(run-workflow): Call resolve-inputs.
| Arun Isaac |
| 2024-10-01 | job-state: Add that the job-state records are for batch systems.•••* ravanan/job-state.scm: Add to the commentary that the job-state
records are per batch system.
| Arun Isaac |
| 2024-10-01 | vectors: Add vector-mapn.•••* ravanan/work/vectors.scm: Import (srfi srfi-26).
(vector-mapn): New public function.
| Arun Isaac |
| 2024-10-01 | vectors: Add unindexed variant of vector-fold.•••* ravanan/work/vectors.scm: Import vector-fold from (srfi srfi-43)
with prefix.
(vector-fold): New public function.
(vector-map->list, vector-append-map, vector-append-map->list): Use
vector-fold.
| Arun Isaac |
| 2024-10-01 | job-state: Re-indent.•••* ravanan/job-state.scm (job-state-script): Re-indent.
| Arun Isaac |
| 2024-10-01 | command-line-tool: Copy path, not location.•••location is an implementation-dependent URI. path is a real filesystem
path.
* ravanan/command-line-tool.scm (copy-input-files-gexp): Copy path,
not location.
| Arun Isaac |
| 2024-10-01 | vectors: Add vector-member.•••* ravanan/work/vectors.scm (vector-member): New public function.
| Arun Isaac |
| 2024-10-01 | workflow: Use bound variables instead of calling getters again.•••* ravanan/workflow.scm (workflow-scheduler)[schedule]: Use bound
variables name and cwl instead of calling getters scheduler-proc-name
and scheduler-proc-cwl again.
| Arun Isaac |
| 2024-10-01 | command-line-tool: Move scheduler to (ravanan workflow).•••* ravanan/command-line-tool.scm: Do not import (rnrs
conditions), (rnrs exceptions) and (ravanan propnet).
(command-line-tool-scheduler): Move to (ravanan workflow) as
workflow-scheduler.
(&job-failure, <scheduler-proc>): Move to (ravanan workflow).
(script->store-stdout-file, script->store-stderr-file,
capture-command-line-tool-output): Export functions.
* ravanan/workflow.scm: Import define-condition-type from (rnrs
conditions), (rnrs exceptions), (srfi srfi-9 gnu) and (ravanan
job-state).
(run-workflow): Call workflow-scheduler instead of
command-line-tool-scheduler.
| Arun Isaac |
| 2024-10-01 | propnet: Allow propagators to update their state.•••This will come in handy later when we implement compound
propagators (propagators that are themselves a propagator network).
* ravanan/command-line-tool.scm (command-line-tool-scheduler)[poll]:
Return two values---the status and the updated state.
* ravanan/job-state.scm (job-state-status): Return two values---the
status and the updated state.
* ravanan/propnet.scm (partition-map): New function.
(poll-propnet): Update state of a propagator in flight.
| Arun Isaac |
| 2024-10-01 | propnet: Present a polling interface to callers.•••(ravanan propnet) now presents a polling interface to the outside
world. Instead of computing the entire propagator network in one
function call, it needs to be polled repeatedly until it is done.
Hence, the polling logic (such as the polling interval) needs to be
outside (ravanan propnet).
* ravanan/propnet.scm (<scheduler>)[poll-interval]: Delete field.
(<propnet-state>): New record type.
(schedule-propnet, capture-propnet-output): New public functions.
(run-propnet): Rename to poll-propnet; return status and state values
instead of fulling computing the propnet.
* ravanan/command-line-tool.scm (%job-poll-interval): Move to (ravanan
workflow).
(command-line-tool-scheduler): Do not initialize poll-interval in
scheduler.
* ravanan/workflow.scm: Import (srfi srfi-71).
(run-workflow): Use the new polling interface to the propnet.
| Arun Isaac |
| 2024-09-23 | command-line-tool: Update path to staged input files.•••*
ravanan/command-line-tool.scm (build-command-line-tool-script)[stage-files,
set-staged-path]: New functions.
After staging, update path to staged input files.
| Arun Isaac |
| 2024-09-23 | command-line-tool: Defer access of input values to runtime.•••* ravanan/command-line-tool.scm (build-command): Pass id argument to
collect-bindings. Defer access of input values to runtime.
[collect-bindings]: Accept id argument.
[input+type-tree+binding->command-line-binding]: Rename to
id+input+type-tree+binding->command-line-binding. Accept id argument
and return G-expressions for command-line-binding values.
(command-line-binding->args): Defer access of input values to runtime.
| Arun Isaac |
| 2024-09-23 | command-line-tool: Separate file copying from canonization.•••*
ravanan/command-line-tool.scm (build-command-line-tool-script)[canonicalize-file-value]:
Separate out file copying into ...
[copy-file-value]: ... new function.
[capture-outputs-gexp, capture-secondary-file, path->value]: Call
copy-file-value after canonicalize-file-value.
| Arun Isaac |
| 2024-09-23 | utils: Move json-ref to (ravanan work utils).•••* ravanan/utils.scm: Move to ...
* ravanan/work/utils.scm (json-ref): ... here.
* ravanan/slurm-api.scm: Import (ravanan work utils) instead
of (ravanan utils).
| Arun Isaac |
| 2024-09-20 | command-line-tool: Add missing argument to canonicalize-file-value.•••* ravanan/command-line-tool.scm (build-command-line-tool-script): Add
missing workflow-output-directory argument to canonicalize-file-value.
| Arun Isaac |
| 2024-09-19 | work/utils: Add alist=?.•••* ravanan/work/utils.scm (alist=?): New public function.
| Arun Isaac |
| 2024-09-19 | propnet: Describe state machine implementation.•••* ravanan/propnet.scm (run-propnet): Add comment describing state
machine implementation.
| Arun Isaac |
| 2024-09-16 | bin: Explain why we support the SLURM_JWT=token format.•••* bin/ravanan (read-jwt): Explain why we support the SLURM_JWT=token
format.
| Arun Isaac |
| 2024-09-13 | bin: Support SLURM_JWT=token format in JWT files.•••* bin/ravanan: Import (srfi srfi-26) and (ravanan utils).
(read-jwt): Support SLURM_JWT=token format.
| Arun Isaac |
| 2024-09-13 | bin: Refactor JWT reading into separate function.•••* bin/ravanan (read-jwt): New function.
(main): Call read-jwt.
| Arun Isaac |
| 2024-09-13 | utils: Add string-trim-prefix.•••* ravanan/utils.scm (string-trim-prefix): New public function.
| Arun Isaac |
| 2024-09-13 | command-line-tool: Add nameroot and nameext fields.•••*
ravanan/command-line-tool.scm (resolve-inputs)[canonicalize-file-value],
(build-command-line-tool-script)[canonicalize-file-value]: Add
nameroot and nameext fields.
| Arun Isaac |
| 2024-09-13 | command-line-tool: Remove unused collect-input-files function.•••* ravanan/command-line-tool.scm (collect-input-files): Delete
function.
| Arun Isaac |
| 2024-09-13 | command-line-tool: Do not embed stdout-filename outside main body.•••stdout-output->value cannot access the inputs variable, and thus
will fail when stdout-filename is a parameter reference that needs to
be evaluated. To avoid this, pass it in as an argument.
*
ravanan/command-line-tool.scm (build-command-line-tool-script)[stdout-output->value]:
Accept stdout-filename as argument instead of G-expression embedding.
Pass stdout-filename as argument to stdout-output->value.
| Arun Isaac |
| 2024-09-13 | command-line-tool: Create vectors of secondary files for recursion.•••* ravanan/command-line-tool.scm (resolve-inputs)[resolve]: When
recursing, create a vector of secondary files.
| Arun Isaac |
| 2024-09-13 | job-state: Pass on slurm API endpoint and jwt recursively.•••* ravanan/job-state.scm (job-state-status): Pass on slurm API endpoint
and jwt recursively.
| Arun Isaac |
| 2024-09-13 | config: Remove.•••* Makefile (config_file): Delete variable.
(sources): Remove $(config_file).
(all, %.go): Do not depend on $(config_file).
(clean): Do not delete $(config_file).
* ravanan/config.scm, ravanan/config.scm.in: Delete files.
* ravanan/command-line-tool.scm: Do not import (ravanan config).
| Arun Isaac |
| 2024-09-13 | command-line-tool: Support secondary files.•••*
ravanan/command-line-tool.scm (resolve-inputs)[canonicalize-file-input,
match-secondary-file-pattern, check-secondary-files]: New functions.
[resolve]: Accept maybe-secondary-files argument.
Check that secondary files are provided with File type inputs.
(copy-input-files-gexp)[copy-input-files]: New function.
Copy secondary files to inputs directory.
(build-command-line-tool-script)[capture-outputs-gexp]: Pass
maybe-secondary-files argument to other-output->value.
[capture-secondary-file]: New function.
[path->value]: Call capture-secondary-file. Accept
maybe-secondary-files argument.
[stdout-output->value]: Pass maybe-secondary-files argument to
path->value.
[other-output->value]: Accept maybe-secondary-files argument and pass
it on to path->value.
Add TODO note about updating paths of staged files in inputs objects.
| Arun Isaac |
| 2024-09-13 | vectors: Add vector-filter-map.•••* ravanan/work/vectors.scm (vector-filter-map): New public function.
| Arun Isaac |
| 2024-09-13 | work/command-line-tool: Comment on match-type clauses.•••* ravanan/work/command-line-tool.scm (match-type): Comment on cond
ladder clauses.
| Arun Isaac |
| 2024-09-13 | work/command-line-tool: Comment on formal-parameter-type clauses.•••* ravanan/work/command-line-tool.scm (formal-parameter-type): Comment
on cond ladder clauses.
| Arun Isaac |
| 2024-09-13 | vectors: Rename module to (ravanan work vectors).•••* ravanan/vectors.scm: Move to ravanan/work/vectors.scm.
* ravanan/command-line-tool.scm, ravanan/job-state.scm,
ravanan/reader.scm, ravanan/workflow.scm: Import (ravanan work
vectors) instead of (ravanan vectors).
| Arun Isaac |
| 2024-09-13 | monads: Rename module to (ravanan work monads).•••* ravanan/monads.scm: Move to ravanan/work/monads.scm.
* ravanan/command-line-tool.scm, ravanan/propnet.scm,
ravanan/reader.scm, ravanan/workflow.scm: Import (ravanan work
vectors) instead of (ravanan vectors).
| Arun Isaac |
| 2024-09-13 | utils: Move assoc-ref* to (ravanan work utils).•••* ravanan/utils.scm: Do not import (rnrs base). Move assoc-ref* to ...
* ravanan/work/utils.scm: ... here. Import assertion-violation
from (rnrs base).
* ravanan/command-line-tool.scm, ravanan/workflow.scm: Do not
import (ravanan utils).
| Arun Isaac |
| 2024-09-13 | command-line-tool: Do not re-intern already interned files.•••* ravanan/command-line-tool.scm (intern-file): Intern file only if it
is not already in the store.
| Arun Isaac |
| 2024-09-13 | command-line-tool: Improve grammar in docstring.•••* ravanan/command-line-tool.scm (copy-input-files-gexp): Make
G-expression singular.
| Arun Isaac |
| 2024-09-13 | command-line-tool: Coerce expressions in glob patterns.•••* ravanan/command-line-tool.scm (build-command-line-tool-script):
Coerce expressions in glob patterns.
| Arun Isaac |
| 2024-09-13 | command-line-tool: Import (ravanan work ui) into G-expression.•••* ravanan/command-line-tool.scm (build-command-line-tool-script):
Import (ravanan work ui) into G-expression.
| Arun Isaac |
| 2024-09-13 | ui: Rename module to (ravanan work ui).•••* ravanan/ui.scm: Move to ravanan/work/ui.scm.
* ravanan/command-line-tool.scm, ravanan/workflow.scm: Import (ravanan
work ui) instead of (ravanan ui).
| Arun Isaac |
| 2024-09-13 | reader: Normalize secondaryFiles in formals.•••* ravanan/reader.scm (normalize-workflow)[normalize-secondary-files]:
New function.
[normalize-formal-input]: Use normalize-secondary-files.
[normalize-formal-output]: New function.
Call normalie-formal-output from main body.
| Arun Isaac |
