| Age | Commit message (Expand) | Author |
|---|
| 2024-10-08 | javascript: Reference inputs, self or runtime symbols; not strings.•••* ravanan/javascript.scm (evaluate-parameter-reference-1): Reference
inputs, self or runtime symbols; not strings.
| Arun Isaac |
| 2024-10-08 | tests: Add tests for javascript G-expressions.•••* tests/javascript.scm: Import (guix gexp) and (ice-9 match).
(gexp->sexp-rec): New function.
("evaluate parameter reference (without context)", "evaluate parameter
reference with string interpolation (without context)", "evaluate
parameter reference with string interpolation of JSON trees (without
context)", "evaluate parameter reference with node (without context)",
"evaluate parameter reference with string interpolation using
node (without context)", "evaluate parameter reference with string
interpolation of JSON trees using node (without context)"): New tests.
| Arun Isaac |
| 2024-10-07 | command-line-tool: Implement GuixManifestRequirement.•••*
ravanan/command-line-tool.scm (%command-line-tool-supported-requirements):
Add GuixManifestRequirement.
(build-command-line-tool-script): Implement GuixManifestRequirement.
| Arun Isaac |
| 2024-10-07 | bin: Pass manifest file paths, not manifests to other functions.•••* bin/ravanan: Move to (ravanan command-line-tool). Add comment about
declarative modules.
(main): Pass manifest file path to run-workflow.
* ravanan/command-line-tool.scm (run-command-line-tool,
build-command-line-tool-script): Accept manifest file path instead of
manifest.
* ravanan/workflow.scm (workflow-scheduler, run-workflow): Accept
manifest file path instead of manifest.
| Arun Isaac |
| 2024-10-07 | bin: Load manifests without their definitions leaking out.•••* bin/ravanan: Do not import (gnu packages) and (guix profiles).
(load-manifest): New function.
(main): Use load-manifest.
| Arun Isaac |
| 2024-10-04 | javascript: Unify simple parameter reference and node fallback.•••* ravanan/command-line-tool.scm: Do not import (ravanan config), and
node from (gnu packages node).
(%worker-node): Move to (ravanan javascript).
(interpolate-parameter-references): Delete function.
(coerce-expression): Always call evaluate-parameter-reference
from (ravanan javascript).
* ravanan/work/command-line-tool.scm (evaluate-parameter-reference):
Delete function.
* ravanan/javascript.scm: Mention fallback to node in module
commentary. Import (ravanan config), (ravanan work command-line-tool),
and node from (gnu packages node).
(evaluate-simple-parameter-reference): Rename to
evaluate-parameter-reference.
(evaluate-parameter-reference-1, evaluate-parameter-reference): Add
expression-lib argument. Return regular values, not maybe-monadic
values.
(evaluate-using-node): New function.
* tests/javascript.scm ("evaluate parameter reference", "evaluate
parameter reference with string interpolation", "evaluate parameter
reference with string interpolation of JSON trees"): Update tests.
("evaluate parameter reference with node", "evaluate parameter
reference with string interpolation using node", "evaluate parameter
reference with string interpolation of JSON trees using node"): New
tests.
| Arun Isaac |
| 2024-10-04 | javascript: Remove "alternating" from tokenizing docstring.•••tokenize-parameter-references need not return an "alternating"
sequence of tokens. There can be consecutive parameter reference
tokens or literal strings.
* ravanan/javascript.scm (tokenize-parameter-references): Remove
"alternating" from docstring.
| Arun Isaac |
| 2024-10-04 | monads: Use write, not display, to print Just values.•••* ravanan/work/monads.scm (<maybe>): Use write, not display, to print
Just values.
| Arun Isaac |
| 2024-10-04 | javascript: Implement parameter reference javascript subset language.•••* ravanan/javascript.scm, tests/javascript.scm: New files.
* ravanan/command-line-tool.scm: Import (ravanan javascript).
(coerce-expression): Use evaluate-simple-parameter-reference.
(interpolate-parameter-references)[tokenize]: Move to (ravanan
javascript) as tokenize-parameter-references.
| Arun Isaac |
| 2024-10-04 | command-line-tool: Move canonicalize-json to (ravanan work utils).•••* ravanan/command-line-tool.scm (canonicalize-json): Move to (ravanan
work utils). Move comment about G-expressions to
build-command-line-tool-script.
* ravanan/work/utils.scm (ravanan): Import (ravanan work vectors).
| Arun Isaac |
| 2024-10-04 | command-line-tool: Merge local and remote coerce-expression.•••coerce-expression and coerce-expression-local share most of their
code. So, it makes sense to merge them.
* ravanan/command-line-tool.scm (coerce-expression): Add optional
context argument.
(coerce-expression-local): Delete function.
(run-command-line-tool): Call coerce-expression instead of
coerce-expression-local.
| Arun Isaac |
| 2024-10-02 | command-line-tool: Embed int value into G-expression.•••* ravanan/command-line-tool.scm (command-line-binding->args): Embed
int value into G-expression instead of evaluating it immediately.
| Arun Isaac |
| 2024-10-02 | command-line-tool: Coerce coresMin expressions.•••* ravanan/command-line-tool.scm (run-command-line-tool): Coerce
coresMin expressions.
| Arun Isaac |
| 2024-10-02 | command-line-tool: Re-indent coerce-expression.•••* ravanan/command-line-tool.scm (coerce-expression): Re-indent.
| Arun Isaac |
| 2024-10-02 | command-line-tool: Add local version of coerce-expression.•••* ravanan/command-line-tool.scm (coerce-expression-local): New
function.
| Arun Isaac |
| 2024-10-02 | reader: Handle coercion of number to number.•••* ravanan/reader.scm (coerce-type): Handle coercion of number to
number.
* tests/reader.scm: New file.
| Arun Isaac |
| 2024-10-02 | config: Remove generated source file.•••config.scm is generated from config.scm.in. Therefore, config.scm must
not be committed.
* ravanan/config.scm: Delete file.
| Arun Isaac |
| 2024-10-02 | Revert "config: Remove."•••This reverts commit d611d990b297b3d573d7d5af83e9376744d51dd5. We need
to bring back a local node to evaluate javascript expressions on the
head node.
| Arun Isaac |
| 2024-10-02 | workflow: Add default inputs before scheduling.•••We add default input values and resolve them before scheduling. Now
that we do that, we no longer have to resolve inputs or check for
required inputs in run-workflow.
* ravanan/workflow.scm (add-defaults): New function.
(workflow-scheduler)[schedule]: Add default inputs and resolve them
before scheduling.
(run-workflow): Do not resolve inputs or check for required inputs.
| Arun Isaac |
| 2024-10-02 | command-line-tool: Make runtime.cores a number.•••*
ravanan/command-line-tool.scm (build-command-line-tool-script)[cores]:
Return a number on the slurm-api batch system.
| Arun Isaac |
| 2024-10-02 | command-line-tool: Support coresMin ResourceRequirement.•••* ravanan/command-line-tool.scm: Import (ravanan reader).
(%command-line-tool-supported-requirements):
Add ResouceRequirement.
(command-line-tool-supported-requirements): Remove ResourceRequirement
for the single-machine batch system.
* ravanan/command-line-tool.scm (run-command-line-tool): Request the
number of CPUs specified in coresMin of the ResourceRequirement.
| Arun Isaac |
| 2024-10-02 | reader: Support coercion to number.•••* ravanan/reader.scm (coerce-type): Support coercion to number.
| Arun Isaac |
| 2024-10-02 | workflow: Move coerce-type to (ravanan reader).•••* ravanan/workflow.scm (coerce-type): Move to ...
* ravanan/reader.scm (coerce-type): ... here.
| Arun Isaac |
| 2024-10-02 | README: Mention "work reuse".•••"Work reuse" is the term used by the CWL specification for caching.
Use it to keep things familiar.
* README.md: Mention "work reuse" in introduction.
| Arun Isaac |
| 2024-10-02 | command-line-tool: Promote find-requirement to top level.•••*
ravanan/command-line-tool.scm (build-command-line-tool-script)[find-requirement]:
Promote to top level.
| Arun Isaac |
| 2024-10-02 | slurm-api: Allow requesting a certain number of CPUs.•••* ravanan/slurm-api.scm (submit-job): Accept cpus argument.
* ravanan/command-line-tool.scm (run-command-line-tool): Pass cpus
argument.
(build-command-line-tool-script)[cores]: New function.
Set runtime.cores based on CPU allocation.
| Arun Isaac |
| 2024-10-02 | propnet: Explain in detail why we never cancel running propagators.•••* ravanan/propnet.scm (poll-propnet): Add detailed comment elaborating
on why we never cancel or forget about running propagators.
| Arun Isaac |
| 2024-10-02 | command-line-tool: Allow per-batch system requirements support.•••*
ravanan/command-line-tool.scm (command-line-tool-supported-requirements):
New public function.
(check-requirements): Accept batch-system and
supported-requirements-for-batch-system arguments.
* ravanan/command-line-tool.scm (run-command-line-tool): Pass
batch-system to build-command-line-tool-script.
(build-command-line-tool-script): Accept batch-system argument. Update
calls to check-requirements.
* ravanan/workflow.scm (%workflow-only-requirements): New variable.
(%supported-requirements): Use %workflow-only-requirements.
(supported-requirements): New function.
(workflow-class->propnet): Accept batch-system argument. Update calls
to check-requirements.
(workflow-scheduler)[schedule]: Pass batch-system to
workflow-class->propnet.
| Arun Isaac |
| 2024-10-02 | utils: Remove unused (ice-9 match) import.•••* ravanan/utils.scm: Do not import (ice-9 match).
| Arun Isaac |
| 2024-10-02 | command-line-tool: Remove unused imports.•••* ravanan/command-line-tool.scm: Do not import (srfi srfi-171), (ice-9
popen), (gcrypt base16), (gcrypt hash), (guix
describe), (yaml), (ravanan glob) and (ravanan reader).
| Arun Isaac |
| 2024-10-02 | propnet: Remove unused (srfi srfi-26) import.•••* ravanan/propnet.scm: Do not import (srfi srfi-26).
| Arun Isaac |
| 2024-10-02 | workflow: Remove unused (ravanan reader) import.•••* ravanan/workflow.scm: Do not import (ravanan reader).
| Arun Isaac |
| 2024-10-02 | command-line-tool: Remove redundant newline after warning.•••* ravanan/command-line-tool.scm (check-requirements): Remove redundant
newline after warning.
| Arun Isaac |
| 2024-10-02 | command-line-tool: Canonicalize inputs before embedding in G-exp.•••* ravanan/command-line-tool.scm (canonicalize-json): New function.
(build-command-line-tool-script): Use canonicalize-json.
| Arun Isaac |
| 2024-10-02 | workflow: Support subworkflows recursively.•••Implementing subworkflows recursively as sub-propnets is the correct
and elegant ouroboros way. Earlier, we were flattening workflows into
their constituent CommandLineTools steps. Among many other issues,
this approach ran into problems when scattering over subworkflows.
* ravanan/workflow.scm: Import assertion-violation from (rnrs base).
(<command-line-tool-state>, <workflow-state>): New record types.
(workflow-class->propnet): New function.
(workflow-class->propagators, workflow->command-line-tool-steps):
Delete functions.
(workflow-scheduler): Represent state using <command-line-tool-state>
and <workflow-state>. Recursively evaluate compound propagators as
propnets.
(run-workflow): Use capture-output and poll from scheduler.
| Arun Isaac |
| 2024-10-01 | workflow: Pass a list, not two vectors, to fold.•••* ravanan/workflow.scm (inherit-requirements-and-hints): Pass a list,
not two vectors, to fold.
| Arun Isaac |
| 2024-10-01 | propnet: Pass on scheduler to schedule calls.•••This enables sub-propnets (or compound propagators) to share the same
scheduler.
* ravanan/propnet.scm (activate-propagator): Accept scheduler as
argument instead of the schedule function.
(poll-propnet): Pass scheduler as argument instead of its schedule
function.
| Arun Isaac |
| 2024-10-01 | command-line-tool: Move inputs resolution to (ravanan workflow).•••We resolve inputs in run-workflow before doing anything else. We thus
avoid bugs due to partially or insufficiently resolved inputs.
* ravanan/command-line-tool.scm: Do not import (web uri).
(%store-files-directory, %store-data-directory,
%store-logs-directory): Export.
(location->path, resolve-inputs, intern-file): Move to (ravanan
workflow).
(build-command-line-tool-script): Do not call resolve-inputs.
* ravanan/workflow.scm: Import (web uri) and (ravanan work types).
(run-workflow): Call resolve-inputs.
| Arun Isaac |
| 2024-10-01 | job-state: Add that the job-state records are for batch systems.•••* ravanan/job-state.scm: Add to the commentary that the job-state
records are per batch system.
| Arun Isaac |
| 2024-10-01 | vectors: Add vector-mapn.•••* ravanan/work/vectors.scm: Import (srfi srfi-26).
(vector-mapn): New public function.
| Arun Isaac |
| 2024-10-01 | vectors: Add unindexed variant of vector-fold.•••* ravanan/work/vectors.scm: Import vector-fold from (srfi srfi-43)
with prefix.
(vector-fold): New public function.
(vector-map->list, vector-append-map, vector-append-map->list): Use
vector-fold.
| Arun Isaac |
| 2024-10-01 | job-state: Re-indent.•••* ravanan/job-state.scm (job-state-script): Re-indent.
| Arun Isaac |
| 2024-10-01 | command-line-tool: Copy path, not location.•••location is an implementation-dependent URI. path is a real filesystem
path.
* ravanan/command-line-tool.scm (copy-input-files-gexp): Copy path,
not location.
| Arun Isaac |
| 2024-10-01 | vectors: Add vector-member.•••* ravanan/work/vectors.scm (vector-member): New public function.
| Arun Isaac |
| 2024-10-01 | workflow: Use bound variables instead of calling getters again.•••* ravanan/workflow.scm (workflow-scheduler)[schedule]: Use bound
variables name and cwl instead of calling getters scheduler-proc-name
and scheduler-proc-cwl again.
| Arun Isaac |
| 2024-10-01 | command-line-tool: Move scheduler to (ravanan workflow).•••* ravanan/command-line-tool.scm: Do not import (rnrs
conditions), (rnrs exceptions) and (ravanan propnet).
(command-line-tool-scheduler): Move to (ravanan workflow) as
workflow-scheduler.
(&job-failure, <scheduler-proc>): Move to (ravanan workflow).
(script->store-stdout-file, script->store-stderr-file,
capture-command-line-tool-output): Export functions.
* ravanan/workflow.scm: Import define-condition-type from (rnrs
conditions), (rnrs exceptions), (srfi srfi-9 gnu) and (ravanan
job-state).
(run-workflow): Call workflow-scheduler instead of
command-line-tool-scheduler.
| Arun Isaac |
| 2024-10-01 | propnet: Allow propagators to update their state.•••This will come in handy later when we implement compound
propagators (propagators that are themselves a propagator network).
* ravanan/command-line-tool.scm (command-line-tool-scheduler)[poll]:
Return two values---the status and the updated state.
* ravanan/job-state.scm (job-state-status): Return two values---the
status and the updated state.
* ravanan/propnet.scm (partition-map): New function.
(poll-propnet): Update state of a propagator in flight.
| Arun Isaac |
| 2024-10-01 | propnet: Present a polling interface to callers.•••(ravanan propnet) now presents a polling interface to the outside
world. Instead of computing the entire propagator network in one
function call, it needs to be polled repeatedly until it is done.
Hence, the polling logic (such as the polling interval) needs to be
outside (ravanan propnet).
* ravanan/propnet.scm (<scheduler>)[poll-interval]: Delete field.
(<propnet-state>): New record type.
(schedule-propnet, capture-propnet-output): New public functions.
(run-propnet): Rename to poll-propnet; return status and state values
instead of fulling computing the propnet.
* ravanan/command-line-tool.scm (%job-poll-interval): Move to (ravanan
workflow).
(command-line-tool-scheduler): Do not initialize poll-interval in
scheduler.
* ravanan/workflow.scm: Import (srfi srfi-71).
(run-workflow): Use the new polling interface to the propnet.
| Arun Isaac |
| 2024-09-23 | command-line-tool: Update path to staged input files.•••*
ravanan/command-line-tool.scm (build-command-line-tool-script)[stage-files,
set-staged-path]: New functions.
After staging, update path to staged input files.
| Arun Isaac |
| 2024-09-23 | command-line-tool: Defer access of input values to runtime.•••* ravanan/command-line-tool.scm (build-command): Pass id argument to
collect-bindings. Defer access of input values to runtime.
[collect-bindings]: Accept id argument.
[input+type-tree+binding->command-line-binding]: Rename to
id+input+type-tree+binding->command-line-binding. Accept id argument
and return G-expressions for command-line-binding values.
(command-line-binding->args): Defer access of input values to runtime.
| Arun Isaac |
