| Age | Commit message (Expand) | Author |
|---|
| 12 hours | command-line-tool: Handle G-exp args when joining with item separator.•••* ravanan/command-line-tool.scm (command-line-binding->args): Handle
G-exp args when joining with item separator.
HEADmain | Arun Isaac |
| 12 hours | workflow: Handle exceptions in run-workflow.•••Since our scheduler is now state-monadic, the exception handler must
guard the context in which the monad is actually run.
* ravanan/workflow.scm (workflow-scheduler): Move exception handler to
...
(run-workflow): ... here.
| Arun Isaac |
| 13 hours | slurm-api: Bind return value of slurm-http-get in the state monad.•••* ravanan/slurm-api.scm (job-state): Bind return value of
slurm-http-get in the state monad.
| Arun Isaac |
| 13 hours | reader: Normalize File array type formals too.•••* ravanan/reader.scm: Import (ravanan work types).
(some-file-type?): New function.
(normalize-formal-input, normalize-formal-output): Normalize File
array type formals too.
* tests/reader.scm: Import (srfi srfi-1) and (ice-9 match).
(normalize-formal-input, normalize-formal-output): New variables.
(json=?): New function.
("Normalize File type formal input", "Normalize File array type formal
input", "Normalize array of File arrays type formal input", "Normalize
File type formal output", "Normalize File array type formal output",
"Normalize array of File arrays type formal output"): New tests.
| Arun Isaac |
| 14 hours | reader: Promote normalize functions to top level.•••It is easier to test top-level functions.
* ravanan/reader.scm (normalize-workflow)[normalize-secondary-files,
normalize-formal-input, normalize-formal-output,
normalize-base-command, normalize-arguments, normalize-steps]: Promote
to top level.
| Arun Isaac |
| 36 hours | workflow: Return correct updated state when polling list states.•••We were returning the original input state. That was obviously a
catastrophe! I have spent the better part of the day debugging this. I
am tired.
* ravanan/workflow.scm (workflow-scheduler)[poll]: Return correct
updated state when polling list states.
| Arun Isaac |
| 47 hours | command-line-tool: Write output JSON atomically to the store.•••* ravanan/command-line-tool.scm (run-command-line-tool,
build-command-line-tool-script): Write output JSON atomically to the
store.
| Arun Isaac |
| 47 hours | work/utils: Add call-with-atomic-output-file.•••* ravanan/work/utils.scm: Import (srfi srfi-71), (system foreign)
and (system foreign-library).
(fsync): New function.
(call-with-atomic-output-file): New public function.
| Arun Isaac |
| 47 hours | utils: Delete unused call-with-temporary-file.•••* ravanan/utils.scm (call-with-temporary-file): Delete function.
| Arun Isaac |
| 3 days | propnet: Make interface state-monadic.•••* ravanan/job-state.scm (job-state-status): Return a state-monadic
value.
* ravanan/propnet.scm: Import (srfi srfi-26).
(schedule-propnet, poll-propnet, capture-propnet-output): Return a
state-monadic value.
* ravanan/workflow.scm (workflow-scheduler)[schedule, poll,
capture-output]: Return a state-monadic value.
(run-workflow): Accept state-monadic values from schedule, poll and
capture-output.
| Arun Isaac |
| 3 days | workflow: Use lists to represent state of scatter steps.•••Lists are more convenient than vectors, and all the more so when
dealing with the state monad.
* ravanan/job-state.scm: Import (srfi srfi-1). Do not import (ravanan
work vectors).
(job-state-status): Use lists instead of vectors to represent state of
scatter steps.
* ravanan/workflow.scm (workflow-scheduler)[schedule, poll,
capture-output]: Use lists instead of vectors to represent state of
scatter steps.
(maybe-vector?): Delete function.
(maybe-list?): New function.
(merge-values): Support merging lists instead of vectors.
| Arun Isaac |
| 3 days | propnet: Introduce <state+status> objects for polling.•••Let pollers return <state+status> objects instead of two separate
values. <state+status> objects are essentially named 2-tuples. This
will be more convenient than multiple values when dealing with the
state monad.
* ravanan/propnet.scm (<state+status>): New record type.
(poll-propnet): Accept and return <state+status> objects.
* ravanan/workflow.scm (workflow-scheduler)[poll]: Accept and return
<state+status> objects.
(run-workflow): Accept <state+status> objects from poll.
| Arun Isaac |
| 3 days | job-state: Do not allow updating of low-level job-state objects.•••Low-level job-state objects tied to the batch system (such as those
that represent single-machine and slurm-api jobs) never get updated.
So, there is no point permitting it.
* ravanan/job-state.scm (job-state-status): Return only the job
status, not the updated job state.
* ravanan/workflow.scm (workflow-scheduler)[poll]: Accept only the job
status, not the updated job state, from job-state-status.
* ravanan/workflow.scm (<command-line-tool-state>)[job-state]: Remove
setter.
| Arun Isaac |
| 3 days | workflow: Return state of scheduled jobs as a state-monadic value.•••* ravanan/command-line-tool.scm (run-command-line-tool): Return state
of scheduled jobs as a state-monadic value.
* ravanan/workflow.scm (workflow-scheduler)[schedule]: Return state of
scheduled jobs as a state-monadic value.
(run-workflow): Run state of scheduled workflow in the state monad.
* ravanan/propnet.scm (poll-propnet): Run state of scheduled jobs in
the state monad.
[schedule-propagators]: New function.
| Arun Isaac |
| 3 days | single-machine: Make interface state-monadic.•••* ravanan/single-machine.scm: Import (ravanan work monads).
(submit-job): Convert to a state-monadic function.
* ravanan/command-line-tool.scm (run-command-line-tool): Run
state-monadic return value from single-machine:submit-job.
| Arun Isaac |
| 3 days | slurm: Make interface state-monadic.•••* ravanan/slurm-api.scm: Import (ravanan work monads).
(slurm-http-request, slurm-http-get, slurm-http-post, submit-job,
job-state): Convert to state-monadic functions.
* ravanan/command-line-tool.scm (run-command-line-tool): Update use of
submit-job.
* ravanan/job-state.scm: Import (ravanan work monads).
(job-state-status): Update use of job-state.
| Arun Isaac |
| 3 days | propnet: Schedule all propagators from inbox in a single step.•••There is no value in making the loop recursion explicit. A single
append-map call is clearer.
* ravanan/propnet.scm (poll-propnet): Schedule all propagators from
inbox in a single step.
| Arun Isaac |
| 3 days | single-machine: Move single machine batch system to separate module.•••* ravanan/command-line-tool.scm: Import (ravanan single-machine).
Import (ravanan slurm-api) with slurm: prefix.
(run-command-line-tool): Call single-machine:submit-job. Prefix
submit-job with slurm:.
* ravanan/single-machine.scm: New file.
| Arun Isaac |
| 3 days | monads: Rename bindings to other-bindings in mlet*.•••other-bindings is a more descriptive name.
* ravanan/work/monads.scm (mlet*): Rename bindings to other-bindings.
| Arun Isaac |
| 3 days | monads: Re-indent mlet*.•••* ravanan/work/monads.scm (mlet*): Re-indent.
| Arun Isaac |
| 3 days | monads: Add mappend-map.•••* ravanan/work/monads.scm (mappend-map): New function.
(state-append-map): New public function.
| Arun Isaac |
| 3 days | monads: Add mmap.•••* ravanan/work/monads.scm (mmap): New function.
(state-map): New public function.
| Arun Isaac |
| 4 days | monads: Add sequence.•••* ravanan/work/monads.scm (sequence): New function.
(state-sequence): New public function.
| Arun Isaac |
| 4 days | monads: Implement the state monad.•••* ravanan/work/monads.scm (<mstate>): New record type.
(state-bind, current-state, set-current-state, run-with-state): New
public functions.
(state-return, state-let*, state-begin): New public macros.
* ravanan/command-line-tool.scm: Be selective to avoid importing
run-with-state from (guix monads).
* .dir-locals.el (scheme-mode): Indent state-let* correctly.
| Arun Isaac |
| 4 days | monads: Add mbegin.•••* ravanan/work/monads.scm (mbegin): New macro.
| Arun Isaac |
| 4 days | monads: Do not export maybe-monad.•••* ravanan/work/monads.scm: Do not export maybe-monad.
| Arun Isaac |
| 4 days | command-line-tool: Re-indent run-command-line-tool.•••* ravanan/command-line-tool.scm (run-command-line-tool): Re-indent.
| Arun Isaac |
| 2025-01-07 | guix: Use %default-gnu-imported-modules.•••%gnu-build-system-modules is deprecated.
* .guix/ravanan-package.scm (ravanan)[arguments]: Replace
%gnu-build-system-modules with %default-gnu-imported-modules.
| Arun Isaac |
| 2025-01-07 | guix: Add NODE makeflag.•••* .guix/ravanan-package.scm (ravanan)[arguments]: Add NODE makeflag.
| Arun Isaac |
| 2025-01-07 | guix: Depend on guile-json-4.•••* .guix/ravanan-package.scm: Import guile-json-4 from (gnu packages
guile).
(ravanan)[inputs]: Add guile-json-4.
| Arun Isaac |
| 2024-12-06 | dir-locals: Indent call-with-inferior correctly.•••* .dir-locals.el (scheme-mode): Indent call-with-inferior correctly.
| Arun Isaac |
| 2024-12-05 | bin: Do not specify the default output port unnecessarily.•••* bin/ravanan (main): Do not specify the
default (current-output-port).
| Arun Isaac |
| 2024-12-05 | bin: Add --output-directory argument.•••* bin/ravanan: Import (ravanan work utils).
(%options): Add --output-directory.
(print-usage): Document it.
(symlink-to-output-directory): New function.
(main): Symlink to output directory if it is specified.
| Arun Isaac |
| 2024-12-04 | bin: Indicate that the --store argument was not specified.•••* bin/ravanan (main): Indicate specifically that the --store argument
was not specified.
| Arun Isaac |
| 2024-12-04 | bin: Indicate that the --scratch argument was not specified.•••* bin/ravanan (main): Indicate specifically that the --scratch
argument was not specified.
| Arun Isaac |
| 2024-12-04 | bin: Indicate that the --slurm-jwt argument was not specified.•••* bin/ravanan (main): Indicate specifically that the --slurm-jwt
argument was not specified.
| Arun Isaac |
| 2024-12-04 | bin: Add --version flag.•••* ravanan/config.scm.in (%project, %version): New variables.
* Makefile (version): New variable.
(%.scm): Substitute %project and %version too.
* bin/ravanan: Import (ravanan config).
(%options): Add --version.
(print-usage): Document it.
(main): Implement it.
| Arun Isaac |
| 2024-11-06 | command-line-tool: Bubble up manifest file errors to the top level.•••* ravanan/command-line-tool.scm: Import (rnrs conditions) and (rnrs
exceptions).
(&manifest-file-error): New condition type.
(load-manifest): Raise &manifest-file-error when loading the manifest
file fails.
* bin/ravanan: Import (rnrs exceptions) and (ravanan
command-line-tool).
(main): Handle manifest file errors bubbled up from lower down the
stack.
| Arun Isaac |
| 2024-11-06 | bin: Remove default Guix manifest.•••* bin/ravanan (main): Remove default Guix manifest.
| Arun Isaac |
| 2024-11-06 | command-line-tool: Support packages in SoftwareRequirement.•••* ravanan/command-line-tool.scm: Import (gnu packages).
(specifications->environment): New function.
(build-command-line-tool-script)[software-package->package-specification]:
New function.
Build environment from packages if specified.
| Arun Isaac |
| 2024-11-06 | command-line-tool: Allow processing manifest objects without a file.•••*
ravanan/command-line-tool.scm (manifest->environment): New function.
(manifest-file->environment)[build-derivation]: Delete function.
Call manifest->environment.
| Arun Isaac |
| 2024-11-06 | bin: Add --guix-channels command-line argument.•••* bin/ravanan: Import (guix channels).
(%options): Add guix-channels.
(print-usage): Document it.
(main): Load channels from channels file if specified.
| Arun Isaac |
| 2024-11-06 | command-line-tool: Build manifest in Guix inferior with channels.•••* ravanan/command-line-tool.scm: Import (guix inferior).
(call-with-inferior): New function.
(run-command-line-tool): Accept channels argument and pass it on to
build-command-line-tool-script.
(build-command-line-tool-script): Accept channels argument and pass it
on to manifest-file->environment.
(manifest-file->environment): Accept channels argument. Build manifest
in Guix inferior when channels is provided.
* ravanan/workflow.scm (workflow-scheduler): Accept channels argument
and pass it on to run-command-line-tool.
(run-workflow): Accept channels argument and pass it on to
workflow-scheduler.
* bin/ravanan (main): Pass #f as channels to run-workflow.
| Arun Isaac |
| 2024-11-06 | command-line-tool: Convert manifest to environment in new function.•••* ravanan/command-line-tool.scm (manifest-file->environment): New
function.
(build-command-line-tool-script): Use manifest-file->environment.
| Arun Isaac |
| 2024-11-06 | utils: Generalize load-manifest and move to (ravanan utils).•••* ravanan/utils.scm: Import (ice-9 filesystem) and (ice-9 match).
(load-script): New function.
* ravanan/command-line-tool.scm: Import (ravanan utils).
(load-manifest): Use load-script.
| Arun Isaac |
| 2024-11-06 | command-line-tool: Implement WorkReuse.•••*
ravanan/command-line-tool.scm (%command-line-tool-supported-requirements):
Add WorkReuse.
* ravanan/command-line-tool.scm (build-command-line-tool-script):
Implement WorkReuse.
| Arun Isaac |
| 2024-11-06 | command-line-tool: Fit Guix manifest into SoftwareRequirement.•••*
ravanan/command-line-tool.scm (%command-line-tool-supported-requirements):
Replace GuixManifestRequirement with SoftwareRequirement.
(build-command-line-tool-script): Look for SoftwareRequirement instead
of GuixManifestRequirement.
| Arun Isaac |
| 2024-11-06 | slurm-api: Find state of purged jobs.•••* ravanan/slurm-api.scm (job-state): Query slurmdb for jobs that have
been purged from slurmctld's active memory.
| Arun Isaac |
| 2024-11-06 | slurm-api: Deduplicate API error checks.•••* ravanan/slurm-api.scm (check-api-error): New function.
(submit-job, job-state): Use check-api-error.
| Arun Isaac |
| 2024-11-06 | slurm-api: Upgrade to API version 0.0.41.•••* ravanan/slurm-api.scm (submit-job, job-state): Upgrade to API
version 0.0.41.
(job-state): Handle job_state as a vector of strings.
| Arun Isaac |
