| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
|
|
"more than one" is simpler English.
|
|
|
|
|
|
|
|
|
|
This will enable us to effect global control over the log level.
|
|
Implement logging system with five log levels. Rewrite warning and
user-error in terms of the new logging functions.
|
|
read-workflow and read-inputs now handle &unsupported-uri-scheme
conditions from location->path.
|
|
location->path now raises an &unsupported-uri-scheme condition on
unsupported URI schemes.
|
|
|
|
|
|
read-yaml-file is a wrapper around read-yaml-file from (yaml). It
raises the relevant exceptions when it fails.
|
|
|
|
read-json-file is a wrapper around call-with-input-file and json->scm.
It raises the relevant exceptions when it fails.
|
|
Return monadic value for the default key in normalize-formal-input.
Values returned from maybe-bind must be monadic; compose it with just.
|
|
* ravanan/workflow.scm (workflow-scheduler): Copy resource-requirement
field on scatter steps.
|
|
It is not the responsibility of load-script to canonicalize file
paths; the caller must do that.
* ravanan/utils.scm (load-script): Do not canonicalize script file
path.
|
|
* ravanan/reader.scm (normalize-software-requirement): New function.
(normalize-requirements): Use normalize-software-requirement.
|
|
* ravanan/reader.scm (normalize-env-var-requirement): Re-indent.
|
|
* ravanan/store.scm (step-store-basename): Canonicalize inputs JSON.
* tests/store.scm: New file.
|
|
*
ravanan/command-line-tool.scm (build-command-line-tool-script)[copy-input-files]:
Pass down inputs directory to recursive calls.
|
|
The comment was there earlier, but the code was missing!
* ravanan/command-line-tool.scm (manifest-file->search-path-sexps): Do
not auto-compile manifest files on inferior.
|
|
Canonicalization is unnecessary, and causes trouble when the manifest
file does not exist.
* ravanan/command-line-tool.scm (manifest-file->search-path-sexps,
manifest-file->profile-derivation): Do not canonicalize manifest file
paths.
|
|
*
ravanan/command-line-tool.scm (build-command-line-tool-script)[vector->gexp,
alist->gexp]: New functions.
[run-command-gexp]: Use vector->gexp and alist->gexp.
|
|
* ravanan/command-line-tool.scm (build-gexp-script)
(build-command-line-tool-script): Present a monadic interface.
* ravanan/workflow.scm (workflow->scheduler-proc): Run monadic value
returned by build-command-line-tool-script through the store.
(workflow->scheduler-proc, workflow-class->propnet): Update docstring
about the guix-store argument.
|
|
In the interest of consistency, return search path S-expressions from
both manifest-file->search-path-sexps and
software-packages->search-path-sexps as monadic values.
*
ravanan/command-line-tool.scm (software-packages->search-path-sexps):
Return monadic value.
* ravanan/command-line-tool.scm (build-command-line-tool-script): Use
monadic values from software-packages->search-path-sexps.
|
|
The monadic interface allows more of the building to be done in one go
when build-gexp-script is invoked.
* ravanan/command-line-tool.scm (manifest-file->environment,
software-packages->environment, manifest->environment): Delete
functions.
(inferior-meval-with-store, manifest-file->search-path-sexps,
manifest-file->profile-derivation, software-packages->manifest,
software-packages->search-path-sexps,
software-packages->profile-derivation): New functions.
(build-command-line-tool-script): Use manifest-file->search-paths,
manifest-file->profile-derivation,
software-packages->search-path-sexps and
software-packages->profile-derivation instead of
manifest-file->environment and software-packages->environment.
|
|
Re-using the connection improves performance and is cleaner than
passing around the Guix daemon socket specification.
* ravanan/command-line-tool.scm (manifest-file->environment,
software-packages->environment, manifest->environment,
build-command-line-tool-script, build-gexp-script): Work with store
connection instead of the Guix daemon socket.
* ravanan/workflow.scm: Import (guix store).
(workflow->scheduler-proc, workflow-class->propnet): Work with store
connection instead of the Guix daemon socket.
(build-workflow): Pass a store connection, not the Guix daemon socket,
to workflow->scheduler-proc.
|
|
Spawning inferiors is expensive. Do it only once and re-use the
connection.
* ravanan/command-line-tool.scm (call-with-inferior): Move to (ravanan
workflow).
(manifest-file->environment, software-packages->environment,
build-command-line-tool-script): Accept inferior instead of channels.
* ravanan/workflow.scm: Import (guix inferior).
(workflow->scheduler-proc, workflow-class->propnet): Accept inferior
instead of channels.
(build-workflow): New function.
(run-workflow): Use build-workflow instead of
workflow->scheduler-proc.
|
|
* ravanan/command-line-tool.scm (build-command-line-tool-script):
Clarify it is "disabling WorkReuse" that is unnecessary.
|
|
* ravanan/command-line-tool.scm (build-command-line-tool-script,
find-requirement): Export.
(run-command-line-tool): Accept script as argument instead of cwl. Add
resource-requirement argument. Remove manifest-file, channels, scratch
and guix-daemon-socket arguments.
* ravanan/workflow.scm (<scheduler-proc>):[cwl-or-propnet]: Rename to
script-or-propnet. [resource-requirement]: New field.
* ravanan/workflow.scm (scheduler-proc): Rename cwl-or-propnet
argument to script-or-propnet. Add resource-requirement argument.
(workflow->scheduler-proc, workflow-class->propnet): Add
manifest-file, channels, scratch, store and guix-daemon-socket
arguments.
(workflow->scheduler-proc): Build script. Capture ResourceRequirement.
(workflow-scheduler): Remove manifest-file, channels, scratch and
guix-daemon-socket arguments.
[schedule]: Run built script instead of CWL. Update call to
run-command-line-tool.
(run-workflow): Do not pass manifest-file, channels, scratch and
guix-daemon-socket arguments to workflow-scheduler. Pass
manifest-file, channels, scratch, store and guix-daemon-socket
arguments to workflow->scheduler-proc.
|
|
* ravanan/workflow.scm (workflow->scheduler-proc): Make scatter and
scatter-method arguments optional.
|
|
* ravanan/workflow.scm (<scheduler-proc>): Rename constructor to
-scheduler-proc.
(scheduler-proc): New function.
|
|
Build out propagator network eagerly descending into the lowest
subworkflows. This will come in handy later on to build the G-exp
scripts ahead of time before the workflow is run.
* ravanan/workflow.scm (<scheduler-proc>)[cwl]: Rename to
cwl-or-propnet.
[formal-inputs, formal-outputs]: New fields.
* ravanan/workflow.scm (workflow->scheduler-proc): New function.
(workflow-class->propnet): Build out propagator network eagerly.
(workflow-scheduler)[schedule]: Handle eagerly built-out propagator
network <schedule-proc> objects.
(run-workflow): Use workflow->scheduler-proc to convert workflow to
<scheduler-proc> object.
|
|
command-line-tool->propagator was packaging up both CommandLineTool
and Workflow class workflows into <scheduler-proc> objects. It was
named erroneously. We now merge it into step->propagator.
* ravanan/workflow.scm (command-line-tool->propagator): Merge into ...
(workflow-class->propnet)[step->propagator]: ... here.
|
|
We build the script using only the CommandLineTool workflow itself,
and not its inputs. We pass the inputs in through a WORKFLOW_INPUTS
environment variable. To maintain bullet-proof caching, we construct
ravanan store paths by hashing both the script and the inputs
together. This change paves the way to pre-building all scripts ahead
of time.
* ravanan/command-line-tool.scm (<command-line-binding>,
command-line-binding->args): Move to (ravanan work command-line-tool).
(run-command-line-tool): Do not pass inputs to
build-command-line-tool-script. Call step-store-files-directory
instead of script->store-files-directory, step-store-data-file instead
of script->store-data-file, step-store-stdout-file instead of
script->store-stdout-file and step-store-stderr-file instead of
script->store-stderr-file. Put inputs into job state objects. Pass
inputs to the script through the WORKFLOW_INPUTS environment variable.
(capture-command-line-tool-output): Accept inputs. Call
step-store-data-file instead of script->store-data-file and
step-store-files-directory instead of script->store-files-directory.
(copy-input-files-gexp): Delete function.
(build-command-line-tool-script): Do not accept inputs; get them from
the WORKFLOW_INPUTS environment variable.
[coerce-argument]: New function.
[run-command-gexp]: Build command inside the G-expression.
[gexp]{copy-input-files}: New function.
* ravanan/job-state.scm (<single-machine-job-state>)[inputs]: New
field.
* ravanan/job-state.scm (<slurm-job-state>)[inputs]: New field.
* ravanan/job-state.scm (job-state-inputs): New public function.
* ravanan/store.scm: Import (srfi srfi-71), (gcrypt hash) and (guix
build utils).
(sha1-hash-sexp): New function.
(script->store-files-directory, script->store-data-file,
script->store-stdout-file, script->store-stderr-file): Delete
functions.
(step-store-files-directory, step-store-data-file,
step-store-stdout-file, step-store-stderr-file): New public functions.
* ravanan/work/command-line-tool.scm: Import (srfi srfi-9 gnu)
and (ravanan work ui).
* ravanan/workflow.scm (&job-failure)[inputs]: New field.
* ravanan/workflow.scm (workflow-scheduler)[poll]: Put inputs into
&job-failure condition.
[capture-output]: Call step-store-stdout-file instead of
script->store-stdout-file and step-store-stderr-file instead of
script->store-stderr-file. Pass inputs to
capture-command-line-tool-output.
(run-workflow): Call step-store-stdout-file instead of
script->store-stdout-file and step-store-stderr-file instead of
script->store-stderr-file.
|
|
* ravanan/command-line-tool.scm (run-command-line-tool): Define
environment only once.
|
|
* ravanan/store.scm: Import (guix base16) and (guix base32).
(intern-file): Encode store paths using base32 instead of base16.
|
|
* ravanan/work/command-line-tool.scm (sha1-hash-bytes): New public
function.
(sha1-hash): Use sha1-hash-bytes.
|
|
* ravanan/workflow.scm (workflow-scheduler): Bind class variable only
in the else arm of the if condition.
|
|
User errors are caused by incorrect input from the user, rather than
by actual bugs in the code. user-error is named after the similarly
named elisp function.
* bin/ravanan: Import (ravanan work ui).
(%options, invalid-option, main): Use user-error instead of error.
|
|
* ravanan/propnet.scm (<propagator>, <propnet-state>): Add record
printers.
|
|
* ravanan/propnet.scm (propnet?, propagator?, scheduler?,
state+status?): Export.
|
|
* bin/ravanan (print-short-usage): New function.
(main): Print usage when arguments are not matched.
|
|
*
ravanan/command-line-tool.scm (build-command-line-tool-script):[cores]:
Re-indent.
|
|
* ravanan/command-line-tool.scm (copy-input-files-gexp): Make inputs
subdirectory only if it does not exist.
|
|
* ravanan/command-line-tool.scm (copy-input-files-gexp): Copy input
files into their own subdirectories.
|
