blob: 124aba0b8dff91b5606247ce051ca58aa1425cc8 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
|
#+OPTIONS: ^:nil
# INSTALLATION
Other options for running this tool.
## GNU Guix
### Running the CLI uploader
Another way to install this tool is inside a [GNU Guix Environment](https://guix.gnu.org/manual/en/html_node/Invoking-guix-environment.html), which can handle installing dependencies for you
1. **Set up and enter a Guix environment with the necessary dependencies.** After installing Guix run:
```sh
guix environment -C guix --ad-hoc git python openssl python-pycurl nss-certs
```
2. **Install the tool.** From there you can follow the [user installation instructions](#installation-with-pip3---user). In brief:
```sh
pip3 install --user schema-salad arvados-python-client
```
Pip installed the following modules
```
arvados-python-client-2.0.1 ciso8601-2.1.3 future-0.18.2 google-api-python-client-1.6.7 httplib2-0.17.1 oauth2client-4.1.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 rsa-4.0 ruamel.yaml-0.15.77 six-1.14.0 uritemplate-3.0.1 ws4py-0.5.1
```
3. Run the tool directly with
```sh
guix environment guix --ad-hoc git python openssl python-pycurl python-magic nss-certs python-pyshex -- python3 bh20sequploader/main.py example/maximum_metadata_example.yaml example/sequence.fasta
```
Note that python-pyshex is packaged in
http://git.genenetwork.org/guix-bioinformatics/guix-bioinformatics
so you'll need it to the GUIX_PACKAGE_PATH - see the README in that
repository. E.g.
```sh
env GUIX_PACKAGE_PATH=~/iwrk/opensource/guix/guix-bioinformatics/ ~/opt/guix/bin/guix environment -C guix --ad-hoc git python python-flask python-pyyaml python-pycurl python-magic nss-certs python-pyshex python-pyyaml --network openssl python-pyshex python-pyshexc minimap2 python-schema-salad python-arvados-python-client --share=/export/tmp -- env TMPDIR=/export/tmp python3 bh20sequploader/main.py --help
```
Latest successful Guix run
```sh
env GUIX_PACKAGE_PATH=~/iwrk/opensource/guix/guix-bioinformatics/ ~/opt/guix/bin/guix environment guix --ad-hoc git python openssl python-pycurl python-magic nss-certs python-pyshex python-arvados-python-client python-schema-salad minimap2 -- python3 bh20sequploader/main.py scripts/uthsc_samples/yaml/AL_UT14.yaml scripts/uthsc_samples/yaml/AL_UT14.fa
```
### Using the Web Uploader
To run the web uploader in a GNU Guix environment/container run it with something like
```
guix environment guix --ad-hoc git python python-flask python-pyyaml python-pycurl python-magic nss-certs --network openssl -- env FLASK_ENV=development PYTHONPATH=$PYTHONPATH:./bh20sequploader FLASK_APP=bh20simplewebuploader/main.py flask run
* Serving Flask app "bh20simplewebuploader/main.py"
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
```
WIP: add gunicorn container
Currently the full webserver container deploy command looks like
```
penguin2:~/iwrk/opensource/code/vg/bh20-seq-resource$ env GUIX_PACKAGE_PATH=~/iwrk/opensource/guix/guix-oinformatics/ ~/iwrk/opensource/guix/guix/pre-inst-env guix environment -C guix --ad-hoc git python python-flask python-pyyaml python-pycurl python-magic nss-certs python-pyshex python-pyyaml --network openssl python-pyshex python-pyshexc minimap2 python-schema-salad python-arvados-python-client --share=/export/tmp -- env TMPDIR=/export/tmp FLASK_ENV=development FLASK_APP=bh20simplewebuploader/main.py flask run
```
Note: see above on GUIX_PACKAGE_PATH.
## Run country semantic enrichment script
cd bh20-seq-resource/scripts/db_enrichment
edit input_location.csv
guix environment guix --ad-hoc git python nss-certs python-rdflib -- python3 country_enrichment.py
## Run the tests
guix package -i python-requests python-pandas python-jinja2 python -p ~/opt/python-dev
. ~/opt/python-dev/etc/profile
## Run Virtuoso-ose
Guix has a package for virtuoso-ose we use
guix package -i virtuoso-ose -p ~/opt/virtuoso
Create a data dir
mkdir -p /export/virtuoso/var/lib/virtuoso/db
chown $USER /export/virtuoso/var/lib/virtuoso/db
Add an ini file
cp ~/opt/virtuoso/var/lib/virtuoso/db/virtuoso.ini .config/
And run from the data dir
cd /export/virtuoso/var/lib/virtuoso/db
guix environment --ad-hoc virtuoso-ose -- virtuoso-t -f
Visit http://localhost:8890/sparql
To update the turtle files do
guix environment -C guix --ad-hoc python python-requests raptor2 curl --network -- python3 ./scripts/update_virtuoso/check_for_updates.py cache.txt dba dba
where dba is the default password.
## Run the Auspice phylo-tree viewer
Auspice is developed by the Nextstrain team and we use it for
exploring the phylogenetic tree. It does not scale well beyond 5,000
SARS-CoV-2 virusses, so we need to limit the input set!
Auspice runs on a separate port.
Running on port 4001 using your repo:
Install node with Guix
~/.config/guix/current/bin/guix package -i node -p ~/opt/node
Load Guix environment variables
. ~/opt/node/etc/profile
Run server after fetching data in ./data
cd auspice
npm install --global --prefix node_modules .
env PORT=4001 ./node_modules/bin/auspice view --datasetDir ./data
|
