1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
|
#+TITLE: COVID-19 PubSeq Uploading Data (part 3)
#+AUTHOR: Pjotr Prins
# C-c C-e h h publish
# C-c ! insert date (use . for active agenda, C-u C-c ! for date, C-u C-c . for time)
# C-c C-t task rotate
#+HTML_HEAD: <link rel="Blog stylesheet" type="text/css" href="blog.css" />
* Uploading Data
/Work in progress!/
* Table of Contents :TOC:noexport:
- [[#uploading-data][Uploading Data]]
- [[#introduction][Introduction]]
- [[#step-1-upload-sequence][Step 1: Upload sequence]]
- [[#step-2-add-metadata][Step 2: Add metadata]]
- [[#obligatory-fields][Obligatory fields]]
- [[#optional-fields][Optional fields]]
- [[#step-3-submit-to-covid-19-pubseq][Step 3: Submit to COVID-19 PubSeq]]
- [[#step-4-check-output][Step 4: Check output]]
- [[#trouble-shooting][Trouble shooting]]
* Introduction
The COVID-19 PubSeq allows you to upload your SARS-Cov-2 strains to a
public resource for global comparisons. Compute it triggered on
upload. Read the [[./about][ABOUT]] page for more information.
* Step 1: Upload sequence
To upload a sequence in the [[http://covid19.genenetwork.org/][web upload page]] hit the browse button and
select the FASTA file on your local hard disk.
We start with an assembled or mapped sequence in FASTA format. The
PubSeq uploader contains a [[https://github.com/arvados/bh20-seq-resource/blob/master/bh20sequploader/qc_fasta.py][QC step]] which checks whether it is a likely
SARS-CoV-2 sequence. While PubSeq deduplicates sequences and never
overwrites metadata, you may still want to check whether your data
already is in the system by querying some metadata as described in
[[./blog?id=using-covid-19-pubseq-part1][Query metadata with SPARQL]] or by simply downloading and checking one
of the files on the [[./download][download]] page. We find GenBank [[https://www.ncbi.nlm.nih.gov/nuccore/MT536190][MT536190.1]] has not
been included yet. A FASTA text file can be [[https://www.ncbi.nlm.nih.gov/nuccore/MT536190.1?report=fasta&log$=seqview&format=text][downloaded]] to your local
disk and uploaded through our [[./][web upload page]]. Make sure the file does
not include any HTML!
Note: we currently only allow FASTA uploads. In the near future we'll
allow for uploading raw sequence files. This is important for creating
an improved pangenome.
* Step 2: Add metadata
The [[./][web upload page]] contains fields for adding metadata. Metadata is
not only important for attribution, is also important for
analysis. The metadata is available for queries, see [[./blog?id=using-covid-19-pubseq-part1][Query metadata
with SPARQL]], and can be used to annotate variations of the virus in
different ways.
A number of fields are obligatory: sample id, date, location,
technology and authors. The others are optional, but it is valuable to
enter them when information is available. Metadata is defined in this
[[https://github.com/arvados/bh20-seq-resource/blob/master/bh20sequploader/bh20seq-schema.yml][schema]]. From this schema we generate the input form. Note that
opitional fields have a question mark in the ~type~. You can add
metadata yourself, btw, because this is a public resource! See also
[[./blog?id=using-covid-19-pubseq-part5][Modify metadata]] for more information.
To get more information about a field click on the question mark on
the web form. Here we add some extra information.
** Obligatory fields
*** Sample ID (sample_id)
This is a string field that defines a unique sample identifier by the
submitter. In addition to sample_id we also have host_id,
provider_sample_id and submitter_sample_id where host is the host the
sample came from, provider sample is the institution sample id and
submitter is the submitting individual id. host_id is important when
multiple sequences come from the same host. Make sure not to have
spaces in the sample_id.
Here we add the GenBank ID MT536190.1.
*** Collection date
Estimated collection date. The GenBank page says April 6, 2020.
*** Collection location
A search on wikidata says Los Angelos is
https://www.wikidata.org/entity/Q65
*** Sequencing technology
GenBank entry says Illumina, so we can fill that in
*** Authors
GenBank entry says 'Lamers,S., Nolan,D.J., Rose,R., Cross,S., Moraga
Amador,D., Yang,T., Caruso,L., Navia,W., Von Borstel,L., Hui Zhou,X.,
Freehan,A. and Garcia-Diaz,J.', so we can fill that in.
** Optional fields
All other fields are optional. But let's see what we can add.
*** Host information
Sadly, not much is known about the host from GenBank. A little
sleuthing renders an interesting paper by some of the authors titled
[[https://www.medrxiv.org/content/10.1101/2020.04.24.20078691v1][SARS-CoV-2 is consistent across multiple samples and methodologies]]
which dates after the sample, but has no reference other than that the
raw data came from the SRA database, so it probably does not describe
this particular sample. We don't know what this strain of SARS-Cov-2
did to the person and what the person was like (say age group).
*** Collecting institution
We can fill that in.
*** Specimen source
We have that: nasopharyngeal swab
*** Source database accession
Genbank which is http://identifiers.org/insdc/MT536190.1#sequence.
Note we plug in our own identifier MT536190.1.
*** Strain name
SARS-CoV-2/human/USA/LA-BIE-070/2020
* Step 3: Submit to COVID-19 PubSeq
Once you have the sequence and the metadata together, hit
the 'Add to Pangenome' button. The data will be checked,
submitted and the workflows should kick in!
* Step 4: Check output
The current pipeline takes 5.5 hours to complete! Once it completes
the updated data can be checked on the [[./download][DOWNLOAD]] page. After completion
of above output this [[http://sparql.genenetwork.org/sparql/?default-graph-uri=&query=PREFIX+pubseq%3A+%3Chttp%3A%2F%2Fbiohackathon.org%2Fbh20-seq-schema%23MainSchema%2F%3E%0D%0APREFIX+sio%3A+%3Chttp%3A%2F%2Fsemanticscience.org%2Fresource%2F%3E%0D%0Aselect+distinct+%3Fsample+%3Fp+%3Fo%0D%0A%7B%0D%0A+++%3Fsample+sio%3ASIO_000115+%22MT536190.1%22+.%0D%0A+++%3Fsample+%3Fp+%3Fo+.%0D%0A%7D&format=text%2Fhtml&timeout=0&debug=on&run=+Run+Query+][SPARQL query]] shows some of the metadata we put
in.
** Trouble shooting
We got an error saying: {"stem": "http://www.wikidata.org/entity/",...
which means that our location field was not formed correctly! After
fixing it to look like http://www.wikidata.org/entity/Q65 (note http
instead on https and entity instead of wiki) the submission went
through. Reload the page (it won't empty the fields) to re-enable the
submit button.
|
