I want some templates of different file formats that I can use to test my scripts and identify possible bugs in my code.
For example, consider nucleotide FASTA, a simple but often abused format, I would want templates to capture regular and irregular formats, like I have seen all of these:
1) Single line sequence
2) Multi-line sequence
>1 AT G
3) Upper and lower case letters in sequence
4) Ns and Xs (and possibly other letters) in sequence
5) Unusual headers (sometimes non-ASCI characters, need to consider the encoding)
>ATG >汉字 ATG
6) Whitespace between records
>1 ATG >2 ATG
7) Duplicated headers
>1 ATG >1 ATC
8) Empty headers or sequences (valid FASTA?)
9) No new line
'\n' character on last line (can mess-up file concatenation)
>1 A# < no new-line here
10) Different newline characters depending on the OS
>1 A# \r\n vs \n
There should be separate templates for nucleotide and protein FASTA, and separate ones for aligned FASTA.
It would ideally include other aspects too, like different compression formats (such as
.bzip2) and different file extensions (such as
I have never seen resources that provide templates covering these, but I think it would be useful. Of course I could build my own templates but it would take time to capture all the likely variations of the formats, particularly for more complex file formats.
Note, I am not just interested in FASTA format, it was an example.
Also note, I know about tools (such as
BioPython) that should handle many formats well, but they may also have bugs. Anyway, in practice sometimes I end up parsing files myself directly because I don't want the overhead or dependency of an external package.
EDIT: Please don't answer this question to say you don't know of any such resources, me neither, hence the question. bli's helpful answer shows that there is at least one test suite that could be used as a starting point. I know that it is normally easy to look up the specification of any particular file format.
>and can contain anything they like except a
\n. ii) sequence lines should be 60 characters per line. Nothing else is relevant, not the number of sequence lines, not the characters in the header, not the extension (which is usually irrelevant outside the Windows world anyway), there is no constraint on what characters a sequence can have (hence no difference between protein and nucleotide fasta specifications). $\endgroup$
globfiles with just 1 particular suffix for example. $\endgroup$
.fabut there is no standard for this. All your tools need to do is i) take the entire line after a
>as a sequence name and ii) everything else as the sequence. That's all, and that's all it take to be fasta. $\endgroup$