12
$\begingroup$

I have a fasta file like

>sample 1 gene 1
atgc
>sample 1 gene 2
atgc
>sample 2 gene 1 
atgc

I want to get the following output, with one break between the header and the sequence.

>sample 1 gene 1   atgc
>sample 1 gene 2   atgc
>sample 2 gene 1   atgc
$\endgroup$
1
  • $\begingroup$ Thanks so much everyone. You're making it hard to choose. I wanted something for a multi line fasta so both terdon and Chris scripts are correct. So I will defer to the saying first come first served. $\endgroup$
    – AudileF
    Oct 18 '17 at 12:29
11
$\begingroup$

If you have multi-line fasta files, as is very common, you can use these scripts1 to convert between fasta and tbl (sequence_name <TAB> sequence) format:

  • FastaToTbl

    #!/usr/bin/awk -f
    {
            if (substr($1,1,1)==">")
        		if (NR>1)
                        	printf "\n%s\t", substr($0,2,length($0)-1)
        		else 
        			printf "%s\t", substr($0,2,length($0)-1)
                else 
                        printf "%s", $0
    }END{printf "\n"}
    
  • TblToFasta

    #! /usr/bin/awk -f
    {
      sequence=$NF
    
      ls = length(sequence)
      is = 1
      fld  = 1
      while (fld < NF)
      {
         if (fld == 1){printf ">"}
         printf "%s " , $fld
    
         if (fld == NF-1)
          {
            printf "\n"
          }
          fld = fld+1
      }
      while (is <= ls)
      {
        printf "%s\n", substr(sequence,is,60)
        is=is+60
      }
    }
    

Save those in your $PATH, make them executable, and you can then do:

$ cat file.fa
>sequence1 
ATGCGGAGCTTAGATTCTCGAGATCTCGATATCGCGCTTATAAAAGGCCCGGATTAGGGC
TAGCTAGATATCGCGATAGCTAGGGATATCGAGATGCGATACG
>sequence2 
GTACTCGATACGCTACGCGATATTGCGCGATACGCATAGCTAACGATCGACTAGTGATGC
ATAGAGCTAGATCAGCTACGATAGCATCGATCGACTACGATCAGCATCAC
$ FastaToTbl file.fa 
sequence1   ATGCGGAGCTTAGATTCTCGAGATCTCGATATCGCGCTTATAAAAGGCCCGGATTAGGGCTAGCTAGATATCGCGATAGCTAGGGATATCGAGATGCGATACG
sequence2   GTACTCGATACGCTACGCGATATTGCGCGATACGCATAGCTAACGATCGACTAGTGATGCATAGAGCTAGATCAGCTACGATAGCATCGATCGACTACGATCAGCATCAC

And, to get the Fasta back:

$ FastaToTbl file.fa | TblToFasta
>sequence1 
ATGCGGAGCTTAGATTCTCGAGATCTCGATATCGCGCTTATAAAAGGCCCGGATTAGGGC
TAGCTAGATATCGCGATAGCTAGGGATATCGAGATGCGATACG
>sequence2 
GTACTCGATACGCTACGCGATATTGCGCGATACGCATAGCTAACGATCGACTAGTGATGC
ATAGAGCTAGATCAGCTACGATAGCATCGATCGACTACGATCAGCATCAC

This can be a very useful trick when searching a fasta file for a string:

TblToFasta file.fa | grep 'foo' | FastaToTbl

If you really want to keep the leading > of the header (which doesn't seem very useful), you could do something like this:

$ perl -0pe 's/\n//g; s/.>/\n>/g; s/$/\n/;' file.fa 
>sequence1 ATGCGGAGCTTAGATTCTCGAGATCTCGATATCGCGCTTATAAAAGGCCCGGATTAGGGCTAGCTAGATATCGCGATAGCTAGGGATATCGAGATGCGATAC
>sequence2 GTACTCGATACGCTACGCGATATTGCGCGATACGCATAGCTAACGATCGACTAGTGATGCATAGAGCTAGATCAGCTACGATAGCATCGATCGACTACGATCAGCATCAC

But that will read the entire file into memory. If that's an issue, add an empty line between each fasta record, and then use perl's paragraph mode to process each "paragraph" (sequence) at a time:

perl -pe  's/>/\n>/' file.fa | perl -00pe 's/\n//g; s/.>/\n>/g; s/$/\n/;'

1Credit to Josep Abril who wrote these scripts more than a decade ago.

$\endgroup$
10
$\begingroup$

There is a very simple BioPython solution, that is minimal, readable, and handles multi-line fasta:

from Bio import SeqIO

for record in SeqIO.parse('example.fa', 'fasta'):
    print('>{}\t{}'.format(record.description, record.seq))
$\endgroup$
8
$\begingroup$

assuming there is only one sequence line per record, use paste with two 'stdin'

cat your.fasta | paste - -
$\endgroup$
1
  • 2
    $\begingroup$ Note that this will fail if you have multi line sequences (as Pierre pointed out), but also if you have any blank lines in the file. You might also want to remove the UuOC: paste - - < file.fa. $\endgroup$
    – terdon
    Oct 17 '17 at 12:24
7
$\begingroup$

You can use these commands:

perl -pe 's/>(.*)/>\1\t/g; s/\n//g; s/>/\n>/g' sequences.fa | grep -v '^$'

Explanation:

  1. Append a tab to every header line
  2. Join all lines
  3. Split the single obtained line by the '>' character
  4. Remove the empty line (the first line is empty due to the fact that '>' is the first character of the FASTA file)
$\endgroup$
5
$\begingroup$

A very useful tool for this kind of data manipulation is bioawk:

$ bioawk -c fastx '{print ">"$name" "$comment"\t"$seq}' test.fa
>sample 1 gene 1    atgc
>sample 1 gene 2    atgc
>sample 2 gene 1    atgc

bioawk is based on awk, with added parsing capabilities. Here, we tell that the format is fasta or fastq with -c fastx, and this makes the $name (between ">" and the first blank character), $comment (after the first blank character) and $seq (the sequence, in one line) variables available within awk instructions.

See for instance this answer for another use case.

$\endgroup$
3
$\begingroup$

Where possible, I recommend using a dedicated parsing library, rather than hacking a parser together: as you can see in the other answers, parsing even simple formats gets complex pretty quickly if you value correctness.

Here’s a small R script that does what we need, using ‘seqinr’:

#!/usr/bin/env Rscript
suppressPackageStartupMessages(library(seqinr))
parsed = read.fasta(file('stdin'), as.string = TRUE)
table = data.frame(unlist(parsed), row.names = sapply(parsed, attr, 'Annot'))
write.table(table, stdout(), sep = '\t', quote = FALSE, col.names = FALSE)

Save it as fasta-to-tsv, make it executable, and use it as follows:

fasta-to-tsv < input.fasta > output.tsv

Equivalent code of similar length can be written in Python or Perl.

$\endgroup$
5
  • $\begingroup$ Could you also explain what packages need to be installed in order for R to do this? seqinr isn't part of vanilla R. $\endgroup$
    – terdon
    Jun 14 '20 at 18:03
  • $\begingroup$ @terdon I’m slightly confused: ‘seqinr’ is the package name, so that’s the one to install. $\endgroup$ Jun 14 '20 at 18:08
  • $\begingroup$ Well, sometimes the name of the package isn't the same as the library you load from it. And in any case, R being the horribly complicated mess that it is, you can never know (or I can't, anyway) whether the package can be installed with install.package or needs to be done through bioconductor or whatever else may be the case. R package management is very far from trivial, so I thought it would be useful to explain how to install the packages necessary to run your code. I've edited the command in now. I'd upvote, but I already did so back in 2017 :) $\endgroup$
    – terdon
    Jun 14 '20 at 18:26
  • $\begingroup$ @terdon “sometimes the name of the package isn't the same as the library you load from it” — No, for R it always is, by definition. As for installation, I’ll give you that there’s a special case for packages not on CRAN but it only really makes sense to spell out the exceptions, not the rule. You wouldn’t mention installation instructions for Perl packages on CPAN, nor for Python on PyPI, NPM for JavaScript or Rust crates. I’ve undone your edit because (a) it’s redundant, and (b) it was wrong. Never install packages via sudo R unless you’re the sysadmin maintaining a cluster. $\endgroup$ Jun 14 '20 at 19:29
  • $\begingroup$ Ah, I tried it without but it failed when trying to install to my user's local library, so I assumed the package could only be installed system wide. But that just proves why it is helpful to include the installation command. I don't know R (and hate it with a passion) so all these things that you mention which are obvious to you, are not to me. And possibly others, which is why yes, I always include commands to install packages in my answers. $\endgroup$
    – terdon
    Jun 14 '20 at 23:49
2
$\begingroup$

Remove empty records (description without sequence):

awk '$2{print RS}$2' FS='\n' RS=\> ORS= f1.fa > f2.fa

Remove blank lines:

sed '/^$/d' f2.fa > f3.fa

Convert multi-line fasta to single-line fasta:

awk '/^>/ {printf("\n%s\n",$0);next; } { printf("%s",$0);} END {printf("\n");}' f3.fa > f4.fa

Finally, @Pierre solution:

cat f4.fa | paste - - > f.txt
$\endgroup$
0
$\begingroup$

In cases where there is no sequence wrapping and each sequence occupies only a single line, the following shell command is probably going to be fastest, easiest, and most convenient.

paste - - < your.fasta > your.new.fasta
$\endgroup$
1
  • $\begingroup$ Oops, looks like Pierre already suggested this. :-) $\endgroup$ May 23 '19 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.