1
$\begingroup$

This is the one i used for a single bam file to filter its mitochondrial as well as unassembled read

samtools view -h SRR2920466.bam | awk '{if($3!= "chrM" && $3 != "chrUn"){print $0}}' | samtools view -Shb - > this.filter.bam

Is there any way i can do it on multiple files with parallel as i do to index and sort bam files using samtools .

So i tried this ,which is of course not working as its not able to pipe properly because it shows me error regarding additional fonts

ls *.bam | parallel "samtools view -h {} | awk '{if($3 != "chrM" && $3 != "chrUn"){print $0}}'| samtools view -Shb {.}.bam"

Any help with my above codes? As i get it there is wrong with my pipe .

After updating the samtools to version 1.9 i ran the small script

i get this error

awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                    ^ unterminated regexp
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                                   ^ unexpected newline or end of string
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                    ^ unterminated regexp
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                                   ^ unexpected newline or end of string
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                    ^ unterminated regexp
awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:                                                   ^ unexpected newline or end of string

I have 30 bam files i tried with 2 to see if its working or not , after the suggestion of @WouterDeCoster i tried the above one after updating samtools . the basic samtool with egrep works but for multiple file i couldn;t do it.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

Your awk command is inside double quotes ("") which means that all of the awk field variables (e.g. $3) are being interpreted by the shell. Since $3 is not set in the shell, your awk is trying to compare an empty string and that's why you get the error:

awk: cmd. line:1: {if( != chrM &&  != chrUn){print ./index_sort.sh}}
awk: cmd. line:1:      ^ syntax error

Note how the error message shows if( != chrM and not if($3 != chrM. Since you need to use double quotes here because you need both an outer (for parallel) and an inner (for awk) quote, and single quotes can't be nested, you will have to escape the awk variables:

ls *.bam | parallel "samtools view -h {} | 
    awk '{if(\$3 != "chrM" && \$3 != "chrUn"){print \$0}}' | samtools view -Shb {.}.bam"
Improve this answer
$\endgroup$
7
  • $\begingroup$ looks like i need to clear my fundamental awk concepts let me run and will update you. now i ran your code i get output like this "�&Ct�wh���x�Y#��J�f���{/.i��7��%M�������C�̛�<kbb�������B��f����<#?��� :P�y��� u��O��r.�_2ؑ�xu�zl�,xϭ"2�*�Uo�ȩ�@ayi��� /0���/���۶� �>�Xh� ��Ί�5h�f~���euSa1����;5��{���R�3L����p%��[<�1)M�y!�w��#���_�`�V5rj��v�}� S-���y)�Nμ4��Jӕ�D���*n�1��ޥ�g�����y���䒋�N��)���<:���mʲ�溎[�ʘ���K��[punit@localhost chrm]$ 62;c62;c62;c62;c^C " $\endgroup$
    – kcm
    Apr 16, 2019 at 11:01
  • 1
    $\begingroup$ @krushnachChandra well yes, of course. You are printing a bam file which is compressed binary data. What were you expecting to see? If you want sam output, remove the b from the options you give to samtools view. The b makes it print bam output. $\endgroup$
    – terdon ♦
    Apr 16, 2019 at 11:12
  • $\begingroup$ so i idea is after filtering the mitochondrial read and unassembled one i want bam output written to file with bam format which will be using for downstream analysis.I don't want to print it on the terminal . tried it with -o to write the output seems like i couldn;t make it work.. $\endgroup$
    – kcm
    Apr 16, 2019 at 11:14
  • 1
    $\begingroup$ @krushnachChandra -o should work. Also, you could redirect in the parallel command: ls *.bam | parallel "samtools view -h {} | awk '{if(\$3 != "chrM" && \$3 != "chrUn"){print \$0}}' | samtools view -Shb > {}.bam". But as I said under your question, the new error is completely unrelated. You probably have a malformed bam file. But please ask a new question about that. Even better, run for bam in *bam; do samtools view -h "$bam" | head -n1 || echo "$bam file is corrupted"; done That should help you find the bad file. $\endgroup$
    – terdon ♦
    Apr 16, 2019 at 11:19
  • $\begingroup$ will run it and see $\endgroup$
    – kcm
    Apr 16, 2019 at 11:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.