How to retrieve sequences from a txt file by protein names from other txt file?

Question

My problem is that I want to retrieve the sequences line + secondary structure line that match the protein names from the header.txt file:

The text file(header.txt) contains the protein names that look like this:

1XXIJ
1P0EA
1H9HI

And the other file(pdb.txt) structured like this:

Line 1:the protein name

Line 2: protein length

Line 3: amino acid sequence

Line 4: secondary structure

Line 5: empty

4LGTD
247
M       S       E       K       L       Q       K       V       L       A       R       
.       .       E       E       H       H       H       H       H       H       H       

1XXIJ
334
M       R       W       Y       P       W       L       R       P       D       F       
.       .       .       .       T       T       H       H       H       H       H       

6U02G
116
E       V       Q       L       V       E       S       G       G       G       L       
.       .       E       E       E       E       E       .       .       E       E       

1P0EA
363
R       P       R       F       S       F       S       I       A       A       R       
.       .       S       E       E       E       E       E       E       E       E  
 
1H9HI
30
G       C       P       R       I       L       I       R       C       K       Q       
.       E       E       .       .       .       .       B       S       S

So my expected output is:

1XXIJ
334
M       R       W       Y       P       W       L       R       P       D       F       
.       .       .       .       T       T       H       H       H       H       H       

1P0EA
363
R       P       R       F       S       F       S       I       A       A       R       
.       .       S       E       E       E       E       E       E       E       E  
 
1H9HI
30
G       C       P       R       I       L       I       R       C       K       Q       
.       E       E       .       .       .       .       B       S       S

Do you guys have any suggestions for me by awk or any(Linux)code? Please let me know if you require additional information. Cheers!

Answer 1

If your file structure is consistent, one potential solution is to use GNU grep:

grep -A4 --no-group-separator -Fwf header.txt pdb.txt

This works with your example data and should be relatively fast on your 'real' data, due to -Fw ('fixed' i.e. not regex, and only match whole words) but it requires GNU grep for the --no-group-separator option (i.e. not BSD grep).

Answer 2

I'm assuming that your second file is tab-delimited and has no commas. The easiest way to do this would be to transpose each record in your second file to a single line and then grep, then transform the result again:

grep -Ff header.txt <(paste -d, - - - - - < pdb.txt) | tr "," "\n"

I just tested and it works:

➜ cat pdb.txt
4LGTD
247
M       S       E       K       L       Q       K       V       L       A       R
.       .       E       E       H       H       H       H       H       H       H

1XXIJ
334
M       R       W       Y       P       W       L       R       P       D       F
.       .       .       .       T       T       H       H       H       H       H

6U02G
116
E       V       Q       L       V       E       S       G       G       G       L
.       .       E       E       E       E       E       .       .       E       E

1P0EA
363
R       P       R       F       S       F       S       I       A       A       R
.       .       S       E       E       E       E       E       E       E       E

1H9HI
30
G       C       P       R       I       L       I       R       C       K       Q
.       E       E       .       .       .       .       B       S       S

➜ cat header.txt
1XXIJ
1P0EA
1H9HI

➜ grep -Ff header.txt <(paste -d, - - - - - < pdb.txt) | tr "," "\n"
1XXIJ
334
M       R       W       Y       P       W       L       R       P       D       F
.       .       .       .       T       T       H       H       H       H       H

1P0EA
363
R       P       R       F       S       F       S       I       A       A       R
.       .       S       E       E       E       E       E       E       E       E

1H9HI
30
G       C       P       R       I       L       I       R       C       K       Q
.       E       E       .       .       .       .       B       S       S
```

Answer 3

One way using awk:

awk 'FNR==NR { a[$1]; next } FNR%5==1 { f = ($1 in a) ? 1 : 0 }f' header.txt pdb.txt 

This adds each of the protein names in header.txt to an array. By adding the first field ($1), rather than the complete record ($0) to the array, we can ensure that we strip off any lagging whitespace which can sometimes cause problems depending on how the list of protein names was created. Next, we can check whether the record number (i.e. line number) of the second input file is a multiple of 5 with remainder 1. This condition returns true for the first line of each record in pdb.txt. We can then use a ternary operator to set a flag variable to one if the first field of the current line (i.e., the protein name) is present in the a array, or to zero to indicate that the current record is not one of the records in the array. The f by itself is just shorthand for f != 0 { print }, which prints the current line when the flag is non-zero.