How to filter out partially similar strings from two lists and make one list?

I have following two lists. List1

m64041_200717_231916/100664277/0_42364 id=2440
m64041_200717_231916/100008029/0_42112 id=244006
m64041_200717_231916/100665802/0_46854 id=2440067
m64041_200717_231916/100008044/0_60161 id=24400
m64041_200717_231916/100928073/0_11804 id=2440152
m64041_200717_231916/10094492/0_44157 id=2440199
m64041_200717_231916/100008373/4102_50601 id=2440234


List2

m64041_200717_231916/100664277/0_42364
m64041_200717_231916/100665802/0_46854
m64041_200717_231916/100928073/0_11804
m64041_200717_231916/10094492/0_44157


These are sequence read ids from a genome file. List1 contains all the ids from the List2 but with some id numbers (id=2440). So the sequence ids are only similar to some extent. I want to compare these two files and I need to filter out List2 sequence ids from List one (with id numbers) as below. Output.

m64041_200717_231916/100664277/0_42364 id=2440
m64041_200717_231916/100665802/0_46854 id=2440067
m64041_200717_231916/100928073/0_11804 id=2440152
m64041_200717_231916/10094492/0_44157 id=2440199


I will be grateful if someone can help me with this issue.

• It sounds easy but I don't really understand exactly what you want to compare. Can U expand? Feb 4 at 7:08
• For another example imagine there are two lists. List1. UnitedStates id=12, UnitedKingdom id=15, Spain id=20, France id=25, List2 UnitedStates, Spain I want to filter out UnitedStates id=12 and Spain id=20 using the List2. List1 and List2 are two files. Output should be, UnitedStates id=12, Spain id=20 Feb 4 at 7:15
• ah, so only keep whats in list 2? Feb 4 at 7:25
• Yeah. list2 will be the query. Feb 4 at 7:29

You really need to learn some python :)

with open('/path/list2.txt', 'r') as fin:
to_keep = set([])
for line in fin:

with open('/path/list1.txt', 'r') as fin:
with open('/path/list3.txt', 'w') as fout:
for line in fin:
if line.strip().split(' ')[0] in to_keep:
fout.write(line)

• Thank you very much. It worked. Yes, I need to learn python. Feb 4 at 17:59
• And another question. Do you know a way to filter out/remove those sequences from the original fasta file using the id list created from the above code? I have a piece of code I used earlier to remove sequences using a list. But since these ids contains a space, It's not working now. I tried to change it, but it didn't work. It'll be an easy thing for you,but I couldn't figure it out. Can you please help? Feb 4 at 18:09
• Sure. Post a new question with an example plz Feb 4 at 20:16
• Why would you hardcode two file names instead of just reading argv for the input file and printing to stdout so you can then redirect? Feb 5 at 15:34
• He asked for files Feb 5 at 19:16

There is no need for a highlevel language in such a simple case, an awk oneliner does just fine:

$$awk -F " " 'FNR==NR {a[1]; next}$$1 in a' List2 List1
m64041_200717_231916/100664277/0_42364 id=2440
m64041_200717_231916/100665802/0_46854 id=2440067
m64041_200717_231916/100928073/0_11804 id=2440152
m64041_200717_231916/10094492/0_44157 id=2440199


This can be done in python easily. If you have different text files then you can load them into a python list and follow from step 2

# Step 1
list1 = ["m64041_200717_231916/100664277/0_42364 id=2440",
"m64041_200717_231916/100008029/0_42112 id=244006",
"m64041_200717_231916/100665802/0_46854 id=2440067",
"m64041_200717_231916/100008044/0_60161 id=24400",
"m64041_200717_231916/100928073/0_11804 id=2440152",
"m64041_200717_231916/10094492/0_44157 id=2440199",
"m64041_200717_231916/100008373/4102_50601 id=2440234"]

list2 = ["m64041_200717_231916/100664277/0_42364",
"m64041_200717_231916/100665802/0_46854",
"m64041_200717_231916/100928073/0_11804",
"m64041_200717_231916/10094492/0_44157"]

# Step 2
split_list1 = [ele.split(" ")[0] for ele in list1 ]
mapdict = {split_list1[i]:list1[i] for i in range(len(list1))}
common = [value for value in split_list1 if value in list2]
required = [mapdict[x] for x in common]
print(required)