For pandas you simply load the fasta names and fasta sequences into separate arrays. The "fasta names" array becomes the index in a pandas Series, whilst the sequence are the values. The dataframe will correctly re-associated the names with each sequence. You load all the files into a single dataframe and use the duplicated() function then write out. The easiest way to do this is concatenate all your files into one via shell ... cat file*.fasta > concat_file.fa Python script ... from Bio import SeqIO import pandas as pd path = "/Users/username/location/concat_file.fa" outpath = "/Users/username/Desktop/duplicates.fasta" fasta_name = [] fasta_seq = [] for record in SeqIO.parse(path, "fasta"): fasta_name.append('>' + record.id) fasta_seq.append(record.seq) df = pd.Series(fasta_seq, index = fasta_name) df_dups = df.index.duplicated() mydups = df[df_dups] mydups.to_csv(outpath, sep="\n", index=True, header=False) ... easy. I checked it on a ~4000 sequence fasta file (dengue), duplicated the last sequence and the only sequence output was the very last sequence.