From this SO post. You can use seqkit to remove duplicate sequences with the command below: seqkit rmdup -s < sequences.txt > out.fa The rmdup option removes duplicates, and the -s option calls duplicates on the basis of sequence, ignoring differences in headers.

Only top voted, non community-wiki answers of a minimum length are eligible