I have a file with Ensembl gene and Transcript IDs :

enter image description here

I want to retain only the first entires from both the column so that the output looks like this:

ENSG00000003137 ENST00000001146
ENSG00000003402 ENST00000309955

and ignore the rest. Have tried awk awk '!a[$1$2]++' and versions of it. Nothing works. Kindly suggest some way to get the desired output.



2 Answers 2


Keep it simple:

awk '!a[$1]++' file.txt 

If you have a different command line separator you need to specify with the -F option.


awk -F' ' '!a[$1]++' text.txt 


awk -F'\t' '!a[$1]++' text.txt 

If you don't have the first column in order maybe should be a good idea sort first.

sort text.txt | awk -F' ' '!a[$1]++'

The name of the array in this case is a. There is a better explanation in this post.


That sound like a job for a perl one-liner:

perl -ane 'unless(exists $seen{$F[0]}){print $_; $seen{$F[0]}++}'

The input is read line-wise and printed back to the command line if the ID in the first column has not been seen before.

  • perl -ane ' reads input line-wise and splits fields to array @F
  • unless(exists $seen{$F[0]}){ checks if ID exists in hash %seen and continues if that is not the case
  • print $_; prints the original input to STDOUT
  • $seen{$F[0]}++}' adds the ID to the hash %seen

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.