I'd like to calculate per site Fst(fixation index) to investigate genomic differentiation between two populations(haploid).

I did SNP calling and got vcf file. In the vcf file, multi allelic sites exist:

1   154 .   G   "A, T"

For example this site is triallelic sites.

  1. Could I calculate Fst in this site by normalizing vcf file?

Normalization means dividing into 2 lows. REF G ALT A(T would be convert G) REF G ALT T(A would be convert G)

In calculation using Popgenome package, we can specify missing sites(Missing sites would not be included in the calculation). In this case, which is better: replacing with 'REF' or replacing with missing sites?

  1. Why most studies calculate only biallelic sites?

  2. Before analyzing, removing variants based on MAF is make sence? For example, one sites include A(56%), T(44.5%), C(0.5%). In this case, could I remove variant C(replace missing site(NA)) before calculation of Fst?

Best regards,

  • $\begingroup$ Please copy-paste tables of text as text into your question posts. This makes them easier to read on some devices, and makes it more easy to modify the tables to improve the question. $\endgroup$
    – gringer
    Commented Mar 25 at 19:32
  • $\begingroup$ Thank you for your advice. $\endgroup$
    – uri
    Commented Mar 26 at 3:36

1 Answer 1

  1. Hmmmm .... I personally would not use a single VCF to calculate Fst in the way you are describing. It will also miss one of the nucleotides. I would aggregate across VCFs for a homologous position and use those frequencies. Its designed for heterozygotes, you are describing a haploid genome, like mtDNA (I think). Fst needs a genuine allelic frequency, whereas you've shoehorned it (possibly).
  2. Bi-allelic because that is a synonymous variation for most amino acids, i.e. without triggering an amino acid change.
  3. In terms of removing a variant ... unless you believe this represents the in vitro error rates (which you can define from an invariant site) ... then okay. If it is above the in vitro error rate then you can't remove it. I know the read-depth here, so I can't comment.

Just to clarify Fst can be used for prokaryotic genomes (haploids), but unless I've misunderstood the question it seems to being derived from a single VCF ... single Metagenome (?). Thats not how it works. I can describe the what the calculation is measuring will artificially inflate Fst and you can't correct for it.

  • 1
    $\begingroup$ Thank you for your explanation. I could understand. My data is haploid and some studies used fixation index per sites to investigate the differentiated region of interested population. $\endgroup$
    – uri
    Commented Apr 14 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.