I am currently investigating how to predict protein structure and contact map predictions. There, I see things like top L/5 and top L/2 scores as a way to evaluate the contact map. What formulas do you use to evaluate these? Also, what is a meaningful evaluation method?

  • $\begingroup$ L is protein length so you divide that by two or five! Sorry, I could not resist that. Anyway, there are a few metrics to rank the contact, so depends on the paper. The original EVFold paper is a good place to start journals.plos.org/plosone/article?id=10.1371/… even if you are interested in the AlphaFold stuff. $\endgroup$ – Matteo Ferla Nov 18 '20 at 12:25
  • $\begingroup$ @MatteoFerla Thanks for answering my question, I understand that L/5 is the length of the protein sequence divided by 5. Does that mean that Top L/5 represents the best prediction for all patches of L/5? $\endgroup$ – BoKuToTsuZenU Nov 21 '20 at 10:22
  • $\begingroup$ @MatteoFerla Also, what does Short, Medium and Long mean in the accuracy column of the forecast? This is the paper: academic.oup.com/bioinformatics/article/35/22/4647/5487385 $\endgroup$ – BoKuToTsuZenU Nov 21 '20 at 10:23
  • $\begingroup$ That sounds like the range cutoffs —A range contact is not the distance of atoms, but the distance in residues in the linear sequence —in that paper <12 is short, >23 long. Honestly, I've not read that paper —the Zhang group have made I-Tasser so are a good group. But this is paper dependent, in the AlphaFold one they divide the range in 64 bins, but still refer to short-range/medium-range/long-range. $\endgroup$ – Matteo Ferla Nov 21 '20 at 12:10

A contact map predictor will usually output the probability of a contact for every residue pair in a protein, often ommitting very close pairs (e.g. less than 6 residue separation).

To calculate the L/n score, rank the predicted contacts descending by probability and look at the first L/n contacts. The fraction of these that are actually contacts in the native structure is the L/n contact precision score.

Often, you would filter the list to look at only contacts of a certain sequence separation, e.g. long range contacts between residues of >23 separation (definitions vary between papers). If you have a good predictor this score is high for L/10 and falls as you go to L/1, which is expected as proteins don't usually have that many long range contacts to find.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.