I am currently investigating how to predict protein structure and contact map predictions. There, I see things like top L/5 and top L/2 scores as a way to evaluate the contact map. What formulas do you use to evaluate these? Also, what is a meaningful evaluation method?
4
-
$\begingroup$ L is protein length so you divide that by two or five! Sorry, I could not resist that. Anyway, there are a few metrics to rank the contact, so depends on the paper. The original EVFold paper is a good place to start journals.plos.org/plosone/article?id=10.1371/… even if you are interested in the AlphaFold stuff. $\endgroup$– Matteo FerlaCommented Nov 18, 2020 at 12:25
-
$\begingroup$ @MatteoFerla Thanks for answering my question, I understand that L/5 is the length of the protein sequence divided by 5. Does that mean that Top L/5 represents the best prediction for all patches of L/5? $\endgroup$– musakoCommented Nov 21, 2020 at 10:22
-
$\begingroup$ @MatteoFerla Also, what does Short, Medium and Long mean in the accuracy column of the forecast? This is the paper: academic.oup.com/bioinformatics/article/35/22/4647/5487385 $\endgroup$– musakoCommented Nov 21, 2020 at 10:23
-
$\begingroup$ That sounds like the range cutoffs —A range contact is not the distance of atoms, but the distance in residues in the linear sequence —in that paper <12 is short, >23 long. Honestly, I've not read that paper —the Zhang group have made I-Tasser so are a good group. But this is paper dependent, in the AlphaFold one they divide the range in 64 bins, but still refer to short-range/medium-range/long-range. $\endgroup$– Matteo FerlaCommented Nov 21, 2020 at 12:10
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
A contact map predictor will usually output the probability of a contact for every residue pair in a protein, often ommitting very close pairs (e.g. less than 6 residue separation).
To calculate the L/n score, rank the predicted contacts descending by probability and look at the first L/n contacts. The fraction of these that are actually contacts in the native structure is the L/n contact precision score.
Often, you would filter the list to look at only contacts of a certain sequence separation, e.g. long range contacts between residues of >23 separation (definitions vary between papers). If you have a good predictor this score is high for L/10 and falls as you go to L/1, which is expected as proteins don't usually have that many long range contacts to find.
