# How to represent a summary of disagreement between two secondary structure predictors given predictions for many proteins?

I have 6000 predicted protein structures.

I am comparing secondary structures predicted by an algorithm with the correct secondary structure assignments.

I need to show which positions in a predicted protein chain have the most differences as to the true protein chain.

How can I do that?

How to represent a summary of disagreement between two secondary structure predictors given predictions for many proteins?

I already prepared confusion matrices. I was looking for something graphical like chart or histogram, etc..

• Are you asking: a. How to represent the disagreement between two secondary structure predictors on one protein or b. How to represent a summary of disagreement between two secondary structure predictors given predictions for many proteins or c. If there is any common way and simple tool for representing a or b? (kind of yes for a, not to my knowledge for b) Mar 6 at 22:40
• @O.Laprevote, I am asking (b). Mar 7 at 5:43

(This comment got too long)

positions in a predicted protein chain

You do mean position in SS sequence, not coordinates in tertiary, yes? From the sounds of it you are talking about SS —not tertiary— and want the analysis to be independent of tertiary.

Therefore, you are comparing for each protein two 1D arrays of sequential elements containing between 3 and 6 states, one for each residue. Technically, you could add another state, a chain break, and have two colossally long 1D arrays. This very much becomes a statistics classification topic.

The properties of these arrays are:

• The values are sequential.
• Neighbouring elements influence each other as a span of residues makes a SS element. Thus a correct classification will propagate. This will mean that all metrics will be hugely inflated.
• Assumption: the divergence is likely on the ends of a span. This could be tested and plotted!

The simplest thing would be collapse the states into a binary loop vs non loop and classify each element of the two arrays and do a typical confusion matrix and its metrics (see Wikipedia). This would have the issue that neighbours influence each other.

For a figure, given N states, one could simply plot a N × N heatmap of the true states vs. predicted states. This would show a hot diagonal of the correct allocations, but will show which states are more prone to misallocation. The frequency of the states may need normalising... but this would still be affected by the length of the SS spans. Therefore, a more sophisticated model may be needed for better normalisation.

For more advanced classifications, more advanced algorithms would be needed, but it still remains a stats problem. It might be worth asking the question again under that discussion framework or look up a relevant multi-state classifier.

• I already prepared confusion matrices. I was looking for something graphical like chart or histogram, etc.. Mar 7 at 9:41
• If so, given N states, wouldn't a simple N_&times;_N heatmap of the true states vs. predicted states suffice? This would show a hot diagonal of the correct allocations, but will show which states are more prone to misallocation. Mar 7 at 9:50