What options should I explore to improve the output of the protein reconstruction?

Question

We developed a neural network-based protein reconstruction tool to reconstruct the main chain from only CA atoms.

1. we generated data from some selected PDBs from the RCSB website to train an NN model.
2. we then use those data to train the NN model.
3. we select some test PDBs, strip all atoms except CA atoms, and save them in files.
4. we pass those CA-only PDBs through the NN model, obtain a reconstructed main chain, and save them in files.
5. we compare original and reconstructed PDB files and calculate CRMSD values.

If we obtain a CRMSD value of 0.3559 and want to improve it what options should we explore?

• Using Keras.
• Not CNN nor RNN. It's a straightforward MLP (4 layers in total). No feedback, no convolution.
• training set is actually large: 1398438 rows and 102 columns in the data file.

Answer 1

The first step is to assess the accuracy, recall and precision of the model then look at confusion matrix. Thereafter consider ROC AUC and if all those are fine ... it's biological. If the model is not 'accurate' where accuracy >0.85 (minimum) its about training the model and that is a different question.

My guess (edit: which from the comments was wrong) is:

• the sample size is insufficient for a presumed deep learning model.
• this will generate problems in the accuracy of the model.

The solution to low sample is called 'data augmentation', a lot of biological knowledge is needed to successfully augment a data set. In this case I suspect the level of biological understanding is strong, but the understanding of deep learning is less advanced.

In summary, low accuracy is a feature of small sample sizes within a deep learning model. To give some idea deep learning needs 500 000 sample size to be reasonable.

---

Update from the comments, the data set is large:

• a sample of 1398438 rows and 102 columns (data points).

This is a good sample for deep learning and low sample size is not the cause.

The model is keras via a 4-layered MLP, so thats fine, but not clear whether its CNN (probably) or RNN.

The likely issue is under-fitting and this is diagnosed via the above statistics for example accuracy and ROC AUC and solution is to enhance the MLP. If there is over-fitting occurring thats a different question.