I want to create a model to predict proteins which could be associated with drug response in cancer cell lines. I have cell line proteomics data, with compound screening data they have gained for a number of these cells. My aim is to relate these proteins to drug response. I am using this exercise as an introduction in the use of ML in biology.

To this end, I have created dictionaries, with selected compounds as the keys, and their respective datasets. Cells are rows, proteins are columns, and the drug response is the y variable.

I have approached the task as a regression task. The aim being to predicted the ln(IC50) - a measure of the concentration required to induce cell death - and subsequently pull out the features (proteins) which have made the largest contribution to this prediction. As my baseline model, I have attempted linear regression. However, my model performance has consistently been very poor. A problem with this dataset is the large number of missing values (heatmap below - yellow = missing). However, even when trying a simple feature selection strategy with sklearn (first function), the explained variance is arounf 0.3 for the best performing compound. I have plotted the residuals in attempt to dig into this further, but the spread looks quite distributed.

Any advise/suggestions would be appreciated. Or any further reading/papers relating to ML in biology which could help a beginner build and use effective models.

Training and test functions shown below.

    def process_training_and_test_data_for_LG(df):
        training_datasets = {}
        test_datasets = {}
        for drug in df.keys():
            print("at "+drug)
            X = df[drug].drop(["cell_line_name", "ln_IC50","putative_target"], axis=1)
            y = df[drug]["ln_IC50"]
            X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

            feature_selector = SelectKBest(score_func=f_regression, k=50)
            feature_selector.fit(X_train, y_train)
            X_train = feature_selector.transform(X_train)
            X_test = feature_selector.transform(X_test)
            #Add data to relevant dictionaries
            training_datasets[drug] = [X_train, y_train]
            test_datasets[drug] = [X_test, y_test]
        return training_datasets, test_datasets

def train_models(training_data):
    trained_models = {}
    for drug in training_data.keys():
        #initiate model
        lm = LinearRegression()
        #extract drug specific training data
        X_train = training_data[drug][0]
        y_train = training_data[drug][1]
        #fit model
        model = lm.fit(X_train, y_train)
        #Append dict of trained models
        trained_models[drug] = model
        print("Model for {} trained".format(drug))
    return trained_models

def test_models(model_dict, test_data):
    prediction_dict = {}
    for drug in model_dict.keys():
        drug_model = model_dict[drug]
        X_test = test_data[drug][0]
        predictions = drug_model.predict(X_test)
        prediction_dict[drug] = predictions
    return prediction_dict

enter image description here

Residual plot per compound


1 Answer 1


The gist looks fine ..

  • pandas - looks fine
  • SelectKBest - cool
  • transformation (obviously there's a lot of them)
  • standard ML

A common ML situation is, the classifier working but the accuracy is notably lower than expected. My personal first thing to check is the use of linear regression as a baseline model. The scatterplots give a gist, we don't know in particular what data points belong to what classification - thats crucial. What's helpful are coloured dots to assess how they sit against the regression line.

What I think has happened is there's been a PCA via SelectKBest, unsupervised learning, which doesn't account for much of variance (cf 0.3), i.e. this is not an accuracy score.

I've three suggestions:

1/ An acceptable baseline model often used is k-nearest neighbour ..

from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
clf = KNeighborsClassifier(n_neighbors=3)
clf.fit(X_train, y_train)
print("Test set accuracy: {:.2f}".format(clf.score(X_test, y_test)))

I think this will produce much better results with an accuracy of >0.7

2/ However, I would check the following (please see caveat),

for drug in df.keys():
    df ....
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

Just make sure that X_train, etc ... is supposed within the loop, rather than immediately outside it (four spaces to the left). Looking at your outputs they look fine.

The thing to check is its not:

for drug in df.keys():
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

The above loop makes sense to me.

The original loop position train_test_split looks like the key loop might repeatedly over-ride the X_train, X_test, y_train, y_test outputs.

3/ Very final point SelectKBest followed by fit. What about fit_transform instead? If the issue is the features are resulting in low description of the variation, maybe thats key here?

Caveat I do stress that its hard to comment beyond general terms because a good biological understanding is needed of the goal of the ML training. In context its not my area and I haven't read any of the papers.

  • $\begingroup$ Many thanks for this. Apologies there was an error in the indentention in the code I posted, I have amended this in the edit. $\endgroup$
    – LJM
    Sep 7, 2022 at 7:16
  • $\begingroup$ Regarding the the use of KNN, I should of mentioned I have approached this as a regression task, and the y variable is a measure of the concentration required to induce cell death) I'll update this. Though changing this to a classification task could simplify it and allow me to look at feature contribution to classification. For the fit/fit_transform point, I thought that fit_transform did the same function performed the same as .fit() followed by .transform(), is there a subtle difference? $\endgroup$
    – LJM
    Sep 7, 2022 at 7:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.