Advise on building an effect ML model for predicting important proteins for drug response

Question

I want to create a model to predict proteins which could be associated with drug response in cancer cell lines. I have cell line proteomics data, with compound screening data they have gained for a number of these cells. My aim is to relate these proteins to drug response. I am using this exercise as an introduction in the use of ML in biology.

To this end, I have created dictionaries, with selected compounds as the keys, and their respective datasets. Cells are rows, proteins are columns, and the drug response is the y variable.

I have approached the task as a regression task. The aim being to predict the ln(IC50) - a measure of the concentration required to induce cell death - and subsequently pull out the features (proteins) which have made the largest contribution to this prediction. As my baseline model, I have attempted linear regression. However, my model performance has consistently been very poor. A problem with this dataset is the large number of missing values (heatmap below - yellow = missing). However, even when trying a simple feature selection strategy with sklearn (first function), the explained variance is around 0.3 for the best performing compound. I have plotted the residuals in attempt to dig into this further, but the spread looks quite distributed.

Any advise/suggestions would be appreciated. Or any further reading/papers relating to ML in biology which could help a beginner build and use effective models.

Training and test functions shown below.

    def process_training_and_test_data_for_LG(df):
        training_datasets = {}
        test_datasets = {}
        for drug in df.keys():
            df[drug].dropna(axis=1,inplace=True)
            print("at "+drug)
            X = df[drug].drop(["cell_line_name", "ln_IC50","putative_target"], axis=1)
            y = df[drug]["ln_IC50"]
            X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

            feature_selector = SelectKBest(score_func=f_regression, k=50)
            feature_selector.fit(X_train, y_train)
        
            X_train = feature_selector.transform(X_train)
            X_test = feature_selector.transform(X_test)
        
            #Add data to relevant dictionaries
            training_datasets[drug] = [X_train, y_train]
            test_datasets[drug] = [X_test, y_test]
        return training_datasets, test_datasets

def train_models(training_data):
    trained_models = {}
    for drug in training_data.keys():
        #initiate model
        lm = LinearRegression()
        #extract drug specific training data
        X_train = training_data[drug][0]
        y_train = training_data[drug][1]
        #fit model
        model = lm.fit(X_train, y_train)
        #Append dict of trained models
        trained_models[drug] = model
        print("Model for {} trained".format(drug))
    return trained_models

def test_models(model_dict, test_data):
    prediction_dict = {}
    for drug in model_dict.keys():
        drug_model = model_dict[drug]
        X_test = test_data[drug][0]
        predictions = drug_model.predict(X_test)
        prediction_dict[drug] = predictions
    return prediction_dict

Answer 1

The missing values might be an issue indeed. You might want to use imputation methods, e.g.

from sklearn.impute import KNNImputer
from sklearn.experimental import enable_iterative_imputer
from sklearn.impute import IterativeImputer

def impute_df(df, feature_columns_to_apply, strategy):
    if strategy == 'mean':
        for col in feature_columns_to_apply:    
            df.loc[:, col] = df.loc[:, col].fillna(df[col].mean())
    elif strategy == 'median':
        for col in feature_columns_to_apply:    
            df.loc[:, col] = df.loc[:, col].fillna(df[col].median())
    elif strategy == 'knn':
        knn_imputer = KNNImputer(n_neighbors=5)
        df_tmp = pd.DataFrame(knn_imputer.fit_transform(df[feature_columns_to_apply]), columns=feature_columns_to_apply)
        df.loc[:, feature_columns_to_apply] = df_tmp[feature_columns_to_apply]
    elif strategy == 'iterative':
        iterative_imputer = IterativeImputer()
        df_tmp = pd.DataFrame(iterative_imputer.fit_transform(df[feature_columns_to_apply]), columns=feature_columns_to_apply)
        df.loc[:, feature_columns_to_apply] = df_tmp[feature_columns_to_apply]
    return df
    
df_all = impute_df(df_all, feature_columns_to_apply, 'mean')

(IterativeImputer is heavy and slow) These are just sample strategies, you might want to use other methods, see e.g. here.
Also consider other models such as random forest / XGBoost, they outperform linear regression in many cases.

Answer 2

The gist looks fine ..

• pandas - looks fine
• SelectKBest - cool
• transformation (obviously there's a lot of them)
• standard ML

A common ML situation is, the classifier working but the accuracy is notably lower than expected. My personal first thing to check is the use of linear regression as a baseline model. The scatterplots give a gist, we don't know in particular what data points belong to what classification - thats crucial. What's helpful are coloured dots to assess how they sit against the regression line.

What I think has happened is there's been a PCA via SelectKBest, unsupervised learning, which doesn't account for much of variance (cf 0.3), i.e. this is not an accuracy score.

I've three suggestions:

1/ An acceptable baseline model often used is k-nearest neighbour ..

from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
clf = KNeighborsClassifier(n_neighbors=3)
clf.fit(X_train, y_train)
print("Test set accuracy: {:.2f}".format(clf.score(X_test, y_test)))

I think this will produce much better results with an accuracy of >0.7

2/ However, I would check the following (please see caveat),

for drug in df.keys():
    df ....
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

Just make sure that X_train, etc ... is supposed within the loop, rather than immediately outside it (four spaces to the left). Looking at your outputs they look fine.

The thing to check is its not:

for drug in df.keys():
    df....
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

The above loop makes sense to me.

The original loop position train_test_split looks like the key loop might repeatedly over-ride the X_train, X_test, y_train, y_test outputs.

3/ Very final point SelectKBest followed by fit. What about fit_transform instead? If the issue is the features are resulting in low description of the variation, maybe thats key here?

---

Caveat I do stress that its hard to comment beyond general terms because a good biological understanding is needed of the goal of the ML training. In context its not my area and I haven't read any of the papers.