We came across a project in our lab that no one exactly knows how to approach. Since, I know a little bit of Python programming, this project was assigned to me.

There is a data from a randomised controlled clinical trial with 60 participants. Half (30) are control and the other half are actual patients (case). From each group, biopsy samples were taken before (pre) and after (post) treatment with drug A that targets cell type B in the tissue.

In each biopsy sample, the B cells were detected and classified either as “normal” or “malignant”.

Associated metadata for each biopsy are displayed here (I have only included 12 records for this file):

name    patient_id  arm treatment
111     0       control     pre
112     1       control     pre
113     2       control     pre
121     0       control     post
122     1       control     post
123     2       control     post
211     75      case        pre
212     76      case        pre
213     77      case        pre
221     75      case        post
222     76      case        post
223     77      case        post

● name: spreadsheet file name

● patient_id: patient identity number

● arm: trial arm (‘case’ or ‘control)

● treatment: treatment condition (‘pre’ or ‘post’ treatment)

Other files (I have only included 20 records for each file: 10 normal and 10 malignant) contain cell detection results from a single biopsy, and each row in a spreadsheet represents an individual detected cell:

● x: x coordinate

● y: y coordinate

● label: classified label (‘normal’ or ‘malignant’)

Just showing an example, file 111 looks like this:

x   y   label
730 724 normal
1962 450 normal
1511 817 normal
1244 455 normal
2529 397 normal
1878 262 normal
2248 369 normal
2007 273 normal
1531 878 normal
1729 834 normal
931  1270 malignant
1282 314 malignant
1630 839 malignant
1543 460 malignant
2493 237 malignant
1311 744 malignant
1999 366 malignant
737 1361 malignant
2252 448 malignant
2620 398 malignant

The rest can be found here, but probably they will not be necessary: https://www.mediafire.com/file/ka7r59kf0swnbnd/OtherFiles.rar/file

I am trying to answer 3 questions here (if you can think of other questions, please let me know):

  1. identifying post-treatment morphological changes due to the effect of drug A.
  2. proposing measures to quantify the changes.
  3. Using appropriate statistical analysis tools to provide insight whether the changes are due to chance or not.
  • 1
    $\begingroup$ By morphological changes, do you mean the spatial arrangement of cells, or bulk properties of the sample? Since you have coordinates, you could look into "cellular neighborhoods", basically looking at each cell and a small number of its physically nearest neighbors. This could help show if cells are clumping together by type or if they're evenly spread out. $\endgroup$ Mar 30 at 14:59
  • $\begingroup$ @Nuclear Hoagie I was initially going for spatial arrangment, but at this point i am just trying to come up with the right questions first. Because, i would like to know how to aaproach this problem. I think i should consider both of them. $\endgroup$
    – eh329
    Mar 30 at 15:40

1 Answer 1


Yes it's easy you load everything into a dataframe, so each variable is one column and perform supervised machine learning. That will solve all questions.

Hypothesis 1 The target is 'pre' 'post' if there is only one drug in the trial. If there is more than one drug in the trial the drugs would become the target, e.g. if you wanted to know the efficacy between 'drug A' and 'drug B'. What you want to know is the effective of the drug. This would help hypothesis 2.

Hypothesis 2 Estimated cell size is the training target. This is more complicated because I'm not sure what x and y mean: but if x and y are cell measurements then an estimate of cell surface area should be made. If x and y can become a single variate of cell surface area than that can form categorical data ... super size, big, medium large, medium, medium-regular, regular etc ... This could then become the training target and probably what you are looking for.

The ML feature selection will assess cell size (if 'pre' and 'post' is the training target; or), or 'pre'-'post' (if 'cell size estimation' is the training target). In either case the expected result is it should provide the strongest weight in the analysis.

Finally Finally and importantly, the more variables you can input the better, e.g. the gender of the patient, age of the patient as categorical data (bins), smoking, drinking ... anything you can think of, the more the better. ML should ignore anything that has not impact on 'pre' / 'post' or 'estimated cell size'.

The ML algorithm is more complicated but start with naive Bayes and see how it goes. Beyond this it does start to get complicated however.

To address the comments,

  • Load everything into a dataframe

A dataframe is a basic unit of operation in Python (pandas) or R. Python also uses numpy arrays. Everything, means everything - all the data you have, metadata all files. For the calculation the columns involving patient id and patient number, this might be better excluded. I'm not sure - it's complicated because you'll have duplication of records - I would do the calculations with and without this data. You can delete patient id (it's the same as patient number - from a calculation point of view). The x, y is a special case.

Thus all the various tables you describe get loaded into one dataframe, this includes all the patient data such as gender, weight, age, social factors ... it's just like one giant excel table, with rows and columns. The columns are labelled according to the variable e.g. Drug therapy = 'pre' and 'post' data. The dataframe will feed directly to the ML algorithm - a good reason for using it.

x and y

Personally I don't think splitting this data into two columns is a good idea. It will confuse the algorithms - its needs to be one column of data - a single variate. To achieve this will be doable. I would use π to calculate the cell surface area. How that is done for an elongated circle - I don't know - but someone will have done it somewhere, you just need to look it up. Thus 2πr2 if its perfectly round where r = diameter/2 ... elongation must have a derivation its just finding it.

This will absolutely make a difference in your calculations - what I would do is make it to a categorical variable by plotting it and constructing a histogram - otherwise you are into transformations and thats complicated. Categorical variables will also permit logistic regression ML and that can be very useful, because it places each variable into positive or negative weights.

  • $\begingroup$ Thanks for your reply. I am not sure if i understand the first line of your response. What do you mean by "load everything into a dataframe, so each variable is one column"? Are you talking about the metadata file or the other files? and which varibales are you talking about? There are a few things that i should clarify too: 1 - There is only on drug in the experiment. 2 - x and y are coordinates of the each cell detected in the image of tissues in a 2D image. $\endgroup$
    – eh329
    Mar 30 at 0:19
  • 1
    $\begingroup$ @EHR answered above below the ----- $\endgroup$
    – M__
    Mar 30 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.