1
$\begingroup$

Some general questions...

So it appears with most classification learners one must come up with a series of quantifiable variables to associate with each observation. This might include mean and std deviation intensity values, texture, and zernike polynomials. Does anyone have any other suggestions as to how one might quantify 3-dimensional fluorescence pixel data features?

Once appropriate quantification's have been acquired for a given class one can then train a classifier. My next question is - does one then have to segment test images and create a matrix of the same feature variables from that segmented image to use the classifier? Or can one use the raw image itself?

What supervised machine learning techniques will evaluate the raw image data itself - either for training or testing? Can manually annotated data sets or clustered data sets provide a direct model for training data? That is to say, are there supervised machine learning algorithms which accept pure pixel data as models for training?

Are there reliable pre-trained classifiers that are already implemented for these type of fluorescence data sets?

Any resources or advice is greatly appreciated.

Thanks in advance for any input!

$\endgroup$
2
$\begingroup$

Whether to quantify things at all with depend on the technique you want to use. An end-to-end CNN (convolutional neural network) for example would just be fed the 3D image for training and predictions. However the major downside to CNNs is that you need a LOT of data to train them, though perhaps you can get lucky and find an existing CNN that you can use and just retrain the last couple layers.

Other than that, you're going to need to come up with as many (hopefully useful) metrics as you can. There are then a lot of techniques one could try, ranging from simple things like regression to random forests to SVMs, among many others. In general I think simple regression and random forests are the easiest to get started with.

$\endgroup$
2
  • $\begingroup$ Excellent. Thank you! Now If i understand correctly, one feeds the image in 1 dimension essentially? That is to say the total number of elements reshaped into one dimension? From what I've seen the layers are all 1 dimensional unless the images have color layers $\endgroup$
    – thietpas
    Jul 20 '18 at 12:10
  • $\begingroup$ Typically yes, though in reality it's a contiguous stretch of memory already, so you're just ignoring the dimensional aspects when running the CNN. $\endgroup$
    – Devon Ryan
    Jul 20 '18 at 13:13
2
$\begingroup$

Adding to Devon's correct answer: Depending on the amount of images you got, there is also the possibility to avoid the feature engineering by building a two level architecture:

  • The first level identifies the regions of interest in your image and scales these regions to the same size.
  • In the second level, the recognized areas are fed to a (multilayer) neural network/deep neural network to be classified.

By using this approach, the number of needed datapoints is lower compared to a plain CNN method.

The drawback is that identifying the regions of interest itself might be a difficult task as you have either to label these regions manually to apply a machine learning approach at this first level, or you have to "engineer" criteria which make the extraction possible.

I have no idea how these images look like, but maybe you are able to extract the regions using OpenCV.

$\endgroup$
3
  • $\begingroup$ Very interesting. I will check this out also. I have been working with some clustering to isolate some examples of objects of interest. I haven't perfected this process yet as the fluorescence data is somewhat noisy. The same question for you as Devon, is it possible to send the image in as a 3d stack array? Or will the image necessarily be sent in on layer of the stack at a time? My interpretation thus far is the latter $\endgroup$
    – thietpas
    Jul 20 '18 at 12:13
  • $\begingroup$ At least to OpenCV and a CNN you can present the images one at a time in a 3D format, i.e.: width x height x color. That means you total dataset has to have the format number_of_images x width x height x color $\endgroup$
    – phngs
    Jul 20 '18 at 12:21
  • $\begingroup$ sounds good thank you the assistance and advice! $\endgroup$
    – thietpas
    Jul 20 '18 at 13:45
0
$\begingroup$

What supervised machine learning techniques will evaluate the raw image data itself - either for training or testing?

  1. You can use convolutional neural networks (CNN). They will directly use the raw image data itself.

  2. After applying CNNs, you can also use recurrent neural layers (RNN), to achieve better results (e.g. gated recurrent layers).

Hope this helps. As a reference you have this article:

Gudla PR1,2, Nakayama K2,3, Pegoraro G1,2, Misteli T2. Article: SpotLearn: Convolutional Neural Network for Detection of Fluorescence In Situ Hybridization (FISH) Signals in High-Throughput Imaging Approaches. Send to Cold Spring Harb Symp Quant Biol. 2017;82:57-70. doi: 10.1101/sqb.2017.82.033761. Epub 2017 Nov 28.

$\endgroup$
4
  • $\begingroup$ Please include some links to resources on CNN and RNN $\endgroup$
    – Bioathlete
    Jan 17 '19 at 13:59
  • $\begingroup$ Gudla PR1,2, Nakayama K2,3, Pegoraro G1,2, Misteli T2. SpotLearn: Convolutional Neural Network for Detection of Fluorescence In Situ Hybridization (FISH) Signals in High-Throughput Imaging Approaches. Send to Cold Spring Harb Symp Quant Biol. 2017;82:57-70. doi: 10.1101/sqb.2017.82.033761. Epub 2017 Nov 28. Abstract and full text link can be found here: ncbi.nlm.nih.gov/pubmed/?term=3D+CNN+fluorescence $\endgroup$ Jan 17 '19 at 14:39
  • 1
    $\begingroup$ Please edit your answer to include those links $\endgroup$
    – Bioathlete
    Jan 17 '19 at 15:02
  • $\begingroup$ Authors: Gudla PR1,2, Nakayama K2,3, Pegoraro G1,2, Misteli T2. Article: SpotLearn: Convolutional Neural Network for Detection of Fluorescence In Situ Hybridization (FISH) Signals in High-Throughput Imaging Approaches. Send to Cold Spring Harb Symp Quant Biol. 2017;82:57-70. doi: 10.1101/sqb.2017.82.033761. Epub 2017 Nov 28. you can find this article in ncbi.nlm.nih.gov/pubmed A detailled link is here: Abstract and full text link can be found here: ncbi.nlm.nih.gov/pubmed/?term=3D+CNN+fluorescence $\endgroup$ Jan 17 '19 at 15:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.