1
$\begingroup$

I have the data below. I need to use a clustering method to classify them and into categories of "Heterozygotote, Allele 1, Allele 2 and No Call. The values in RFU1 and RFU2 are used to determine the call variable.

Any values in RFU1 and RFU2 that are negative should automatically be classified as No Call. I want to develop my own validation check to check the PCR program calls.

Data

structure(list(Well = structure(1:96, .Label = c("A01", "A02", 
"A03", "A04", "A05", "A06", "A07", "A08", "A09", "A10", "A11", 
"A12", "B01", "B02", "B03", "B04", "B05", "B06", "B07", "B08", 
"B09", "B10", "B11", "B12", "C01", "C02", "C03", "C04", "C05", 
"C06", "C07", "C08", "C09", "C10", "C11", "C12", "D01", "D02", 
"D03", "D04", "D05", "D06", "D07", "D08", "D09", "D10", "D11", 
"D12", "E01", "E02", "E03", "E04", "E05", "E06", "E07", "E08", 
"E09", "E10", "E11", "E12", "F01", "F02", "F03", "F04", "F05", 
"F06", "F07", "F08", "F09", "F10", "F11", "F12", "G01", "G02", 
"G03", "G04", "G05", "G06", "G07", "G08", "G09", "G10", "G11", 
"G12", "H01", "H02", "H03", "H04", "H05", "H06", "H07", "H08", 
"H09", "H10", "H11", "H12"), class = "factor"), Sample = c(NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, 
NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA), 
    Call = structure(c(4L, 4L, 1L, 1L, 1L, 1L, 3L, 3L, 1L, 1L, 
    3L, 3L, 4L, 4L, 1L, 1L, 1L, 1L, 1L, 1L, 3L, 3L, 1L, 1L, 4L, 
    4L, 1L, 1L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 4L, 4L, 1L, 1L, 
    3L, 3L, 3L, 3L, 1L, 1L, 3L, 3L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, 
    1L, 2L, 2L, 1L, 1L, 3L, 3L, 1L, 1L, 4L, 1L, 1L, 1L, 1L, 1L, 
    1L, 1L, 1L, 1L, 2L, 2L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 
    2L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, 1L, 3L, 3L), .Label = c("Allele 1", 
    "Allele 2", "Heterozygote", "No Call"), class = "factor"), 
    Type = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = "Auto", class = "factor"), 
    RFU1 = c(-0.295502405, 0.964070798, 3381.332182, 3532.769062, 
    3431.836843, 3242.966511, 2104.791167, 2220.008503, 3548.252161, 
    3506.51418, 2290.273178, 2281.587684, -5.64819475, -11.73109864, 
    3784.914039, 3619.00781, 3618.211608, 3248.106466, 3394.650325, 
    3339.870196, 2449.202902, 2426.835174, 3432.153478, 2761.392304, 
    -9.267907504, -7.365704356, 3743.092314, 3787.241702, 2172.027787, 
    2096.845649, 2135.649551, 2149.145547, 2293.757257, 2348.099108, 
    2321.019045, 2022.168867, -17.93532331, -12.59832941, 3805.416768, 
    3498.998138, 2304.597239, 2509.63987, 2181.11547, 2261.011876, 
    3432.453036, 3662.758933, 2371.11049, 3068.827061, 2612.107589, 
    2687.824075, 3179.315918, 3688.525218, 3465.327523, 3405.154043, 
    2535.514915, 2452.200079, 374.435299, 423.6015308, 3742.515563, 
    3578.777925, 2634.955017, 2527.514043, 3817.579252, 3550.999412, 
    -10.72035816, 3294.486334, 3352.40368, 3463.150507, 3472.576514, 
    3741.898759, 3571.369947, 3720.645869, 3739.569593, 3855.583168, 
    418.6837047, 49.47548241, 2171.034284, 2155.314713, 3432.363384, 
    3582.508917, 3425.415274, 3487.203299, 3505.23909, 3413.342096, 
    113.5100691, 128.6414053, 2454.588175, 2323.061591, 3188.705702, 
    3376.950546, 3291.072437, 3181.001961, 3195.013863, 3776.919197, 
    2284.22659, 2277.338631), RFU2 = c(-8.346468029, 235.4058561, 
    637.9218251, 650.3759507, 617.4161748, 604.0792911, 4270.310727, 
    4199.615749, 689.863543, 712.6144338, 4274.287194, 4541.168491, 
    -1.626221758, -2.437395631, 802.0941252, 730.5998997, 686.9037384, 
    625.8245403, 644.3586836, 642.8833044, 4937.691887, 5159.479928, 
    725.4449756, 573.3910899, -4.006398006, 213.2859144, 739.7910786, 
    731.0150586, 4408.81923, 4767.533969, 4302.641493, 4325.913445, 
    4597.47663, 4666.904418, 4800.357526, 4142.535329, -17.23239968, 
    178.5311942, 778.305843, 743.1438168, 4214.507094, 4553.703511, 
    4629.339014, 4459.697405, 661.7299014, 727.1054982, 4553.170272, 
    5482.231486, 4520.517999, 4737.802036, 643.3599887, 726.4314715, 
    696.5968338, 697.6099599, 411.8118071, 409.4943424, 5687.32635, 
    5757.51512, 766.4240193, 779.2403225, 4745.055632, 4582.267792, 
    749.5679421, 675.8747055, -7.254521898, 628.3467565, 631.116767, 
    672.7064514, 687.2642132, 718.1192917, 731.785499, 668.3686048, 
    784.8055727, 791.3155894, 4471.047168, 4501.597841, 4504.670332, 
    4442.621066, 682.0632225, 706.6204595, 680.5242182, 683.9558692, 
    684.2909706, 618.6535251, 5727.684954, 6098.485474, 5099.952926, 
    4779.742057, 571.4303822, 614.9258218, 602.9830491, 651.2847695, 
    591.8833499, 742.2387568, 4443.376841, 4716.792177)), class = "data.frame", row.names = c(NA, 
-96L))

What I have tried so far

library(cluster)
library(factoextra)
library(formattable)

df <- df[,c(1,5,6)]
df$RFU1[df$RFU1 < 0] <- 0
df$RFU2[df$RFU2 < 0] <- 0
df$RFU1 <- formattable(df$RFU1, digits = 2, format = "f")
df$RFU2 <- formattable(df$RFU2, digits = 2, format = "f")
df$Well <- as.numeric(df$Well)


clusters <- kmeans(df, centers = 4)
Kmeans_plot <- fviz_cluster(clusters, data = df)

This is the plot generated enter image description here

Points in the top right 57,58,75,76,85,86 should be in another cluster (classed as allele 2 but placed in cluster 1 (rest of points in this cluster are heterozygote)

Also points 24, 55, 56 should be clustered with cluster 2 as all these points are classed as allele 1

I need an algorithm that can detect optimal cluster center, some PCR runs may only have 3 different call variables instead of 4

Programs output - Software company stated their algorithm isn't sophisticated

enter image description here

$\endgroup$
1
$\begingroup$

I would stay away from using k-means, and instead use a method that doesn't a priori define a number of clusters to detect. It also looks as though your clusters aren't exactly spherical, which is an assumption of k-means. I personally am a fan of dbscan, which is available in the R package of the same name. The other poster recommended t-SNE (available in the Rtsne package) but wasn't exactly clear on how / why to use it. t-SNE is a nonlinear dimension reduction technique used specifically for visualization, so you could cluster your points in n-dimensional PCA space and then plot the results in two-dimensional t-SNE space. I think the other poster was recommending plotting your data in t-SNE space before clustering to get an idea of how many true clusters are available, which is another option you could pursue.

$\endgroup$
1
  • $\begingroup$ Yes, you are right.I will modify the answer $\endgroup$
    – M__
    Feb 28 '20 at 19:51
0
$\begingroup$

What happens when

clusters <- kmeans(df, centers = 3)

Anyway K means is pretty good and hasn't done a bad job of your data, although I don't know what you mean by the "cluster centre". There are methods to establish 'centroids'.

With K-means you need to vary the group size or have a very good a priori why it is 4, rather than any other number.

The analysis to perform is tSNE ... either by itself or else following PCA. This will provide a clear a priori of the number of groups in your analysis. I don't know R, but it is present in scikit-learn in Python.


Your observation that some of points are in the wrong cluster ... increase the number of groups in k-means, or switch to PCA, tSNE or PCA-tSNE. If the points remain in the "wrong groups" in all analyses at different group sizes for k-means, then basically thats the result. K-means is good, but not perfect, and misclassification will happen.

Another route is the ML repertoire of lasso regression, ridge regression, decision trees, random forests etc ... they all do the same basic job but I'd look at K-means, PCA, tSNE first and if ok ... last. Regression style calculations will give different answers.


t-SNE is a dimensionality reduction approach to produce a bivariate plot and the fondness of this approach is purely observational rather than empirical. It works and also works very well following PCA. Its the method used in Seurat. K-means is used in ML, and certainly has a fan base, but it is not to say it will give you a correct answer.

$\endgroup$
2
  • $\begingroup$ T-SNE isn't a clustering algorithm, it's a way of taking lots of PCs from PCA and squishing them down so you can see them all in 2 dimensions. It can help you eyeball clusters which will be based on more than just the top 2 PCs. But it won't literally identify clusters or assign points to clusters. $\endgroup$
    – swbarnes2
    Feb 28 '20 at 21:41
  • $\begingroup$ Oh of course I know clustering statistics very well because they were originally used in phylogeny. The problem with clustering algorithms is their is a lot of them - they give different answers - but k-means has gained respectabilty within ML. However the OP wants "clusters" in a non-statistical sense - groups/discrimination/centroids $\endgroup$
    – M__
    Feb 28 '20 at 21:54
0
$\begingroup$

I don't think clustering algorithms are working here, it looks like you can do this much simpler by looking at whatever determines the X axis in your plot. The samples split cleanly when looking in just one dimension.

$\endgroup$
6
  • $\begingroup$ Hi @swbarnes2 thank you for your opinion. Could you provide the code you used? I have been trying to work this out for a while. $\endgroup$ Feb 28 '20 at 21:49
  • $\begingroup$ Why are you making well numeric? Isn't that going to make it be included in the clustering calculations? $\endgroup$
    – swbarnes2
    Feb 28 '20 at 22:19
  • $\begingroup$ the well variable was originally a factor, and k-means only runs on numeric. I just want to develop an algorithm that can determine how many centroids per dataset. I have multiple PCR outputs and have to run this algorithm to verify if the calls generate from the program's algorithm are accurate. (I have attached a screenshot of what the program generates. If I plot the RFU1 and RFU2 how can I use clustering to classify and return "no call, allele 1, allele 2 or heterozygote" $\endgroup$ Feb 29 '20 at 0:14
  • $\begingroup$ Then you should remove the well from the data columns, not make it a number that gets incorporated into the kmeans clustering. You have 2D data. You can't run PCA, let alone t-SNE on 2D data. You don't need sophisticated algorithms to handle 2D data. $\endgroup$
    – swbarnes2
    Feb 29 '20 at 4:24
  • $\begingroup$ I used the elbow method to find the optimal cluster numbers and ran k-means following this. Every time I run the k-means I get different clusters. How can I run that I get the same cluster pattern and cluster numbers each time? $\endgroup$ Mar 3 '20 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.