2
$\begingroup$

I am using a few clustering algorithms as well as my own tool for protein sequences. Also, I have benchmarked clustering results (i.e. ground truth) to compare with the results. Is there any tool/software/script to generate evaluation metrics for the clustering results.

The clustering results/benchmarked are in the following format

1  Name1
1  Name2
2  Name3
2  Name4
2  Name5
.....

UPDATE: I am evaluating the clustering performance using scikit

$\endgroup$
  • $\begingroup$ from your other comment you have already found scikit, if your number of clusters/ground truth labels is reasonably low (say <50), take a look at the contingency matrix as well, it can give a very clear view of what is going on in your clustering scikit-learn.org/stable/modules/… $\endgroup$ – Pallie Mar 12 at 10:43
  • $\begingroup$ Thanks, @Pallie !! Yes, scikit looks good as some clustering papers used the same measures as well. The number of labels for my work is reasonably high (thousands and millions). $\endgroup$ – SBDK8219 Mar 12 at 17:54
1
$\begingroup$

The definitive answer is no, that is why they are not so popular. There are metrics that can be used:

  • you can examine "minimum distance" as a criterion under the assumption that the lowest distance between taxa represents the largest homology (e.g. if it is sequence data), which would be translate to other biological scenarios.
  • resampling, e.g. bootstrapping, jack-knifing and examine the tree structure of each node against the biological plausibility of the answer being correct.

For both points a biological a priori, i.e. biological plausibility of assessing the tree structure is required to confirm/reject the clustering method.

Clustering algorithms are not like the probabilistic methods, where the highest likelihood is the singular criteria and of course they do give different answers. Its a brief explanation I know but hope it makes sense.

| improve this answer | |
$\endgroup$
1
$\begingroup$

I am evaluating the clustering performance using scikit

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Okay, but thats a different question. You appear to be implying you have input output associations for machine learning, this information was not present in the question. $\endgroup$ – Michael Mar 12 at 18:09
  • 1
    $\begingroup$ I have mentioned that I have both output and ground truth. I am sorry for the confusion. $\endgroup$ – SBDK8219 Mar 12 at 20:38
  • 1
    $\begingroup$ No problem, its just ML is a different approach for verification involving e.g. ROC and accurary indices. If you can do ML, then its really good. $\endgroup$ – Michael Mar 13 at 6:32
  • $\begingroup$ Thanks, @Michael !! I think I have to use ROC for further analysis. There are some drawbacks using scikit for my problem $\endgroup$ – SBDK8219 Mar 16 at 19:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.