# Interpretation of hierarchical clustering with bootstrapping

I have data that includes 'cases' and 'controls' and have carried out hierarchical clustering.

I used the pvclust package in R to bootstrap the results and significant branches are highlighted with red rectangles (based on au>0.95):

What is clear is that no clustering occurs that separates 'cases' and 'controls', and this is in fact what we expected and want to show. We want to show that the measured variable does not distinguish between cases and controls.

1. List itemApart from saying visually no clear clusters emerge that distinguishes between cases and controls are there any objective measures that can be used to say no significant clustering occurs between two groups?
2. One observation I have here is that the AU values and the BP values are very different, even though both p-values should be interpreted in a similar fashion, am I missing something?
3. Perhaps pvclust (bootstrapping) is not the right option here, is there a better way of showing quantifiably that no significant clustering occurs between two groups? Perhaps some kind of supervised clustering (I am not sure what this even means in this context)?
• I am unfamiliar with this package but some red flags: 1) you use correlation as a distance measure, it makes more sense as a similarity measure, 2) I don't see a cutoff in the tree to determine what your actual clustering is. This might give a view where clusters do emerge with distinct case/control patterns. edit: I see the red brackets on the bottom now, are these your final cluster labels? – Pallie Apr 15 '19 at 11:56
• Can you explain more about your data please? Is this gene expression? Or something else? – benn Apr 15 '19 at 12:23
• @Pallie, I have cut off a section of the picture which has patient ID's in it so there is a blank white space which obscures the red boxes sorry. If you look at the red boxes on the dendrogram they highlight branches with AU values >95 which I interpret as meaning these are the statistically significant clusters. Benn, this I not gene expression, it is a measure of antibodies to about 100 bacterial strains in cases vs controls. Any advice on a better measure to use perhaps to show the measure does not cluster patients and controls? Thank you both – reubenmcg Apr 15 '19 at 22:18
• I'm not sure doing clustering and then trying to prove that the clustering is significant is the right approach, it is very heavily dependent on the clustering algorithm you used. Can't you just do an ANOVA between case/control to prove there is a/no significant difference? – Pallie Apr 16 '19 at 8:44
• ANOVA requires normalised data and its unlikely to be Gaussian (it sounds a bit odd saying "unlikely to be normal). – Michael Apr 16 '19 at 19:13

Good question. You have performed the first step via unsupervised learning and AU bootstraps give great results, orthodox bootstraps give no significant (or very few) clusters. The contrast between these approaches is unusual because they should be somewhere close to each other. If you can resolve that, your analysis is fine. So at present IMO the unsupervised learning needs further investigation. ... deleted

... antibodies ... oh I see. Trees, like you've done, don't work here because of the cross-reactivity creates network phylogenies. This is the antonym of bifurcating trees. If someone has a robust bifurcating phylogeny with antibody data and gets good bootstraps .. utterly amazing. However, if you subject your data to a network analysis you'll find connections all over the place - in practical terms this really messes up bootstraps. Thats the likely reason for you results. There is no point doing a complex bifurcating tree analysis because the antibody cross-reactivity will mess it up. k-means has been implemented as unsupervised learning in these instances and would provide the starting point in using clustering to define a supervised learning problem.

• could you briefly explain how K-means clustering would work here? would I simple choose k=2 (cases vs controls) and show that cases and controls do not cluster into 2 groups? Could I still bootstrap the result to give significance? – reubenmcg Apr 17 '19 at 1:15
• No not in this instance ... with "traditional" k-means is you have to set the groups a priori 2,3,4,5 etc..... and the trick is defining which a priori grouping provides the "best fit" (I forget the criterion). The tree will provide some indication of the max group size. K-means will then assign all you data into the designated number of clusters as best it can. K-means is respectable - albeit it doesn't sound cool. I think its because it minimises false positives and robust. The study I was involved with (malaria) used k-means as the singular proof :-O – Michael Apr 17 '19 at 12:47
• In this study, the network phylogeny defined the groupings (it couldn't assign significance to it) and these groupings were explored via K-means. Trees (network trees) were there just because I like them, all the tree was saying that was statically significant was "these things cross-react" (hardly revolutionary). Otherwise it is a bit hard to describe k-means within a comments section, but let me know if you get the gist. – Michael Apr 17 '19 at 15:22
• I am doing supervised learning on quite similar data to yours at present but for viruses. However, this is a bit too close my current work to be providing detailed technical advice. – Michael Apr 17 '19 at 15:22

I've got it. The question you are asking is homogeneity between you target group and your general population of patients. What you want to show is "complete" homogeneity between the two groups regardless of any sub-sampling, ie. if there is groups within groups that might break your hypothesis. K-means is perfect here. The issue with K-means is designating the number of groups, but here thats irrelevant because its not the question you are asking. You are asking homogeneity and if your hypothesis is true no matter how many groups you designate for K-means, e.g. 2,3,4,5,6,7,8,9,10 (or maybe a more reduced sampling) homogeneity between target (treatment?) and general population is observed. This would therefore be a definitive analysing ASSUMING that you don't encounter a group where there is exclusively one population and the other is absent. Obviously the number of K-mean groups can't exceed the number of treated/target patients.

It would be a really good application of k-means because it avoids the question, So why was that a priori selected?

Personally I would first present your preferred a priori group size and then say same result for all group sizes. Its simple, recognised and neat and easy for a reviewer to follow. There are ML solutions to K-means, but I think "keep it simple".

• This makes allot of sense, and just knowing to google homogeneity has led me down what I think is the correct path, I am now struggling to find R packages that let me do this with explanations on how to sue them. I think clValid will work – reubenmcg Apr 18 '19 at 2:06
• There is a direct function called 'kmeans' in R, but its not an approach I'd use unless I was reiterating within a data pipeline (R is great for automation). For this a general stats package SPSS or Stata is easier. ML k-means are in Sci-kit learn (Python), but this is really complicated and lots of things can go wrong. Glad this helped. – Michael Apr 18 '19 at 16:54