This answer has two stages,
- Diagnostics, what analytically has gone wrong?
- Where to go from here.
I am pretty confident about point 1. I've been in comparable situations and I know trees. Stage 2, is more open to opinion, however there's no point criticising something (point 1) without pointing out how it can be corrected (point 2).
If your question is within group heterogeneity, excepting for the large group of non-responders associated with D1, thats a different answer.
Low number of DEGs is about your control group? Difficult to comment because the DEG calculation is not declared, nor volcano plots, cut-offs, FDR etc...
You have to keep in mind this data mining and the there are some interesting groups emerging. For first stage analytics looks reasonable (low DEGs dunno).
Result You have 9-10 clusters and those are informative. You can see this because they are forming a checker-board across the heat-map.
Issue The issue is that the deeper branches below the 9-10 clusters are not informative. Do see? The deep branches are likely to be repositioned in different orders in different analyses. Those alternative order might make better biological sense. On the heat-map shown there's a risk of attaching two much important to deep branching orders for a UPMGA tree for a correlation dendrogram.
- Heterogeneity across the matrix is not permitted in UPGMA and causes artefacts - because you are trying to say there's a uniform rate when in most biology that assumption doesn't work. This will usually result in incorrect branching orders.
- You are forcing the data into a bifurcating tree: it might not be at all, could be a large polytomy (star tree).
- Even data sets designed for trees (correlation matrixes are not good data sets for trees) ... branching orders go wrong for loads of reasons.
The second stage analysis would be to take the correlation matrix and apply the neighbour-joining (nj) algorithm to it and see whether the deeper branch align differently. There are also much better tree methods, but that starts getting complicated fast, because there are alternative approaches, such as co-variance. Advanced tree methods (Bayes and ML) will model shifts heterogeneity - however it needs some expertise to get it working.
I'd start assessing the assumptions UPGMA vs neighbour-joining does that take me closer to a biological explanation, then a 9x9 checkboard? Thus would the number of squares in a checkerboard by greatly reduced via nj?
Machine learning Supervised learning (ML), is the trendy way into the data set and in this day an age thats probably the way I'd go. What I'd do is use the 9-10 clusters here as an initial assessment of the training targets. I'd then perform whats called a "feature selection analysis" and assess whether that output makes biological sense. The feature selection should start identifying what parts of your data are causing the shifts between those 9-10 clusters.
