# Multiway intersection of 8 lists of genes

I have 8 lists of differentially expressed genes for 8 time points (2, 4, 6, 8, 10, 12, 14, 16 hours), the number of genes in each of these lists are not equal.

For example there are 630 common genes between h2 and h4 time points but I have 1143 genes in h2 list and 768 genes in h4 list , so common genes in h2 vs h4 would be 630/1143 and common genes h4 vs h2 would be 630/768.

That would would be great if I have a code to produce a matrix or plot of all possible pairwise combinations of my 8 time points.

EDIT: Sorry, I did not see the R tag and wrote the code in bash. I still left the bash example below and added an R version.

Depends on what you want to do, but there are two common ways in which you could store this. Either as wide format:

      h2  h4  h6 ...
Gene1 1   1   0
Gene2 0   1   1
Gene3 1   1   1
...


Where 1 denotes presence and 0 absence, or as long format:

      timepoint
Gene1 h2
Gene1 h4
Gene2 h4
Gene2 h6
Gene3 h2
Gene3 h4
Gene3 h6
...


The best format probably depends on what you want to do. Since you did not specify a method/language, I took the liberty to write a few lines of bash to go from the gene lists to the matrices. Bash examples assume your files are named h2, h4 etc and are tab separated with gene names in the first column. R examples assume tables is a list of your data frames and timepoints is a vector c("h2", "h4", ...).

Generate table in wide format:

1. Concatenate gene names of all tables to a separate file
2. Extract unique values in this file
3. Perform outer join between the separate file and each table

R:

all_names <- c()
for (tp in timepoints){
all_names <- append(all_names, tables[[tp]]$genes) } all_names <- unique(all_names) wide_table <- matrix(nrow=length(all_names), ncol=(1+length(timepoints))) wide_table <- as.data.frame(wide_table) colnames(wide_table) <- append("genes", timepoints) for (tp in timepoints){ tmp <- tables[[tp]] tmp$tp <- 1
tmp <- merge(wide_table, tmp, by="genes", all=T)
wide_table$tp <- tmp$tp
}


Bash:

for timefile in $yourfiles;do # make a file with all gene names awk -v tp=$timefile '{print $1}' >> names_file done sort names_file | uniq > tmp && mv tmp names_file for timefile in$yourfiles;do
join -a1 -a2  -j 1 -o 0,1.2,2.2 -e 0 $names_file <(awk '{print$1,1}' $timefile | sort) done  Generate table in long format: 1. For each table, add a column with timepoint 2. Concatenate all tables R: df <- data.frame() for (tp in timepoints){ tables[[tp]][[timepoint]] <- tp df <- rbind(df, tables[[tp]] ) }  Bash:  for timefile in$yourfiles;do
awk -v tp=$timefile '{print$1,timefile}' >> long_table
done


EDIT2: To follow up on this, here is a simple way (using the package dplyr) to count the number of common genes between all possible combination of time points. Assuming you have your data in long format:

df <- data.frame(genes=c("G1","G1","G2","G3","G3"), tp=c("h2","h4","h2","h4","h6"))

genes tp
1    G1 h2
2    G1 h4
3    G2 h2
4    G3 h4
5    G3 h6


You can get the co-occurrences using :

library(dplyr)
# Make timepoint ordinal factor
df <- mutate(df, tp = factor(tp, ordered=T))
df <- df %>%
# Compute all pairwise co-occurrences
full_join(df, by="genes") %>%
group_by(tp.x, tp.y) %>%
summarise(N_genes = length(unique(genes))) %>%
# Exclude self pairs (gene in h2 and h2)
filter(tp.x != tp.y) %>%
# Check for duplicate pairs in reverse order (h2-h4 and h4-h2)
mutate(comb=ifelse(tp.x < tp.y,
yes = paste(tp.x, tp.y),
no  = paste(tp.y, tp.x)))
# Remove the duplicate pairs
df <- df[which(!duplicated(df\$comb)),]


And visualise it using for example a heatmap: (ofc it looks stupid because I have a tiny sample where not all combinations are even present)

# Visualize number of genes present in each combination
library(ggplot2)
ggplot(df, aes(x=tp.x, y=tp.y, fill=N_genes)) + geom_tile()

• excuse me, i did not quite understand. i have 8 separate lists of genes so how i know about 0 and 1 (absence and presence of a genes among the lists)? Jun 21, 2018 at 17:41
• @FereshTeh 0 and 1 were just placeholder values to give the info "geneY is present (1) / absent (0) at timepoint hX" . You could have expression values, ratios, pvalues or whatever variable instead of 1, and NA instead of 0, for example. Does it make sense ? Jun 21, 2018 at 17:45
• thanks a lot, actually i just have 8 lists of genes, lets say that each list has one column of genes. i want a pairwise comparison for common genes for any possible combinations of these time points. Jun 21, 2018 at 18:10
• Oh sorry, i did not understand the last part about pairwise comparisons. First, if you want to filter common genes, you could just filter rows (genes) which have values available for more than x timepoints. For pairwise comparison, maybe that'd be easier if you provided more details/background. (E.g. do you want to compute a single summary stat per condition and compute it, or is it per gene ?) Jun 21, 2018 at 18:28
• thanks a lot, i think per condition. actually i need common genes. for instance i have 630 common genes between h2 and h4 time points, while i have 1143 genes in h2 and 768 genes in h4 itself. so, i have two ways comparison; 630/1143 and 630/768. that would be great if i could do the same for all of my time points. something like having a matrix of percentage or a graph of them. Jun 21, 2018 at 20:36

An UpSet Plot is a useful way of visualising these kind of multi-dimensional overlaps. There's an R package for producing them, with a reasonable set of examples.

The input is a binary matrix indicating set membership:

         h2    h4    h6...
GeneA     0     1     0
GeneB     1     1     0
GeneC     0     1     1
...


And the output is a pair of bar charts, showing the size of various sets, with a dot matrix showing the set membership. An example from the UpSetR documentation:

library(UpSetR)
package = "UpSetR"), header=T, sep = ",")
upset(mutations, sets = c("PTEN", "TP53", "EGFR", "PIK3R1", "RB1"),
sets.bar.color = "#56B4E9", order.by = "freq",
empty.intersections = "on")


I would simply do this...

# sepcify your timepoints, each timepoint is a list of genes!!
timepoints <- list(h2,h4,h6,h8, h10,h12,h14,h16)
# initialize an empty array that you will fill with your timepoints
vect = vector()
# go over all combinations and fill the array with the set intersection values you want (I guess...)
for (i in timepoints) {
for (j in timepoints) {
vect <- c(vect, length(intersect(i,j))/length(i))}}
# reshape your array to a more ordered matrix
your_matrix <- matrix(vect, nrow=length(timepoints))
# assign columns and rows names
colnames(your_matrix) <- c("h2","h4","h6","h8","h10","h12","h14","h16")
rownames(your_matrix) <- c("h2","h4","h6","h8","h10","h12","h14","h16")