I have 8 lists of differentially expressed genes for 8 time points (2, 4, 6, 8, 10, 12, 14, 16 hours), the number of genes in each of these lists are not equal.

For example there are 630 common genes between h2 and h4 time points but I have 1143 genes in h2 list and 768 genes in h4 list , so common genes in h2 vs h4 would be 630/1143 and common genes h4 vs h2 would be 630/768.

That would would be great if I have a code to produce a matrix or plot of all possible pairwise combinations of my 8 time points.


3 Answers 3


EDIT: Sorry, I did not see the R tag and wrote the code in bash. I still left the bash example below and added an R version.

Depends on what you want to do, but there are two common ways in which you could store this. Either as wide format:

      h2  h4  h6 ...
Gene1 1   1   0
Gene2 0   1   1
Gene3 1   1   1

Where 1 denotes presence and 0 absence, or as long format:

Gene1 h2
Gene1 h4
Gene2 h4
Gene2 h6
Gene3 h2
Gene3 h4
Gene3 h6

The best format probably depends on what you want to do. Since you did not specify a method/language, I took the liberty to write a few lines of bash to go from the gene lists to the matrices. Bash examples assume your files are named h2, h4 etc and are tab separated with gene names in the first column. R examples assume tables is a list of your data frames and timepoints is a vector c("h2", "h4", ...).

Generate table in wide format:

  1. Concatenate gene names of all tables to a separate file
  2. Extract unique values in this file
  3. Perform outer join between the separate file and each table


all_names <- c()
for (tp in timepoints){
  all_names <- append(all_names, tables[[tp]]$genes)
all_names <- unique(all_names)

wide_table <- matrix(nrow=length(all_names), ncol=(1+length(timepoints)))
wide_table <- as.data.frame(wide_table)
colnames(wide_table) <- append("genes", timepoints)

for (tp in timepoints){
 tmp <- tables[[tp]]
 tmp$tp <- 1
 tmp <- merge(wide_table, tmp, by="genes", all=T)
 wide_table$tp <- tmp$tp


Note: you will might need to add headers to this table

for timefile in $yourfiles;do
  # make a file with all gene names
  awk -v tp=$timefile '{print $1}' >> names_file

sort names_file | uniq > tmp && mv tmp names_file

for timefile in $yourfiles;do
  join -a1 -a2  -j 1 -o 0,1.2,2.2 -e 0 $names_file <(awk '{print$1,1}' $timefile | sort)

Generate table in long format:

  1. For each table, add a column with timepoint
  2. Concatenate all tables


df <- data.frame()
for (tp in timepoints){
  tables[[tp]][[timepoint]] <- tp
  df <- rbind(df, tables[[tp]] )


   for timefile in $yourfiles;do
      awk -v tp=$timefile '{print $1,timefile}' >> long_table

EDIT2: To follow up on this, here is a simple way (using the package dplyr) to count the number of common genes between all possible combination of time points. Assuming you have your data in long format:

df <- data.frame(genes=c("G1","G1","G2","G3","G3"), tp=c("h2","h4","h2","h4","h6"))

  genes tp
1    G1 h2
2    G1 h4
3    G2 h2
4    G3 h4
5    G3 h6

You can get the co-occurrences using :

# Make timepoint ordinal factor
df <- mutate(df, tp = factor(tp, ordered=T))
df <- df %>% 
  # Compute all pairwise co-occurrences
  full_join(df, by="genes") %>% 
  group_by(tp.x, tp.y) %>% 
  summarise(N_genes = length(unique(genes))) %>%
  # Exclude self pairs (gene in h2 and h2)
  filter(tp.x != tp.y) %>% 
  # Check for duplicate pairs in reverse order (h2-h4 and h4-h2)
  mutate(comb=ifelse(tp.x < tp.y, 
                     yes = paste(tp.x, tp.y), 
                     no  = paste(tp.y, tp.x)))
# Remove the duplicate pairs
df <- df[which(!duplicated(df$comb)),]

And visualise it using for example a heatmap: (ofc it looks stupid because I have a tiny sample where not all combinations are even present)

# Visualize number of genes present in each combination
ggplot(df, aes(x=tp.x, y=tp.y, fill=N_genes)) + geom_tile()
  • $\begingroup$ excuse me, i did not quite understand. i have 8 separate lists of genes so how i know about 0 and 1 (absence and presence of a genes among the lists)? $\endgroup$
    – Zizogolu
    Commented Jun 21, 2018 at 17:41
  • $\begingroup$ @FereshTeh 0 and 1 were just placeholder values to give the info "geneY is present (1) / absent (0) at timepoint hX" . You could have expression values, ratios, pvalues or whatever variable instead of 1, and NA instead of 0, for example. Does it make sense ? $\endgroup$
    – cmdoret
    Commented Jun 21, 2018 at 17:45
  • $\begingroup$ thanks a lot, actually i just have 8 lists of genes, lets say that each list has one column of genes. i want a pairwise comparison for common genes for any possible combinations of these time points. $\endgroup$
    – Zizogolu
    Commented Jun 21, 2018 at 18:10
  • $\begingroup$ Oh sorry, i did not understand the last part about pairwise comparisons. First, if you want to filter common genes, you could just filter rows (genes) which have values available for more than x timepoints. For pairwise comparison, maybe that'd be easier if you provided more details/background. (E.g. do you want to compute a single summary stat per condition and compute it, or is it per gene ?) $\endgroup$
    – cmdoret
    Commented Jun 21, 2018 at 18:28
  • $\begingroup$ thanks a lot, i think per condition. actually i need common genes. for instance i have 630 common genes between h2 and h4 time points, while i have 1143 genes in h2 and 768 genes in h4 itself. so, i have two ways comparison; 630/1143 and 630/768. that would be great if i could do the same for all of my time points. something like having a matrix of percentage or a graph of them. $\endgroup$
    – Zizogolu
    Commented Jun 21, 2018 at 20:36

An UpSet Plot is a useful way of visualising these kind of multi-dimensional overlaps. There's an R package for producing them, with a reasonable set of examples.

The input is a binary matrix indicating set membership:

         h2    h4    h6...
GeneA     0     1     0
GeneB     1     1     0
GeneC     0     1     1

And the output is a pair of bar charts, showing the size of various sets, with a dot matrix showing the set membership. An example from the UpSetR documentation:

mutations <- read.csv(system.file("extdata", "mutations.csv", 
                      package = "UpSetR"), header=T, sep = ",")
upset(mutations, sets = c("PTEN", "TP53", "EGFR", "PIK3R1", "RB1"),
      sets.bar.color = "#56B4E9", order.by = "freq", 
      empty.intersections = "on")

Example UpSet Plot


I would simply do this...

# sepcify your timepoints, each timepoint is a list of genes!!
timepoints <- list(h2,h4,h6,h8, h10,h12,h14,h16)
# initialize an empty array that you will fill with your timepoints
vect = vector()
# go over all combinations and fill the array with the set intersection values you want (I guess...)
for (i in timepoints) {
  for (j in timepoints) {
    vect <- c(vect, length(intersect(i,j))/length(i))}}
# reshape your array to a more ordered matrix  
your_matrix <- matrix(vect, nrow=length(timepoints))
# assign columns and rows names
colnames(your_matrix) <- c("h2","h4","h6","h8","h10","h12","h14","h16")
rownames(your_matrix) <- c("h2","h4","h6","h8","h10","h12","h14","h16")

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.