# Parsing MSigDB Supplementary Collections

MSigDB recently added Supplementary Collections. The formatting is similar to the original MSigDB gene sets, but there is an additional metadata.txt file (link).

It looks like (condensed for easier reading):

STANDARD_NAME   Fan_Embryonic_CTX_Big_Groups_Cajal_Retzius
ORGANISM    Homo sapiens
ORGAN_SYSTEM    Central Nervous System
PMID    29867213
PUBLICATION_TITLE   Spatial transcriptomic survey of human embryonic cerebral cortex by single-cell RNA-seq analysis
AUTHORS Fan X,Dong J,Zhong S,Wei Y,Wu Q,Yan L,Yong J,Sun L,Wang X,Zhao Y,Wang W,Yan J,Wang X,Qiao J,Tang F,
GEOID   GSE103723
...
STANDARD_NAME   Fan_Embryonic_CTX_Big_Groups_Brain_Endothelial
ORGANISM    Homo sapiens
ORGAN_SYSTEM    Central Nervous System
PMID    29867213
PUBLICATION_TITLE   Spatial transcriptomic survey of human embryonic cerebral cortex by single-cell RNA-seq analysis
AUTHORS Fan X,Dong J,Zhong S,Wei Y,Wu Q,Yan L,Yong J,Sun L,Wang X,Zhao Y,Wang W,Yan J,Wang X,Qiao J,Tang F,
GEOID   GSE103723
...
STANDARD_NAME   Fan_Embryonic_CTX_Big_Groups_Excitatory_Neuron
ORGANISM    Homo sapiens
ORGAN_SYSTEM    Central Nervous System
PMID    29867213
PUBLICATION_TITLE   Spatial transcriptomic survey of human embryonic cerebral cortex by single-cell RNA-seq analysis
AUTHORS Fan X,Dong J,Zhong S,Wei Y,Wu Q,Yan L,Yong J,Sun L,Wang X,Zhao Y,Wang W,Yan J,Wang X,Qiao J,Tang F,
GEOID   GSE103723
...


I could not find any description of the formatting. Is this a standard format? Is there a parser for it (preferably in R)? It looks like I can just transpose each record to get it in a table format, but I don't know if I can trust all records to be consistent.

If you want to separate each set, you can do something like this

dat <- readLines("~/Desktop/scsig.v1.0.metadata.txt")
dat <- gsub("\t"," ",dat)

# get the indeces of each type
standardName <- grep("STANDARD_NAME",dat)
organism <- grep("ORGANISM",dat)
organSystem <- grep("ORGAN_SYSTEM",dat)
pmid <- grep("PMID",dat)

# return the values
standardName <- gsub(pattern = "STANDARD_NAME",replacement = "",dat[standardName])
organism <- gsub(pattern = "ORGANISM",replacement = "",dat[organism])
organSystem <- gsub(pattern = "ORGAN_SYSTEM",replacement = "",dat[organSystem])
pmid <- gsub(pattern = "PMID",replacement = "",dat[pmid])

df <- data.frame(standardName,organism,organSystem,pmid,stringsAsFactors = FALSE)

standardName      organism             organSystem      pmid
1      Fan_Embryonic_CTX_Big_Groups_Cajal_Retzius  Homo sapiens  Central Nervous System  29867213
2  Fan_Embryonic_CTX_Big_Groups_Brain_Endothelial  Homo sapiens  Central Nervous System  29867213
3  Fan_Embryonic_CTX_Big_Groups_Excitatory_Neuron  Homo sapiens  Central Nervous System  29867213
4              Fan_Embryonic_CTX_Big_Groups_Glial  Homo sapiens  Central Nervous System  29867213
5       Fan_Embryonic_CTX_Big_Groups_Brain_Immune  Homo sapiens  Central Nervous System  29867213
6          Fan_Embryonic_CTX_Big_Groups_Microglia  Homo sapiens  Central Nervous System  29867213


This method is very verbose, but it's a start and it can be easily improved with a loop.

 #---- EDIT


Not knowing how the format works, there may be two issues:

1) gene sets with missing values only

STANDARD_NAME Aizarani_Liver_C33_Stellate_cells_2
ORGANISM
ORGAN_SYSTEM
PMID 31292543


2) gene sets with missing value and type

STANDARD_NAME Aizarani_Liver_C33_Stellate_cells_2
PMID 31292543


Case 1 is not an issue, as the code above will simply add an empty cell and go to the next line. For Case 2 we can use unique to get a list of all define types, create an empty data.frame with those types and have a loop (or sapply) to add the values in the right columns.

We can check the nature of metadata.txt:

 types <- gsub("^(.*)\t.*\$","\\1",dat)
sapply(unique(types), function(t) length(grep(t,types)))

STANDARD_NAME                ORGANISM            ORGAN_SYSTEM                    PMID       PUBLICATION_TITLE
257                     257                     257                     257                     257
AUTHORS                   GEOID            EXACT_SOURCE    EXTERNAL_DETAILS_URL                    CHIP
257                     257                     257                     257                     257
CATEGORY_CODE             CONTRIBUTOR         CONTRIBUTOR_ORG       DESCRIPTION_BRIEF RAW_PUBLICATION_MEMBERS
257                     514                     257                     257                     257


Your are lucky and you don't have to worry about shifting because of missing values. There is a fixable issue with CONTRIBUTORS, but it is related to the regex, not the file format.

• That should work, but if any of the records are missing a field, then they will be shifted. – burger Oct 31 '19 at 18:52
• I didn't think of that. To avoid this issue, you could get the list of all the types (STANDARD_NAME, ORGANISM,...), use unique to avoid repetition and create an empty data.frame with those columns. When you return the values, you can check the type and fill the right column. I will try to update my answer when I have some time. – fra Nov 4 '19 at 10:47