1
$\begingroup$

The following (thx to these answers) retrieves the info, in particular a long summary (such as "The membrane-associated protein encoded by this gene is a member of the superfamily of ATP-binding cassette... etc."), for a few mouse genes.

library(dplyr)

org.Mm.eg.db::org.Mm.egSYMBOL2EG %>% 
  as.data.frame() %>% 
  pull(gene_id) %>% 
  head() %>%  # !!!
  rentrez::entrez_summary(id = ., db = "gene") %>%
  { do.call("rbind", .) } %>%
  as.data.frame() %>%
  select(name, description, summary)

Note the head. The protocol/server does not allow large queries. How can I get the same information (incl. the long summary) for all the mouse genes at once?

$\endgroup$
2
  • $\begingroup$ Chunk your query, then use purrr::insistently. No need for a loop, this can be vectorised. I also recommend against a fixed-length sleep; using an exponential backoff is more polite. $\endgroup$ Jun 1, 2021 at 13:51
  • $\begingroup$ Thanks, I'll certainly have occasion for this one. $\endgroup$
    – user66081
    Jun 2, 2021 at 14:04

2 Answers 2

1
$\begingroup$

You could wrap the call too entrez_summary in a loop.
For example to cut the data into chunks of five genes:

gene.vector # this would be the genes you are interested in
gene.chunks <- ceiling(length(gene.vector) / 5)
# initializing an empty list
summary.list <- vector(mode = "list", length = gene.chunks)
for(i in gene.chunks){
   # select the ith set of five genes from your gene vector
   goi <- gene.vector[(i - 1) * 5 + 1:5]
   # if there are less than five elements there will be NAs that need to be removed
   goi <- goi[!is.na(goi)]
   # adding your code 
   summary.list[[i]] <- org.Mm.eg.db::org.Mm.egSYMBOL2EG %>% 
     as.data.frame() %>% 
     filter(gene_id %in% goi) %>% # here the current five genes are selected
     pull(gene_id) %>% 
     rentrez::entrez_summary(id = ., db = "gene") %>%
     { do.call("rbind", .) } %>%
     as.data.frame() %>%
     select(name, description, summary)
  
   Sys.sleep(10) # wait for 10 seconds
}

There will also be a way to do this using dplyr group. In this case the loop is a little easier to debug.

I added Sys.sleep at the end to reduce the frequency with which you are making requests from entrez. If you are posting too many requests in a short time this will be like a DDOS attack. You (and potentially your whole IP range including other people at your institution) could get banned from the service for a couple of hours. Maybe try it with a subset first.

If you really want to get the information for all mouse genes this probably will be rather slow. Alternatively, you can go to the entrez webpage and try to download the information as a list. Unfortunately, I don't know exactly where you will find this information.

$\endgroup$
1
$\begingroup$

The following bash script downloads the summaries into an XML file.

#!/usr/bin/env bash

# This requires the ncbi tool `gene2xml`
# sudo apt install ncbi-tools-bin

DIR="download_cache"
mkdir -p "$DIR"

URL="ftp://ftp.ncbi.nlm.nih.gov/gene/DATA/ASN_BINARY/Mammalia/Mus_musculus.ags.gz"
wget -S -O - "$URL" | gzip -d | gene2xml -b T -l T | gzip -9 > "$DIR/mm.xml.gz"

URL="ftp://ftp.ncbi.nlm.nih.gov/gene/DATA/ASN_BINARY/Mammalia/Homo_sapiens.ags.gz"
wget -S -O - "$URL" | gzip -d | gene2xml -b T -l T | gzip -9 > "$DIR/hs.xml.gz"

These XML files can be converted to tables along these lines, viz.:


import xml.etree.ElementTree as ET
import gzip

from pathlib import Path

from tcga.utils import mkdir, unlist1, relpath


def parse(src):
    def find_text(e, path):
        e = e.findall(path)
        if e:
            return unlist1(e).text
        else:
            return ""

    yield ('gene_id', 'description', 'summary')

    with gzip.open(src, mode='r') as fd:
        e: ET.Element
        for (x, e) in ET.iterparse(fd, events=['end']):
            if e.tag == "Entrezgene":
                gene_id = find_text(e, "**/Gene-track_geneid")
                descptn = find_text(e, "**/Gene-ref_desc")
                summary = find_text(e, "Entrezgene_summary")

                yield (gene_id, descptn, summary)

                e.clear()


def main():
    for src in Path(__file__).parent.glob("download_cache/*.xml.gz"):
        trg = mkdir(Path(__file__).with_suffix('')) / Path(src.stem).with_suffix('.tsv.gz')
        if trg.is_file():
            print("Skipping existing file:", relpath(trg))
        else:
            print("Writing:", relpath(trg))
            with gzip.open(trg, mode='wt') as fd:
                for t in parse(src):
                    print(*t, sep='\t', file=fd)


if __name__ == '__main__':
    main()

Btw., a table of vertebrate gene homology is available here (it's just a tab-separated table), and can be used to impute summaries by homology.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.