0
$\begingroup$

I want to make function get_abstract, but i don't know what this can look like if the function must return idlist.

from Bio import Entrez, Medline, SeqIO


list_of_ids = []
Entrez.email = 'ski89@gmail.com'

handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retmax=100000)
record = Entrez.read(handle_one)
idlist = record["IdList"]
for ID in idlist:
    list_of_ids.append(ID)

    if len(list_of_ids) == 100000:
        handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retstart=100000, retmax=100000)
        record = Entrez.read(handle_one)
        idlist = record["IdList"]
        for ID in idlist:
            list_of_ids.append(ID)

            if len(list_of_ids) == 200000:
                handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retstart=200000, retmax=100000)
                record = Entrez.read(handle_one)
                idlist = record["IdList"]
                for ID in idlist:
                    list_of_ids.append(ID)

so the args will be the start and TERM:

get_abstracts(start,TERM): 
handle_one = Entrez.esearch(db="pubmed", term=TERM, 
                               retmode="text", retstart=start, retmax=100000)
                    record = Entrez.read(handle_one)
                    idlist = record["IdList"]
                    for ID in idlist:
                        list_of_ids.append(ID)

Can someone help refactor this? cheers

$\endgroup$
3
  • 1
    $\begingroup$ Do you really mean to have a for loop nested inside another for loop that both reuse the same variables? $\endgroup$
    – Devon Ryan
    Nov 2 '17 at 19:29
  • $\begingroup$ This type of questions are more suited for code review SE. What is your bioinformatic question? Do you need a new method to do it? A faster one? $\endgroup$
    – llrs
    Nov 3 '17 at 7:10
  • $\begingroup$ Refactoring is for speed up script. I dont know any alternatives for download many abstracts. Selenium is slow. And wanna be better programmer , thats why I ask. I dont have any helper. $\endgroup$
    – MTG
    Nov 3 '17 at 13:55
4
$\begingroup$

Here is a sample of code that should get you started.

I created a separate get_abstracts function. Then that function get called repetitively until less than 100K ids are returned. If you wanted to get fancy this whole process could be called as a recursive function but that seemed overly complex.

BUFFER_SIZE = 100000

def get_abstracts(start, term):
    """Get N number of records from Entrez
    :returns list: List of ids returned from search
    """
    handle_one = Entrez.esearch(db="pubmed", term=term, 
                       retmode="text", retstart=start, retmax=BUFFER_SIZE)
    record = Entrez.read(handle_one)
    idlist = record["IdList"]
    return idlist

list_of_ids = []
start_at = 0

# Iterate the search returning 100000
while true:
    ids = get_abstracts(start_at, 'e coli')

    list_of_ids.extend(ids)

    # Break if less than 100K results were returned 
    if len(ids) < BUFFER_SIZE:
        break
    start_at += BUFFER_SIZE

# Do something with list_of_ids
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.