0
$\begingroup$

I want to make function get_abstract, but i don't know what this can look like if the function must return idlist.

from Bio import Entrez, Medline, SeqIO


list_of_ids = []
Entrez.email = '[email protected]'

handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retmax=100000)
record = Entrez.read(handle_one)
idlist = record["IdList"]
for ID in idlist:
    list_of_ids.append(ID)

    if len(list_of_ids) == 100000:
        handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retstart=100000, retmax=100000)
        record = Entrez.read(handle_one)
        idlist = record["IdList"]
        for ID in idlist:
            list_of_ids.append(ID)

            if len(list_of_ids) == 200000:
                handle_one = Entrez.esearch(db="pubmed", term='e coli', 
                           retmode="text", retstart=200000, retmax=100000)
                record = Entrez.read(handle_one)
                idlist = record["IdList"]
                for ID in idlist:
                    list_of_ids.append(ID)

so the args will be the start and TERM:

get_abstracts(start,TERM): 
handle_one = Entrez.esearch(db="pubmed", term=TERM, 
                               retmode="text", retstart=start, retmax=100000)
                    record = Entrez.read(handle_one)
                    idlist = record["IdList"]
                    for ID in idlist:
                        list_of_ids.append(ID)

Can someone help refactor this? cheers

$\endgroup$
3
  • 1
    $\begingroup$ Do you really mean to have a for loop nested inside another for loop that both reuse the same variables? $\endgroup$
    – Devon Ryan
    Nov 2, 2017 at 19:29
  • $\begingroup$ This type of questions are more suited for code review SE. What is your bioinformatic question? Do you need a new method to do it? A faster one? $\endgroup$
    – llrs
    Nov 3, 2017 at 7:10
  • $\begingroup$ Refactoring is for speed up script. I dont know any alternatives for download many abstracts. Selenium is slow. And wanna be better programmer , thats why I ask. I dont have any helper. $\endgroup$
    – MTG
    Nov 3, 2017 at 13:55

1 Answer 1

4
$\begingroup$

Here is a sample of code that should get you started.

I created a separate get_abstracts function. Then that function get called repetitively until less than 100K ids are returned. If you wanted to get fancy this whole process could be called as a recursive function but that seemed overly complex.

BUFFER_SIZE = 100000

def get_abstracts(start, term):
    """Get N number of records from Entrez
    :returns list: List of ids returned from search
    """
    handle_one = Entrez.esearch(db="pubmed", term=term, 
                       retmode="text", retstart=start, retmax=BUFFER_SIZE)
    record = Entrez.read(handle_one)
    idlist = record["IdList"]
    return idlist

list_of_ids = []
start_at = 0

# Iterate the search returning 100000
while true:
    ids = get_abstracts(start_at, 'e coli')

    list_of_ids.extend(ids)

    # Break if less than 100K results were returned 
    if len(ids) < BUFFER_SIZE:
        break
    start_at += BUFFER_SIZE

# Do something with list_of_ids
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.