3
$\begingroup$

I'm trying to download around 2000 proteomes from NCBI, but all I have is the genus and species e.g: lophodermium_seditiosum trichoderma_harzianum

Does anyone know a good way of doing this in script form?
My code at the moment is:

from Bio import Entrez
from Bio import SeqIO
import os

file_name = "proteome1.faa"
net_handle = Entrez.efetch(db="protein", id="trichoderma_harzianum", rettype="gp", retmode="text")
out_handle = open(file_name, "w")
out_handle.write(net_handle.read())
out_handle.close()
net_handle.close()
print("Saved")


for seq_record in SeqIO.parse("proteome1.faa", "genbank"):
    print(seq_record.id)
    print(repr(seq_record.seq))
    print(len(seq_record))

But I think it wants a number as the id, I don't know if it's possible to convert species name to a numerical ID within the script, or if that's the best way of going about this.

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
3
$\begingroup$

The general procedure is explained here:

First step is to search the Taxonomy database with the organism name. Accepted common names usually work at all taxonomic levels. Use the scientific name or formal name if no results are obtained with the common name.

In code:

from Bio import Entrez

Entrez.email = 'user@example.org' # Put your email here

def get_tax_id(species):
    species = species.replace('_', '+').strip()
    search = Entrez.esearch(term=species, db='taxonomy', retmode='xml')
    record = Entrez.read(search)
    return record['IdList'][0]

if __name__ == '__main__':
    organisms = ['lophodermium_seditiosum', 'trichoderma_harzianum']
    taxids = [get_tax_id(organism) for organism in organisms]
    print(taxids)

This gives us the taxids for our species, respectively 128023 and 5544. Now we can use these to get the proteomes. For Lophodermium seditiosum we find two results, but for Trichoderma harzianum there are 65900 results, so you might want to tweak the query to filter the results. For example, if you only want the sequences appearing in PDB you add AND pdb[filter] so that the query becomes txid5544[Organism:exp] AND pdb[filter].

def get_proteomes(taxid):
    query = 'txid{}[Organism:exp]'.format(taxid)
    handle = Entrez.esearch(db='protein', term=query)
    result = Entrez.read(handle)
    if int(result['Count']) <= 2000:
        ids = ','.join(result['IdList'])
        proteomes = Entrez.efetch(db='protein', id=ids, rettype='fasta').read()
        return proteomes
    else:
        raise RuntimeError('Too many results!')

if __name__ == '__main__':
    print(get_proteomes('128023'))

Returns:

>ADI44294.1 actin, partial [Lophodermium seditiosum]
CKAGFAGDDAPRAVFPSIVGRPRHHGIMIGMGQKDS

>ADI44293.1 actin, partial [Lophodermium seditiosum]
CKAGFAGDDAPRAVFPSIVGRPRHHGIMIGMGQKDS
Improve this answer
$\endgroup$
3
  • $\begingroup$ So I'm getting a 'Supplied ID parameter is empty' error from this code, the functions are defined at the beginning and are the same as in the answer listoforganisms = [x.split('\t')[0] for x in open("OGTlist.csv").readlines()] IDlist = [] #convert to id if __name__ == '__main__': organisms = listoforganisms for organism in organisms: taxid = get_tax_id(organism) IDlist.append(taxid) print (taxid) print(get_proteomes((taxid))) $\endgroup$
    – Biomage
    Jun 22, 2018 at 9:27
  • $\begingroup$ Can you post a small sample of your OGTlist.csvfile? $\endgroup$
    – BioGeek
    Jun 22, 2018 at 9:40
  • 1
    $\begingroup$ Got it working with, if __name__ == '__main__': organisms = listoforganisms for organism in organisms: taxid = get_tax_id(organism) print(organism) print (taxid) stringid = str(taxid) print(get_proteomes(stringid.strip("'[]'"))) Just needed to strip the list elements to the right form $\endgroup$
    – Biomage
    Jun 22, 2018 at 13:24
1
$\begingroup$

You can use the NCBI Datasets to download the protein sequences of all annotated protein-coding genes for an organism starting with the organism name. There is a web-interface that allows exploring by taxonomy or specific organism search as well as a command line utility and an API. For example, the command line utility can be used as follows:

$ datasets download genome taxon 'trichoderma harzianum' \
    --annotated --exclude-genomic-cds --exclude-gff3 \
    --exclude-rna --exclude-seq --reference
Collecting 1 genome accessions [================================================] 100% 1/1
Downloading: ncbi_dataset.zip    3.92MB done

This downloads a zip archive containing the protein.faa file for the Trichoderma harzianum reference assembly.

Note, unlike the Entrez method described above, this will not download all proteins for a given organism from the NCBI Protein portal. Thus, Lophodermium seditiosum does not return any results as there is no available genome for this assembly in the NCBI Assembly database.

On the other hand, the NCBI Datasets method will be significantly faster and return cleaner results. I would argue that if you are interested in obtaining the 'proteomes' of a given set of organisms, this is a better choice as it includes only those cases where a complete proteome is available based on the annotation of the genome and does not include cases where only a few proteins from an organism are available.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.