Tidying MEGAN Taxonomy for Python/R Analysis

Question

I am analysing some WGS data in MEGAN and would like to do some additional analysis in Python/R

I am having trouble Tidying the Taxonomic data in a format which would be conducive to this. Originally I tried exporting Taxon Path to count and using the Split function in Pandas to break up taxonomy into new columns as this worked well for KEGG Pathways. Due to varying specificity of taxonomic assignments this didn't work.

I've had more luck Exporting Data as a BIOM file format and converting it to TSV resulting in a file with the following column containing all taxonomic info (See below)

Ideally , I'd like to generate columns for; Domain, Phylum , Class , Order, Family and Species by accessing the features contained in Taxonomy. I'm unaware of any functions which would let me do that.

an idea of what I'm trying to achived is

Domain	Phylum	Class	Order	Family	Species
Bacteria					uncultured bacterium
Bacteria	Bacteroidetes	Bacteroidia	Bacteroidaceae	Bacteroides	Bacteroides acidifaciens

From

Taxonomy
d__Bacteria; s__uncultured bacterium
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides acidifaciens
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides caccae
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides cellulosilyticus
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides faecis
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides finegoldii
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides fragilis
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides intestinalis
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae;
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae;
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides salyersiae
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides stercoris
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides thetaiotaomicron
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides uniformis
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides xylanisolvens
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides cellulosilyticus CAG:158
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__uncultured Bacteroides sp.
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides sp. 2_2_4
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides sp. 3_1_23
d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides sp. D2

Answer 1

Save the following as a python script (e.g., named splitty.py). Make the script executable by doing chmod +x splitty.py. Then, if your MEGAN data are in a file called taxonfile.csv, run splitty.py by doing ./splitty.py taxonfile.csv. Get help on the script by doing splitty.py -h.

#!/usr/bin/env python3

"""
Splitty

Use this to split up a taxonomy table
"""

def argsparser():
    """
    argsparser
    """
    import argparse
    parser = argparse.ArgumentParser(prog="splitty",
                                    formatter_class=argparse.ArgumentDefaultsHelpFormatter,
                                    description="Uset splitty to get taxon table.")
    parser.add_argument("table", help="Taxon table in semicolon separated format.")
    return parser

def parseline(line_in):
    """
    Parse each line_in as return a dict
    """
    transl = {"d": "Domain",
             "p": "Phylum",
             "c": "Class",
             "o": "Order",
             "f": "Family",
             "g": "Genus",
             "s": "Species"}
    return {transl[val[0]]: val[1] for val in [i.split("__") for i in line_in]}

def main(args):
    """
    The main routine.
    """
    import pandas as pd
    df_list = []
    with open(args.table, "r") as input_handle:
        for line in input_handle.readlines():
            linelist = parseline(list(filter(None, [i.strip() for i in line.rstrip().split(";")]))[1:])
            df_list.append(pd.DataFrame(linelist, index=["Taxon"]))
    print(pd.concat(df_list).to_csv(sep="\t", index=False))

if __name__ == "__main__":
    main(argsparser().parse_args())