1
$\begingroup$

can anyone help me improve this pandas code?

import pandas as pd

df = pd.DataFrame(
    [
        [
            'chr1', 222
        ],
        [
            'chr1', 233
        ],
        [
            'chr1', 2123
        ],
        [
            'chr2', 244
        ]
    ], columns = ['chrom', 'pos']
)

df2 = pd.DataFrame(
    [
        [
            'chr1', 221, 223
        ],
        [
            'chr1', 230, 240
        ],
    ], columns = ['chrom', 'start', 'end']
)

Gives me 2 dfs with genomic coordinates. The first one is an exact position:

    chrom   pos
0   chr1    222
1   chr1    233
2   chr1    2123
3   chr2    244

and the second is a range:

    chrom   start   end
0   chr1    221 223
1   chr1    230 240

I need to find the count of exact coordinates that are in one of the ranges (in the same chrom)

This works but is slow:

c=0
for chrom, data in df.groupby('chrom'):
    tmp = df2.query(f'chrom == "{chrom}"')
    for p in data.pos:
        for s, e in zip(tmp.start, tmp.end):
            if s < p < e:
                c+=1

Then c = 2

I think I can use agg to do this without iteration (and hopefully faster) but I can't get it working. Can anyone show me how?

PS I am also asking this on stackoverflow.

$\endgroup$
4
  • 2
    $\begingroup$ codereview.stackexchange.com is probably the best place to ask questions about optimising code. $\endgroup$
    – user438383
    Feb 12 at 9:36
  • 1
    $\begingroup$ That actually a really good question. A bioinformatics question. It's self contained and we'll explained. If U don't know just don't answer. No need for negatively. $\endgroup$ Feb 12 at 10:35
  • $\begingroup$ I'm actually trying to help you, since code review (hint) is a much better place to get your code reviewed than either here or SO. You seem to be new here - so different sites have different functions. It's not about negativity / whether or not I can answer the question, it's about the correct questions are on the correct sites. $\endgroup$
    – user438383
    Feb 12 at 11:24
  • $\begingroup$ Ok fair enough. $\endgroup$ Feb 12 at 11:26
3
$\begingroup$
# pip install pyranges or conda install -c bioconda pyranges

import pyranges as pr

g1 = pr.from_string("""Chromosome Start End
chr1   222 223
chr1   223 224
chr1   233 234
chr1   235 236
chr1   2237 238
chr1  2123 2124
chr2   244 245""")

g2 = pr.from_string("""Chromosome Start End
chr1    221  223
chr1    230  240
chr2    0 1000""")

r = g2.count_overlaps(g1)
r
# +--------------+-----------+-----------+------------------+
# | Chromosome   |     Start |       End |   NumberOverlaps |
# | (category)   |   (int32) |   (int32) |          (int64) |
# |--------------+-----------+-----------+------------------|
# | chr1         |       221 |       223 |                1 |
# | chr1         |       230 |       240 |                2 |
# | chr2         |         0 |      1000 |                1 |
# +--------------+-----------+-----------+------------------+
# Unstranded PyRanges object has 3 rows and 4 columns from 2 chromosomes.
# For printing, the PyRanges was sorted on Chromosome.

r.df
# 0       chr1    221   223               1
# 1       chr1    230   240               2
# 2       chr2      0  1000               1
$\endgroup$
1
  • $\begingroup$ How did I not know about pyranges!? Thanks! $\endgroup$ Feb 24 at 21:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.