1
$\begingroup$

I have already asked this question in another forum but have not got an appropriate answer, so wondering if anyone here can help me? I have a big file with over 3 million columns (SNPs genotypes) and 800 rows (one row for each individual).

The first row is the SNPs_ID name with the SNP position on each chromosome (see the example below please), in total there are 17 chromosomes. I want to extract SNPs for each chromosome and store them in a separate file. I have done it with "R", but I am looking for a faster solution and I am wondering if anyone can help me to figure out another solution with bash, perl or python codes?

here is an example of my input file (SNPs on chromosome 1,2,3,4...,17)

"Ha412HOChr01:180159" "Ha412HOChr01:210724" "Ha412HOChr01:303270" "Ha412HOChr01:303280"....... "Ha412HOChr17:303402"
0 1 0 0 ......0
0 1 0 0 ......0
0 1 0 0 ......0
0 2 0 0 ......0
0 1 1 1 ......1
0 2 0 0 ......0

and this is an example of my desired output for example for chromosome 1:

out.chrom1
"Ha412HOChr01:180159" "Ha412HOChr01:210724" "Ha412HOChr01:303270" "Ha412HOChr01:303280" 
0 1 0 0 
0 1 0 0
0 1 0 0 
0 2 0 0 
0 1 1 1 
0 2 0 0 
0 0 0 0 
0 2 0 0
0 1 2 2 
$\endgroup$
  • 1
    $\begingroup$ Hi, good question but perhaps post your R code? $\endgroup$ – Michael G. Oct 10 at 4:32
  • $\begingroup$ Are the leading spaces part of your input file? Does the second row, for example really start with 4 spaces? If not, please edit and remove them to avoid confusion. Also, are those really the IDs? Do they all start with Ha412HO, then Chr and number(s), followed by : and the position? Or can the Ha412HO change? $\endgroup$ – terdon Oct 10 at 14:27
  • $\begingroup$ I just edited my question according to your comment. And yes, the IDs start with Ha412HOChr followed by chromosome, the : and SNP position $\endgroup$ – Anna1364 Oct 10 at 15:26
2
$\begingroup$

I believe a pandas approach would be fast enough (probably faster than any other Python approach without pandas).

After converting your table to a pandas data frame, you can do something like df.filter(regex=(".*Chr01")) for the first chromosome then write the resulting data to file with the to_csv() method.

import pandas as pd

data = pd.read_csv('input.txt', delimiter = "\t", index_col = False)

for i in ["Chr01", "Chr02", "Chr03"]:
    regex_pattern = ".*" + i
    print(data.filter(regex=regex_pattern))
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.