0
$\begingroup$

[![example csv file][1]][1]

First I want to start by parsing and doing some operation on a text/csv file. How could I read a csv/text file in nextflow?

Suppose, I have above csv file with 3 columns. Lets say I want to create a channel which emits an array from minimum of column 2 (7.9e5) to maximum of column 2 (8.8e5) with an increment of 1e4. How could I create the following channel and later access it in the script? Any help? The

array=([1]="7.9e5 8.0e5" [2]="8.0e5 8.1e5" [3]="8.1e5 8.2e5" [4]="8.2e5 8.3e5" [5]="8.3e5 8.4e5" [6]="8.4e5 8.5e5" [7]="8.5e5 8.6e5" [8]="8.6e5 8.7e5" [9]="8.7e5 8.8e5" [10]="8.8e5 8.9e5")

I need to do some operations with this array in the script which will generate multiple files for each element in the array. These multiple files need to be merged in a single file. Sorry, I could not provide any nextflow script that I tried, because I not sure how create an array with splitCsv

I have tried in python:

df1=pd.read_csv('data1/omni-ec-eur-fin-pass-qc.bim',sep='\t',header=None)
arr1=np.arange(df1.iloc[:,3].min()-1,df1.iloc[:,3].max()+1,3000000)
diffs = []
for x in zip(arr1[0::],arr1[1::]):
    diffs.append(x)
arr2=np.arange(1,len(diffs)+1,1)
dict(zip(arr2,diffs))

Which gave me:

{1: (12343, 3012343),
 2: (3012343, 6012343),
 3: (6012343, 9012343),
 4: (9012343, 12012343),...

But I need something like this:

([1]="12343 3012343" [2]="3012343 6012343" [3]="6012343   9012343" [4]="9012343 12012343"....)

It could be arrays (22 arrays for 22 chromosome) or a csv file containing arrays for each chromosome which could be used as an input channel. There are 22 bim files (one for each chromosome) stored in the output channel which would be used to create the above mentioned array. The original data inside the bim looks like this:

    0   1   2   3   4   5
0   1   rs11240777  0   798959  A   G
1   1   rs4475691   0   846808  A   G
2   1   rs7537756   0   854250  G   A
3   1   rs13302982  0   861808  A   G
4   1   rs1110052   0   873558  C   A

The 4th column would be used to create the arrays. [1]: https://i.sstatic.net/cxqyl.png

Improve this question
$\endgroup$
18
  • 1
    $\begingroup$ PLease edit your question and give us an example file and minimal nextflow script to work with. $\endgroup$
    – terdon ♦
    Commented Apr 9, 2021 at 13:00
  • $\begingroup$ Is there any book to learn nextflow operations? I was trying to find any book to learn nextflow but failed. At least if there is any example of creating array in nextflow would be great. $\endgroup$
    – zillur rahman
    Commented Apr 10, 2021 at 20:22
  • 1
    $\begingroup$ @zillurrahman Generally, it's better to off-load all work to one or more processes. Writing a process to create a range of values, like in your example, is trivial in Perl/Python. You can use your favorite language in a process' script block. Please see: nextflow.io/docs/latest/process.html#scripts-a-la-carte $\endgroup$
    – Steve
    Commented Apr 11, 2021 at 10:05
  • 1
    $\begingroup$ @zillurrahman Do you really need to create the array string? If all you eventually want to do is create a channel of tuples, your first process should write a simple CSV which can then be split using the splitCsv operator. $\endgroup$
    – Steve
    Commented Apr 11, 2021 at 23:19
  • 2
    $\begingroup$ @zillurrahman Also, your comments above should be edits to your question. Small snippets of unformatted code isn't especially helpful. Please include a small minimal example and the code that reproduces the issue. $\endgroup$
    – Steve
    Commented Apr 11, 2021 at 23:30

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.