Timeline for Nextflow: run sbatch (slurm job script) inside a process
Current License: CC BY-SA 4.0
17 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 30, 2021 at 21:49 | vote | accept | zillur rahman | ||
| Mar 30, 2021 at 10:32 | comment | added | Steve | No worries at all @zillurrahman. I've updated my answer with an example on how to do this. I also refactored some of your code, hopefully it still works... | |
| Mar 30, 2021 at 6:49 | comment | added | zillur rahman |
Thanks @Steve . Yes, it starts a job array 1-22 for each chromosome. The initial input is one single bfile (bed,bim,fam), in step1a it creates 22 bfiles (bed,bim,fam) for each chromosome. The rest of steps operate on those 22 files to create final output. Any relatable example to break the job into two would be great.
|
|
| S Mar 30, 2021 at 6:08 | history | suggested | Steve | CC BY-SA 4.0 |
Added text from job script image
|
| Mar 30, 2021 at 3:12 | comment | added | Steve | Just realized my edit hasn't been approved yet, but you should be able to see it. | |
| Mar 29, 2021 at 22:57 | comment | added | Steve | See also: bioinformatics.stackexchange.com/editing-help | |
| Mar 29, 2021 at 22:50 | review | Suggested edits | |||
| S Mar 30, 2021 at 6:08 | |||||
| Mar 29, 2021 at 22:45 | comment | added | Steve | Please correct me if I'm wrong, but it looks like this script kicks off a job array. This isn't a good fit inclusion "as-is" into a Nextflow workflow, since it won't be portable at all. I think breaking it up into two scripts is the best option here. I'll see if I can give an example of this. Also, you can paste text in a code-block. I've updated your question with the text. Click edit to see the changes I've made. | |
| Mar 29, 2021 at 13:15 | comment | added | zillur rahman |
Thank you @Steve. I added the job file as a png (not sure how to add text file). In line 21 hce-ec-eur-fin-pass-qc is the only input which create chr(1-22).(bed,bim,fam) files (for 22 chromosomes) which are used in the subsequent commands. I tried to replace hce-ec-eur-fin-pass-qc with $1 but got no output. Maybe you could give a better idea.
|
|
| Mar 29, 2021 at 13:09 | history | edited | zillur rahman | CC BY-SA 4.0 |
added a png file
|
| Mar 29, 2021 at 10:08 | comment | added | Steve | So only files declared in your output block will get published into your publishDir. Ignore output in the publishDir while testing and developing. Or better, comment out the publishDir directive until you get your script block working in some capacity. In any case, your job script will show you what's going on. Happy to take a look if you need help. | |
| Mar 29, 2021 at 7:37 | comment | added | zillur rahman | The publishDir is empty and in the workDir there are only log files (chr.log, chr.step1.log, chr.step2.log etc.). It is strange! | |
| Mar 28, 2021 at 23:59 | comment | added | Steve | When you say your script ran without error but you got no output files, are you looking in the workDir or publishDir? When I'm developing, I usually only write the output block declaration after I've tested the tool/script a couple of times. This is because your task process can still run successfully without an output block declaration. I can then look in the workdir to see what files have been created, etc, then finally declare them in the output block. Then I start on the next (downstream) process(es) and repeat. | |
| Mar 28, 2021 at 23:01 | history | edited | zillur rahman | CC BY-SA 4.0 |
added 87 characters in body
|
| Mar 28, 2021 at 22:46 | history | edited | zillur rahman | CC BY-SA 4.0 |
added 790 characters in body
|
| Mar 28, 2021 at 12:28 | answer | added | Steve | timeline score: 2 | |
| Mar 28, 2021 at 8:41 | history | asked | zillur rahman | CC BY-SA 4.0 |