I was trying to run a slurm job script inside nextflow process. Is it possible? For instance, I have a job script 'PRE_PHASING_QC.JOB' and outside nextflow it runs perfectly, I want to integrate it in a nextflow script. May be my example code could be an explanation of what I want.
prephase='PRE_PHASING_QC.JOB'
process prephasing {
input:
file(prephasing) from prephase
"""
sbatch "${prephasing}
"""
}
The error:
sbatch: error: This does not look like a batch script
...
I tried to make my job script executable, put it in bin and then include it in the nextflow process according to @Steve. So, the script runs without any error but got no output. The original job script takes 3 files (bed,bim,fam) as input and produce multiple outputs. I guess that I have to declare outputs inside nextflow process. So far I tried as follows:
process prephasing {
publishDir params.out, mode: 'copy'
output:
tuple val(chr),val(num),val(steps),path("${chr}${num}.${steps}.{bed,bim,fam,log}") into out1
script:
"""
PRE_PHASING_QC.job
"""
It gives:
Error executing process > 'prephasing'
Caused by:
Missing value declared as output parameter: chr
My expected output would be chr{1-22}.step{1-10}.{bed,bim,fam,log}
#!/bin/bash
#SBATCH --job-name=prephase
#SBATCH --account=p697_norment
#SBATCH --time=1:00:00
#SBATCH --ntasks=1
#SBATCH --mem-per-cpu=7600M
#SBATCH --cpus-per-task=7
#SBATCH --array=1-22
source /cluster/bin/jobsetup
set -o errexit
module load plink
export prefix=chr${SLURM_ARRAY_TASK_ID}
export hrc_sites_file=/cluster/projects/p697/projects/moba_qc_imputation/resources/HRC/HRC.r1-1.GRCh37.wgs.mac5.sites.tab.gz
export HRC_1000G_check_bim=/cluster/projects/p697/github/norment/moba_qc_imputation/software/HRC-1000G-check-bim.pl
# Step 1a (exclude duplicated), 1b (exclude indels and non-ACTG alleles), 1c (exclude ambiguous SNPs)
plink --bfile hce-ec-eur-fin-pass-qc --chr ${SLURM_ARRAY_TASK_ID} --out $prefix --make-bed || true
plink --bfile $prefix --list-duplicate-vars ids-only suppress first --out $prefix.stepla || true
plink --bfile $prefix --exclude $prefix.stepla.dupvar --make-bed --out $prefix.stepla || true
plink --bfile $prefix.stepla --snps-only just-acgt --make-bed --out $prefix.steplb || true
awk '{ if (($5=="T" && $6=="A")||($5=="A" && $6=="T")ii($5=="C" && $6=="G")||($5=="G" && $6=="C")) print $2, "ambig" ; else print $2 ;}' $prefix. steplb.bim | grep ambig > $prefix.steplb.snplist.txt || true
plink --bfile $prefix.steplb --exclude $prefix.steplb.snplist.txt --make-bed --out $prefix.steplc || true
# Step 2. Zero out Mendelian errors
plink --bfile $prefix.steplc --me 1 1 --set-me-missing --make-bed --out $prefix.step2 || true
# Step 3. Missingness and HWE + Step 4. Compute allele frequencies
# NB! --mind 0.02 filter is removed from the next command, to avoid sample QC per chunk, which may leave inconsistent number of individuals avail able per chunk.
plink --bfile $prefix.step2 --hwe le-6 --geno 0.02 --make-bed --out $prefix.step3 || true
plink --bfile $prefix.step3 --freq --make-bed --out $prefix.step4 || true
# Step 4-10. Run Oxford perl script and apply first 5 lines of its Run-plink.sh command, followed by '--update-name' command (update marker name s)
perl ${HRC-IRC_1000G_check_bim} -b $prefix.step4.bim -f $prefix.step4.frq -r ${hrc_sites_file} -h || true
plink --bfile $prefix.step4 --exclude Exclude-$prefix.step4-HRC.txt --make-bed --out $prefix.step5 || true
plink --bfile $prefix.step5 --update-chr Chromosome-$prefix.step4-HRC.txt --make-bed --out $prefix.step6 || true
plink --bfile $prefix.step6 --update-map Position-$prefix.step4-HRC.txt --make-bed --out $prefix.step7 || true
plink --bfile $prefix.step7 --flip Strand-Flip-$prefix.step4-HRC.txt --make-bed --out $prefix.step8 || true
plink --bfile $prefix.step8 --a2-allele Force-Allele1-$prefix.step4-HRC.txt --make-bed --out $prefix.step9 || true
plink --bfile $prefix.step9 --update-name ID-$prefix.step4-HRC.txt --real-ref-alleles --make-bed --out $prefix.step10 || true
@Steve. I added the job file as a png (not sure how to add text file). In line 21hce-ec-eur-fin-pass-qcis the only input which create chr(1-22).(bed,bim,fam) files (for 22 chromosomes) which are used in the subsequent commands. I tried to replacehce-ec-eur-fin-pass-qcwith$1but got no output. Maybe you could give a better idea. $\endgroup$