I'm expecting to get 17 different paired-end fastq files (34 in total), so I want to make a bash script to just run my code through all the fastq files in a directory at once. How can I change the name of the input and output files each time the script runs through each file? So when it moves to the file_002, all names have file_002 at the beginning instead of file_001, and so on. And also, when merging the R1 and R2 reads how can I make that it only merges the correspondant files with a loop? for examples merging only file_001_R1 with file_001_R2, file_002_R1 with file_002_R2, file_003_R1 with file_003_R2, and so on.

This is my code:

for file in directory_name
pear -f file_001_R1.fastq.gz -r file_001_R2.fastq.gz -o file_001.fastq
cutadapt -g TGATAACAATTGGAGCAGCCTC...GGATCGACCAAGAACCAGCA -o file_001_barcode.fastq file_001.fastq
cutadapt -g GTGTACAAATAATTGTCAAC...CTGTCTCTTATACACATCTC -o file_001_UMI.fastq file_001.fastq
seqkit concat file_001_barcode.fastq file_001_UMI.fastq > file_001_concatenation.fastq
seqkit rmdup -s file_001_concatenation.fastq -o file_001_unique_pairs.fastq
seqkit subseq -r file_001_unique_pairs.fastq > file_001_unique_barcodes.fasta
bowtie -q --suppress 1,2,4,6,7,8 -x ref_index file_001_unique_barcodes.fasta > file_001_barcodes_allignment.bowtie
sort file_001_barcodes_allignment.bowtie | uniq -c > file_001_barcode_counts.txt
awk 'BEGIN{print "Barcode,TF_variant,Code"}{print $3","$2","$1}' file_001_barcode_counts.txt > file_001_barcode_counts.csv

Thank you.

  • 2
    $\begingroup$ I assume the code is directory_name/*, $file is now by each file within directory_name. At the moment $file is simply directory_name as its written. There'll be bash experts here that will describe the loop for the paired end files. $\endgroup$
    – M__
    Commented Jun 29, 2023 at 16:25
  • $\begingroup$ Hi, please consider upvoting and accepting one of the answers. These are good answers from two different perspectives: one pulling the directory file contents; the second using a numerical loop. Both would work. $\endgroup$
    – M__
    Commented Jun 30, 2023 at 16:43

2 Answers 2


Here's what I use - along with an example you can modify as needed:


for R1_FASTQ in "$FASTQ_DIR"*R1*.fastq.gz
                         BASE=$(basename "$R1_FASTQ" _R1.fastq.gz)
                         echo "BASE: $BASE"
                         echo "R1: $R1_FASTQ"
                         echo "R2: $R2_FASTQ"
                         echo "R3: $R3_FASTQ"
                         # process R1/R2 here using "$R1_FASTQ" "$R2_FASTQ" for input, and "${BASE}_XXXXXXX.fastq" for output

Basically, the strategy is to iterate over all the R1 FASTQs in a directory, calculate a BASE name that can be used to rename output files with arbitrary extensions (like .sam, .bam, etc...), and use BASH shell parameter expansion to calculate and set variables to the R2 (and R3 if necessary) reads present in an Illumina run that uses standard Illumina naming conventions.

This may need modification depending on the naming conventions you are using.


Something like this? This uses printf to create a zero-padded 3-digit number corresponding to 17 numbers from 1 to 17:

for fileBase in $(printf 'file_%03d\n' $(seq 1 17));
   do echo ${fileBase}_R1.fq.gz ${fileBase}_R2.fq.gz;
   pear -f ${fileBase}_R1.fastq.gz -r ${fileBase}_R2.fastq.gz -o ${fileBase}.fastq

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.