2
$\begingroup$

I am removing illumina adapters of the NGS data with a loop. My NGS data is storage in /data/HTS_seq/. I used this function:

 for infile in /data/HTS_seq/*_read1.fastq; do outfile=$infile\_trim.fastq cutadapt -a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC -e 0.1 -O 3 -m 30 --max-n 2 -o $outfile $infile; done

 for infile in /data/HTS_seq/*_read2.fastq; do outfile=$infile\_trim.fastq cutadapt -a AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTA -e 0.1 -O 3 -m 30 --max-n 2 -o $outfile $infile; done

and It gives to me this error:

  IOError: [Errno 20] Not a directory: '/data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq/_trim.fastq'
  Traceback (most recent call last):
  File "/usr/local/bin/cutadapt", line 10, in <module>
cutadapt.main()
  File "/usr/local/lib/python2.7/dist-packages/cutadapt/scripts/cutadapt.py", line 652, in main
options.discard_untrimmed)
  File "/usr/local/lib/python2.7/dist-packages/cutadapt/scripts/cutadapt.py", line 327, in trimmed_and_untrimmed_files
trimmed = untrimmed = xopen(output_path, 'w')
  File "/usr/local/lib/python2.7/dist-packages/cutadapt/xopen.py", line 178, in xopen
return open(filename, mode)
      IOError: [Errno 20] Not a directory: '/data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq/_trim.fastq'

Which is my error? Thank you in advance.

$\endgroup$
0
1
$\begingroup$

First of all, you aren't actually setting the $outfile variable. Your loop has this basic format:

for var in *; do newVar="something" command $newVar; done

For example:

$ for var in 1 2 3 4; do newVar="new:$var" echo "$var: $newVar"; done
1: 
2: 
3: 
4: 

As you can see, newVar is never actually being set. That's because setting a variable is its own operation. So you need to do that, end it with a control character like ;, or &, or a newline, and then run your cutadapt:

$ for var in 1 2 3 4; do newVar="new:$var"; echo "$var: $newVar"; done
1: new:1
2: new:2
3: new:3
4: new:4
 

Or, with your data:

for infile in /data/HTS_seq/*_read1.fastq; do 
    outfile="$infile"_trim.fastq 
    cutadapt -a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC \ 
      -e 0.1 -O 3 -m 30 --max-n 2 -o "$outfile" "$infile"
done


for infile in /data/HTS_seq/*_read2.fastq; do 
    outfile="$infile"_trim.fastq
    cutadapt -a AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTA \ 
      -e 0.1 -O 3 -m 30 --max-n 2 -o $outfile $infile
done

But that would give a weird file name, like foo_S1_L001_read1.fastq_trim.fastq. You probably want names like foo_S1_L001_read1_trim.fastq instead, so you would want:

for infile in /data/HTS_seq/*_read1.fastq; do 
    outfile="${infile%.fastq}"_trim.fastq
    cutadapt -a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC \ 
      -e 0.1 -O 3 -m 30 --max-n 2 -o "$outfile" "$infile"
done


for infile in /data/HTS_seq/*_read2.fastq; do 
    outfile="${infile%.fastq}"_trim.fastq
    cutadapt -a AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTA \ 
      -e 0.1 -O 3 -m 30 --max-n 2 -o $outfile $infile
done

Now, the specific error you show makes me think that isn't the command you ran. Could you perhaps have run this one instead by mistake (note the / instead of \ before the _trim.fastq)?

for infile in /data/HTS_seq/*_read1.fastq; do
 outfile=$infile/_trim.fastq cutadapt -a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC \ 
-e 0.1 -O 3 -m 30 --max-n 2 -o $outfile $infile
done

Your error message is:

  IOError: [Errno 20] Not a directory: '/data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq/_trim.fastq'

Which means that cutadapt tried to open /data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq/ as a directory and then look for a file named _trim.fastq inside it. The only way I can think that this would happen is if you ran the command I gave above and not the one you show in your question.

$\endgroup$
1
  • $\begingroup$ It works! Thank you Terdon $\endgroup$ Jul 1 '20 at 9:53
-1
$\begingroup$

There is a bug here

/data/HTS_seq/*_read1.fastq 

This will go to your root directory / and there is definiately no folder called data in the root. To prove it type ls / and that is where the command is pointing, then compare this to ls ./.

To solve it go to the directory HTS_seq and type pwd then copy and paste the directory structure. I suspect ./data/HTS_seq will solve it

 for infile in ./data/HTS_seq; ...

This will look for the folder data in your current working directory rather than the root directory.


Okay you've been assigned /data as root ... no biggie.

PV589_03664AAB_AGGACT_read1.fastq/_trim.fastq

Why is / in the file name? What happens if you ls /data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq/_trim.fastq? and ls /data/HTS_seq/ Finally could you please ls /data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq\/_trim.fastq?

The \/ is important. What I think has happened is a / is in the file name and that will cause problems because it will think that _trim.fastq is a directory when its part of a file name, and would explain the error.

Then do the same thing using head instead of ls see what happens.

If the last ls and head works then rename the files by typing rename in the command line. If rename works it will be easy to remove / from all your files using the power of the rename function (its a Perl function BTW).

$\endgroup$
3
  • $\begingroup$ Hi Michael, thanks you for your answer. I typed pwd in HTS_seq folder and I got this direction: /data/HTS_seq Anyways I typed the code with the . : for infile in ./data/HTS_seq/*_read1.fastq; do outfile=$infile_trim.fastq cutadapt -a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC -e 0.1 -O 3 -m 30 --max-n 2 -o $outfile $infile; done and I got the same error. Which is my mistake? Thanks you $\endgroup$ Jun 30 '20 at 13:03
  • $\begingroup$ Hi @AdriánP.L. please try the above, I should have read the error more carefully first time around $\endgroup$
    – M__
    Jun 30 '20 at 14:19
  • 1
    $\begingroup$ You can never have a / in a file name. The / alone with \0 are the only two characters that can never be in file names. $\endgroup$
    – terdon
    Jun 30 '20 at 15:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.