Mapping to assemblies in subdirectories

Question

I am performing a bash loop one liner:.

for k in */Assembly/*/*.fastq; do minimap2 -ax map-ont assembly.fasta $k > mapping.sam; echo $k; done

The file hierarchy is as follows:

\Drain_4 
    \Assembly
        \A4_assembly
           assembly.fasta
           A4.fastq
        \D3_assembly
        \I8_assembly
        \U7_assembly
\Drain_5
\Drain_7

Where \ denotes a directory. This is the error i am getting:

[ERROR] failed to open file '.fasta': No such file or directory A4.fastq

Both input files are in the same directory.

The fastq is my current directory. I'm guessing i will need to provide the complete pathname for both my input files?

I should mention that in all folders the assembly.fasta file is named the same but the fastq file is different (has a different prefix depending on which folder it is in.

Answer 1

The most likely issue is that the assembly.fasta file is not found in the current directory. Taking @m's comments into consideration, a minimal rewriting of your script to fix this would be as follows:

for k in */Assembly/*/*.fastq; do minimap2 -ax map-ont $(dirname ${k})/assembly.fasta $k > $(dirname ${k})/mapping.sam; echo $k; done

But I'd recommend at least adding additional quote and brace protection, just in case file names contained spaces:

for k in */Assembly/*/*.fastq;
  fDir = $(dirname ${k});
  do minimap2 -ax map-ont "${fDir}/assembly.fasta" "${k}" > "${fDir}/mapping.sam";
  echo ${k};
done

edit: fixed up variable capitalisation

Answer 2

The solution is quite simple you issue the one liner either from within A4_assembly directory or else NC_D2 (see below). It will then find assembly.fasta and the error will disappear. That's because the current working directory is assigned to the directory in which the one liner is issued. Directory A4_assembly is where assembly.fasta is situated, thus is will work.

If will fail to find the fastq file thus you will need to rewrite the loop, simple,

for k in *.fastq; do minimap2 -ax map-ont assembly.fasta $k > mapping.sam; echo $k; done

... will work, executed from within Assembly_4

The one liner suggests you are executing the command from the directory above the Drain_4 directory. If this is how you intended the one liner to operate I don't think its the solution to the one liner, because there is a risk you will encounter logical errors (not bugs) ..

*/Assembly/*/*.fastq

If you did execute this commend from above Drain_4 it will looking all over the place here in every Drain and every assembly directory therein. If you executed this anywhere else it will fail to find *.fastq because the start of the wildcards is the current working directory in a bash for relative paths.

However, when you dump the file it goes into one one place:

 > mapping.sam

So there is a risk are performing loads and loads of minimaps but they are simply overwriting each of other, i.e. they are replacing each other.

The other issue is

NC_D2/

This directory is not represented in your directory file.

Fundamentally the problem is that assembly.fasta is always in one location which appears to be in the directory above the directory structure you described, viz. */Assembly/*/*.fastq ... according to the one liner, but in the directory structure its in A4_assembly it is also likely to be in \D3_assembly amongst loads of others and other Drains (master directories).

Thus even if it 'finds' the reference genome (if that is what it is?), it will see that for everything ... and I mean everything.

There are dynamic solutions to what I think the code using bash is looking to achieve because you can issue regex from within the do ... done, or simply do it as a script.

---

I suggest to simply execute the first script I wrote in every Assembly directory then everything will simply work and there's no risk of logical errors.