Skip to main content
fix
Source Link
Karel Břinda
  • 1.9k
  • 10
  • 19

You can split your FASTA file sequence-by-sequence using

split -a 6 -p '^>' your_file.fa seq_

and then use any existing read simulator supporting coverage (ART, DWGsim, etc.). If you want to have all the reads mixed (not ordered by the original sequence), you can use RNFtools.

Edit 1:

As @terdon pointed out, the previous command works on OS X only. An analogical one liner for Linux (but with a slightly different naming scheme using numbers rather than letters) can be

csplit -f seq_ -n 6 your_file.fa '/^>/' {*}

To make this command work also on OS X, one needs to install coreutils (e.g., using brew) and then use gcsplit instead of csplit.

Edit 2:

Once FASTA is split by sequences, the simulation becomes straightforward and many different approaches can be used. My favorite one is using GNU Parallel. Imagine that you have your coverages in a text file called covs.txt on separate lines and in the same order as the sequences in your_file.fa, e.g.,

40
30
...

Then you can simulate reads from the original sequences using DWGsim by

ls -1 seq_* | paste covs.txt - \
    | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}

and merge the obtained FASTQ files using:

cat sim_seq_*.bwa.read1.fastq > reads.1.fq
cat sim_seq_*.bwa.read2.fastq > reads.2.fq

One possible danger of this approach is that we have assumed that the number of seq_* files is the same as the number of lines in covs.txt, which might not be true (by mistake). We should check this prior to the simulation step, e.g., by:

[[ "$(ls -1 seq_* | wc -l)" == "$(cat covs.txt | wc -l)" ]] \
    || echo "Incorrent number of lines in covs.txt"

Another caveat is that the simulated reads are not in a random order (they are grouped by source sequences).

You can split your FASTA file sequence-by-sequence using

split -a 6 -p '^>' your_file.fa seq_

and then use any existing read simulator supporting coverage (ART, DWGsim, etc.). If you want to have all the reads mixed (not ordered by the original sequence), you can use RNFtools.

Edit 1:

As @terdon pointed out, the previous command works on OS X only. An analogical one liner for Linux (but with a slightly different naming scheme using numbers rather than letters) can be

csplit -f seq_ -n 6 your_file.fa '/^>/' {*}

To make this command work also on OS X, one needs to install coreutils (e.g., using brew) and then use gcsplit instead of csplit.

Edit 2:

Once FASTA is split by sequences, the simulation becomes straightforward and many different approaches can be used. My favorite one is using GNU Parallel. Imagine that you have your coverages in a text file called covs.txt on separate lines and in the same order as the sequences in your_file.fa, e.g.,

40
30
...

Then you can simulate reads from the original sequences using DWGsim by

ls -1 seq_* | paste covs.txt - \
    | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}

and merge the obtained FASTQ files using:

cat sim_seq_*.bwa.read1.fastq > reads.1.fq
cat sim_seq_*.bwa.read2.fastq > reads.2.fq

One possible danger of this approach is that we have assumed that the number of seq_* files is the same as the number of lines in covs.txt, which might not be true. We should check this prior to the simulation step, e.g., by:

[[ "$(ls -1 seq_* | wc -l)" == "$(cat covs.txt | wc -l)" ]] \
    || echo "Incorrent number of lines in covs.txt"

Another caveat is that the simulated reads are not in a random order (they are grouped by source sequences).

You can split your FASTA file sequence-by-sequence using

split -a 6 -p '^>' your_file.fa seq_

and then use any existing read simulator supporting coverage (ART, DWGsim, etc.). If you want to have all the reads mixed (not ordered by the original sequence), you can use RNFtools.

Edit 1:

As @terdon pointed out, the previous command works on OS X only. An analogical one liner for Linux (but with a slightly different naming scheme using numbers rather than letters) can be

csplit -f seq_ -n 6 your_file.fa '/^>/' {*}

To make this command work also on OS X, one needs to install coreutils (e.g., using brew) and then use gcsplit instead of csplit.

Edit 2:

Once FASTA is split by sequences, the simulation becomes straightforward and many different approaches can be used. My favorite one is using GNU Parallel. Imagine that you have your coverages in a text file called covs.txt on separate lines and in the same order as the sequences in your_file.fa, e.g.,

40
30
...

Then you can simulate reads from the original sequences using DWGsim by

ls -1 seq_* | paste covs.txt - \
    | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}

and merge the obtained FASTQ files using:

cat sim_seq_*.bwa.read1.fastq > reads.1.fq
cat sim_seq_*.bwa.read2.fastq > reads.2.fq

One possible danger of this approach is that we have assumed that the number of seq_* files is the same as the number of lines in covs.txt, which might not be true (by mistake). We should check this prior to the simulation step, e.g., by:

[[ "$(ls -1 seq_* | wc -l)" == "$(cat covs.txt | wc -l)" ]] \
    || echo "Incorrent number of lines in covs.txt"

Another caveat is that the simulated reads are not in a random order (they are grouped by source sequences).

Typo
Source Link
Karel Břinda
  • 1.9k
  • 10
  • 19

You can split your FASTA file sequence-by-sequence using

split -a 6 -p '^>' your_file.fa seq_

and then use any existing read simulator supporting coverage (ART, DWGsim, etc.). If you want to have all the reads mixed (not ordered by the original sequence), you can use RNFtools.

Edit 1:

As @terdon pointed out, the previous command works on OS X only. An analogical one liner for Linux (but with a slightly different naming scheme using numbers rather than letters) can be

csplit -f seq_ -n 6 your_file.fa '/^>/' {*}

To make this command work also on OS X, one needs to install coreutils (e.g., using brew) and then use gcsplit instead of csplit.

Edit 2:

Once FASTA is split by sequences, the simulation becomes straightforward and many different approaches can be used. My favorite one is using GNU Parallel. Imagine that you have your coverages in a text file called covs.txt on separate lines and in the same order as the sequences in your_file.fa, e.g.,

40
30
...

Then you can simulate reads from the original sequences using DWGsim by

ls -1 seq_* | paste covs.txt - \
    | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}

and merge the obtained FASTQ files using:

cat sim_seq_*.bwa.read1.fastq > reads1reads.1.fq
cat sim_seq_*.bwa.read2.fastq > reads2reads.2.fq

One possible danger of this approach is that we have assumed that the number of seq_* files is the same as the number of lines in covs.txt, which might not be true. We should check this prior to the simulation step, e.g., by:

[[ "$(ls -1 seq_* | wc -l)" == "$(cat covs.txt | wc -l)" ]] \
    || echo "Incorrent number of lines in covs.txt"

Another caveat is that the simulated reads are not in a random order (they are grouped by source sequences).

You can split your FASTA file sequence-by-sequence using

split -a 6 -p '^>' your_file.fa seq_

and then use any existing read simulator supporting coverage (ART, DWGsim, etc.). If you want to have all the reads mixed (not ordered by the original sequence), you can use RNFtools.

Edit 1:

As @terdon pointed out, the previous command works on OS X only. An analogical one liner for Linux (but with a slightly different naming scheme using numbers rather than letters) can be

csplit -f seq_ -n 6 your_file.fa '/^>/' {*}

To make this command work also on OS X, one needs to install coreutils (e.g., using brew) and then use gcsplit instead of csplit.

Edit 2:

Once FASTA is split by sequences, the simulation becomes straightforward and many different approaches can be used. My favorite one is using GNU Parallel. Imagine that you have your coverages in a text file called covs.txt on separate lines and in the same order as the sequences in your_file.fa, e.g.,

40
30
...

Then you can simulate reads from the original sequences using DWGsim by

ls -1 seq_* | paste covs.txt - \
    | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}

and merge the obtained FASTQ files using:

cat sim_seq_*.bwa.read1.fastq > reads1.fq
cat sim_seq_*.bwa.read2.fastq > reads2.fq

One possible danger of this approach is that we have assumed that the number of seq_* files is the same as the number of lines in covs.txt, which might not be true. We should check this prior to the simulation step, e.g., by:

[[ "$(ls -1 seq_* | wc -l)" == "$(cat covs.txt | wc -l)" ]] \
    || echo "Incorrent number of lines in covs.txt"

Another caveat is that the simulated reads are not in a random order (they are grouped by source sequences).

You can split your FASTA file sequence-by-sequence using

split -a 6 -p '^>' your_file.fa seq_

and then use any existing read simulator supporting coverage (ART, DWGsim, etc.). If you want to have all the reads mixed (not ordered by the original sequence), you can use RNFtools.

Edit 1:

As @terdon pointed out, the previous command works on OS X only. An analogical one liner for Linux (but with a slightly different naming scheme using numbers rather than letters) can be

csplit -f seq_ -n 6 your_file.fa '/^>/' {*}

To make this command work also on OS X, one needs to install coreutils (e.g., using brew) and then use gcsplit instead of csplit.

Edit 2:

Once FASTA is split by sequences, the simulation becomes straightforward and many different approaches can be used. My favorite one is using GNU Parallel. Imagine that you have your coverages in a text file called covs.txt on separate lines and in the same order as the sequences in your_file.fa, e.g.,

40
30
...

Then you can simulate reads from the original sequences using DWGsim by

ls -1 seq_* | paste covs.txt - \
    | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}

and merge the obtained FASTQ files using:

cat sim_seq_*.bwa.read1.fastq > reads.1.fq
cat sim_seq_*.bwa.read2.fastq > reads.2.fq

One possible danger of this approach is that we have assumed that the number of seq_* files is the same as the number of lines in covs.txt, which might not be true. We should check this prior to the simulation step, e.g., by:

[[ "$(ls -1 seq_* | wc -l)" == "$(cat covs.txt | wc -l)" ]] \
    || echo "Incorrent number of lines in covs.txt"

Another caveat is that the simulated reads are not in a random order (they are grouped by source sequences).

Typo
Source Link
Karel Břinda
  • 1.9k
  • 10
  • 19

You can split your FASTA file sequence-by-sequence using

split -a 6 -p '^>' your_file.fa seq_

split -a 6 -p '^>' your_file.fa seq_

and then use any existing read simulator supporting coverage (ART, DWGsim, etc.). If you want to have all the reads mixed (not ordered by the original sequence), you can use RNFtools.

Edit 1:

As @terdon pointed out, the previous command works on OS X only. An analogical one liner for Linux (but with a slightly different naming scheme using numbers rather than letters) can be

csplit -f seq_ -n 6 your_file.fa '/^>/' {*}

csplit -f seq_ -n 6 your_file.fa '/^>/' {*}

To make this command work also on OS X, one needs to install coreutils (e.g., using brew) and then use gcsplit instead of csplit.

Edit 2:

Once FASTA is split by sequences, the simulation becomes straightforward and many different approaches can be used. My favorite one is using GNU Parallel. Imagine that you have your coverages in a text file called covs.txt on separate lines and in the same order as the sequences in your_file.fa, e.g.,

40 30 ...

40
30
...

Then you can simulate reads from the original sequences using DWGsim by

ls -1 seq_* | paste covs.txt - | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}

ls -1 seq_* | paste covs.txt - \
    | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}

and merge the obtained FASTQ files using:

cat sim_seq_*.bwa.read1.fastq > reads1.fq cat sim_seq_*.bwa.read2.fastq > reads2.fq

cat sim_seq_*.bwa.read1.fastq > reads1.fq
cat sim_seq_*.bwa.read2.fastq > reads2.fq

AOne possible dangerousdanger of this approach is that we have assumed that the number of seq_* files is the same as the number of lines in covs.txt, which might not be true. We should check this prior to the simulation step, e.g., by:

[[ "$(ls -1 seq_* | wc -l)" == "$(cat covs.txt | wc -l)" ]] || echo "Incorrent number of lines in covs.txt"

[[ "$(ls -1 seq_* | wc -l)" == "$(cat covs.txt | wc -l)" ]] \
    || echo "Incorrent number of lines in covs.txt"

Another caveat is that the simulated reads are not in a random order (they are grouped by source sequences).

You can split your FASTA file sequence-by-sequence using

split -a 6 -p '^>' your_file.fa seq_

and then use any existing read simulator supporting coverage (ART, DWGsim, etc.). If you want to have all the reads mixed (not ordered by the original sequence), you can use RNFtools.

Edit 1:

As @terdon pointed out, the previous command works on OS X only. An analogical one liner for Linux (but with a slightly different naming scheme using numbers rather than letters) can be

csplit -f seq_ -n 6 your_file.fa '/^>/' {*}

To make this command work also on OS X, one needs to install coreutils (e.g., using brew) and then use gcsplit instead of csplit.

Edit 2:

Once FASTA is split by sequences, the simulation becomes straightforward and many different approaches can be used. My favorite one is using GNU Parallel. Imagine that you have your coverages in a text file called covs.txt on separate lines and in the same order as the sequences in your_file.fa, e.g.,

40 30 ...

Then you can simulate reads from the original sequences using DWGsim by

ls -1 seq_* | paste covs.txt - | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}

and merge the obtained FASTQ files using:

cat sim_seq_*.bwa.read1.fastq > reads1.fq cat sim_seq_*.bwa.read2.fastq > reads2.fq

A possible dangerous of this approach is that we have assumed that the number of seq_* files is the same as the number of lines in covs.txt, which might not be true. We should check this prior to the simulation step, e.g., by:

[[ "$(ls -1 seq_* | wc -l)" == "$(cat covs.txt | wc -l)" ]] || echo "Incorrent number of lines in covs.txt"

Another caveat is that the simulated reads are not in a random order (they are grouped by source sequences).

You can split your FASTA file sequence-by-sequence using

split -a 6 -p '^>' your_file.fa seq_

and then use any existing read simulator supporting coverage (ART, DWGsim, etc.). If you want to have all the reads mixed (not ordered by the original sequence), you can use RNFtools.

Edit 1:

As @terdon pointed out, the previous command works on OS X only. An analogical one liner for Linux (but with a slightly different naming scheme using numbers rather than letters) can be

csplit -f seq_ -n 6 your_file.fa '/^>/' {*}

To make this command work also on OS X, one needs to install coreutils (e.g., using brew) and then use gcsplit instead of csplit.

Edit 2:

Once FASTA is split by sequences, the simulation becomes straightforward and many different approaches can be used. My favorite one is using GNU Parallel. Imagine that you have your coverages in a text file called covs.txt on separate lines and in the same order as the sequences in your_file.fa, e.g.,

40
30
...

Then you can simulate reads from the original sequences using DWGsim by

ls -1 seq_* | paste covs.txt - \
    | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}

and merge the obtained FASTQ files using:

cat sim_seq_*.bwa.read1.fastq > reads1.fq
cat sim_seq_*.bwa.read2.fastq > reads2.fq

One possible danger of this approach is that we have assumed that the number of seq_* files is the same as the number of lines in covs.txt, which might not be true. We should check this prior to the simulation step, e.g., by:

[[ "$(ls -1 seq_* | wc -l)" == "$(cat covs.txt | wc -l)" ]] \
    || echo "Incorrent number of lines in covs.txt"

Another caveat is that the simulated reads are not in a random order (they are grouped by source sequences).

Formatting
Source Link
Karel Břinda
  • 1.9k
  • 10
  • 19
Loading
Make splitting work on Linux
Source Link
Karel Břinda
  • 1.9k
  • 10
  • 19
Loading
better names for the output files
Source Link
Karel Břinda
  • 1.9k
  • 10
  • 19
Loading
Source Link
Karel Břinda
  • 1.9k
  • 10
  • 19
Loading