How to simulate NGS reads, controlling sequence coverage?

Question

I have a FASTA file with 100+ sequences like this:

>Sequence1
GTGCCTATTGCTACTAAAA ...
>Sequence2
GCAATGCAAGGAAGTGATGGCGGAAATAGCGTTA
......

I also have a text file like this:

Sequence1 40
Sequence2 30
......

I would like to simulate next-generation paired-end reads for all the sequences in my FASTA file. For Sequence1, I would like to simulate at 40x coverage. For Sequence2, I would like to simulate at 30x coverage. In other words, I want to control my sequence coverage for each sequence in my simulation.

Q: What is the simplest way to do that? Any software I should use? Bioconductor?

Answer 1

I am not aware of any software that can do this directly, but I would split the fasta file into one sequence per file, loop over them in BASH and invoke ART the sequence simulator (or another) on each sequence.

For more information about ART, please see their paper here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3278762/

Answer 2

I am working on a Illumina sequencing simulator for metagenomics: InSilicoSeq

It is still in alpha release and very experimental, but given a multi-fasta and an abundance file, it will generate reads from your input genomes with different coverages.

From the documentation:

iss generate --genomes genomes.fasta --abundance abundance_file.txt \
    --model_file HiSeq2500 --output HiSeq_reads

Where:

# multi-fasta file
>genome_A
ATGC...
>genome_B
CCGT...
...

# abundance file (total abundance must be 1!)
genome_A    0.2
genome_B    0.4
...

I didn't design it to work with coverage but rather abundance of the genome in a metagenome, so you might have to do a tiny bit of math ;)

Answer 3

The polyester bioconductor package can do this. It says it simulates RNA-seq reads, but I don't know if that's really any different from other NGS reads.

It can use a range of error and bias models, or learn them from a dataset.

Answer 4

The wgsim package by Heng Li (of BWA and samtools fame) is my go-to tool for simulating Illumina reads. It doesn't provide any convenient way to simulate differential coverage across different sequences, but it shouldn't be to hard to run wgsim multiple times, generating the desired level of coverage for each sequence of interest.

I would implement a Python script to slurp up your test file, and call wgsim (using the subprocess module) for each sequence. This will probably require you to have each sequence in a separate file. :-(

Answer 5

This python script takes a fasta file and tsv file with counts and prints the sequences in the fasta files that many times as it is specified in the tsv file (assuming the format in the question). So if bar.tsv and foo.fasta will be your files:

from Bio import SeqIO

repeat = {}
for line in open("bar.tsv"):
    seq_id, coverage = line.split()
    repeat[seq_id] = int(coverage)

for seq_record in SeqIO.parse(foo.fasta, "fasta"):
    for i in range(repeat.get(seq_record.name, 0)):
        print(">",seq_record.name,"_",i,sep='')
        print(seq_record.seq)

Answer 6

If you're ok with some randomness you can generate reads from your sequence file using a Poisson random variable. You'll need to do some math to figure out what value of lambda to use in order for the expected coverage at each base pair in your read to match what you set in your text file.

For example you have a sequence S of length 1,000, a read length of 50, and an insert size of 100. For each base b in S generate a Poisson random variable p. You will then generate p reads from base b to b+50. Then, generate the paired read starting at b+50+100.

Again, you would have to play with it to figure out what lambda to use but this would give you basically what you want, as long as you're ok with not having exactly the coverage you're targeting for each read.

Answer 7

Simulating NGS reads while controlling sequence coverage is now easy with RNFtools (from version 0.3.1). See the tutorial, especially section Sequence extraction.

Environment preparation

First, install BioConda and add the required channels. Then either install RNFtools in the default Conda environment

conda install rnftools

or create and activate a separate Conda environment (preferable)

conda create -n rnftools rnftools
source activate rnftools

Simulation

Assume that you have a reference file ref.fa and a tab-separated coverage file coverage.tsv (e.g., those from your example). Then the following RNFtools Snakefile will do the job you want:

import rnftools
import csv

rnftools.mishmash.sample("simulation_with_coverage_control", reads_in_tuple=1)

fa = "ref.fa"
tsv = "coverage.tsv"

with open(tsv) as f:
    table = csv.reader(f, delimiter='\t')
    for seqname, cov in table:

        rnftools.mishmash.DwgSim(
            fasta=fa,
            sequences=[seqname],
            coverage=float(cov),
            read_length_1=10, # quick test with supershort reads
            read_length_2=0,
        )

include: rnftools.include()
rule: input: rnftools.input()

When you save this file (Snakefile) and run snakemake, RNFtools will simulate reads using DWGsim with the coverages defined your text file, and save all the simulated reads in simulation_with_coverage_control.fq.

You can play with all the parameters. In particular, you can use a different simulator (e.g., Art-Illumina using rnftools.mishmash.ArtIllumina). See the RNFtools documentation for more information.

Answer 8

Another next-gen read simulation tool is gemsim. I haven't tested it, but I would be interested if anyone has had any experience with it.

Answer 9

Go to ART website. Download your package, binary or source: Uncompress the .gz file, open the folder "art_bin_MountRainier", then "examples". Open the text file "run_test_examples_illumina.sh". This file contains numerous example of samples codes. That will give you an idea on how you can use the software. Here is an example:

# 4) simulation of paired-end reads of 100bp with the mean fragment size 500 and standard deviation 10
#   using the built-in seperated quality profiles for A, C, G, and T 
$art -ss HS20 -i ./testSeq.fa -o ./paired_end_sep -l 100 -f 10 -p -m 500 -s 10 -sp -sam
#convert both aln files to a bed file
../aln2bed.pl paired_end_sep.bed paired_end_sep1.aln paired_end_sep2.aln

Answer 10

You can split your FASTA file sequence-by-sequence using

split -a 6 -p '^>' your_file.fa seq_

and then use any existing read simulator supporting coverage (ART, DWGsim, etc.). If you want to have all the reads mixed (not ordered by the original sequence), you can use RNFtools.

Edit 1:

As @terdon pointed out, the previous command works on OS X only. An analogical one liner for Linux (but with a slightly different naming scheme using numbers rather than letters) can be

csplit -f seq_ -n 6 your_file.fa '/^>/' {*}

To make this command work also on OS X, one needs to install coreutils (e.g., using brew) and then use gcsplit instead of csplit.

Edit 2:

Once FASTA is split by sequences, the simulation becomes straightforward and many different approaches can be used. My favorite one is using GNU Parallel. Imagine that you have your coverages in a text file called covs.txt on separate lines and in the same order as the sequences in your_file.fa, e.g.,

40
30
...

Then you can simulate reads from the original sequences using DWGsim by

ls -1 seq_* | paste covs.txt - \
    | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}

and merge the obtained FASTQ files using:

cat sim_seq_*.bwa.read1.fastq > reads.1.fq
cat sim_seq_*.bwa.read2.fastq > reads.2.fq

One possible danger of this approach is that we have assumed that the number of seq_* files is the same as the number of lines in covs.txt, which might not be true (by mistake). We should check this prior to the simulation step, e.g., by:

[[ "$(ls -1 seq_* | wc -l)" == "$(cat covs.txt | wc -l)" ]] \
    || echo "Incorrent number of lines in covs.txt"

Another caveat is that the simulated reads are not in a random order (they are grouped by source sequences).