I am trying to check my variant generation pipeline for humans. The pipeline is for NGS. I am searching for any dataset(.fastq+.bam+.vcf) for any specific sample. Now, GIAB has a nice dataset for that. However, the problem is that it's not a tiny dataset. And it's not feasible to test my pipeline with this data every time. I tried to generate a synthetic dataset with some read simulators(NEAT, MASON2). However, their aligned read files(.bam) files seem not very good. You can check my Issue for NEAT in link. So, my question is, Is there any publicly available tiny dataset(.fastq,.bam, .vcf) with which I can validate my pipeline fast? Or, is there any simulator with which I can simulate .bam, .fastq files from a given input .vcf? I am using fastqc,bwa,samtools,GATK4 mainly and GRCH38 reference in my pipeline.
I would suggest subsampling an existing dataset such as the GIAB resource that you've already indicated.
Use a utility such as seqtk to subsample the appropriate number of reads from the dataset, and use that.
seqtk to get 1 million reads each from paired fastqs while preserving pairing:
# note -s100 random seed which should be used for both to keep pairs
seqtk sample -s100 giab_read1.fq 1000000 > giab_sub_r1.fq
seqtk sample -s100 giab_read2.fq 1000000 > giab_sub_r2.fq