Questions tagged [benchmarking]
Benchmarking problems relate to problems where efficiency is key, beyond just solving a problem itself. What's the fastest or most efficient way to solve a problem?
17 questions
3
votes
0
answers
53
views
Improving DNA Sequence Shuffling Code for Speed and Accuracy
I'm working on a Python script that shuffles DNA sequences while preserving their protein-coding potential. My goal is to adjust the codon usage in such a way that the ACGT (adenine, cytosine, guanine,...
- 31
2
votes
0
answers
40
views
Converting Plink .bed files to .bcf *with* an AC (allele count) field?
I have some extremely large (N ~ 400,000) Plink .bed files that I want to phase using shapeit5.
Shapeit5 requires input data be in .bcf or .vcf format, and that it have an "AC" (allele count)...
- 225
0
votes
1
answer
23
views
Assessing Quality of Error Correction
The main idea:
Given fastq/fasta files containing reads(simulated from ref. genome, with errors introduced) and another file with the ground truth(simulated but without errors) and then the resulting ...
- 1
3
votes
3
answers
139
views
Introducing a fixed number of random substitutions in a sequence
I'm writing a function that introduces $n$ substitutions inside a sequence of nucleotides.
I have a working version, but I'm looking for any other ways of doing this:
...
- 193
1
vote
0
answers
64
views
Benchmarking for variant identification using RNA-seq data
I am in need to benchmark the variant identification pipeline which uses RNA seq data alone without any matched-normal. I would like to know the reference dataset (and the pipeline on which the ...
- 11
1
vote
1
answer
85
views
Refactoring a script mapping ID to sequence
I wrote simple Python script which has an excessive run time, notably when using large data sets. For example a data set of 1 000 000 sequences.
I am seeking assistance refactoring the code and would ...
- 147
1
vote
2
answers
59
views
Improving list speed: glycosylation example
I discussed a question with @gaspanic Python/Biopython - Replace amino acid residue on MSA with "z" from a list of unaligned positions
. The issue emerged was speeding up lists in Python. ...
M__♦
- 13k
5
votes
2
answers
5k
views
Samtools sort: most efficient memory and thread settings for many samples on a cluster
We're preparing to analyze thousands of .bam files beginning with re-alignment, sorting, etc.
Complicated question: Has anyone investigated the optimal thread and ...
- 1,414
1
vote
1
answer
59
views
Challenging benchmarks for supervised learning on sparse scRNA-seq data
One challenging aspect of modeling scRNA-seq data is data sparsity, that is, scRNA-seq measurements typically suffer from large fractions of observed zeros (i.e. dropouts), where a given gene in a ...
- 123
3
votes
2
answers
91
views
Gold standard benchmark
This page is claimed to contain a gold standard benchmark for viral genome assembly.
https://github.com/cbg-ethz/5-virus-mix
The claim is here:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5411778/
...
- 359
9
votes
10
answers
25k
views
Remove/delete sequences by ID from multifasta
I have a fasta file like this:
>Id1
ATCCTT
>Id2
ATTTTCCC
>Id3
TTTCCCCAAAA
>Id4
CCCTTTAAA
I want to delete sequences that have the following IDs.
<...
- 385
2
votes
1
answer
112
views
efficient counting of dinucleotides/trinucleotides on fastq reads?
What's an efficient way of counting of dinucleotides/trinucleotide pattern on fastq.gz file reads?
I know there are tools like seqtk that will be very efficient at reading through the .fastq.gz files,...
- 2,364
21
votes
8
answers
10k
views
What is the fastest way to get the reverse complement of a DNA sequence in python?
I am writing a python script that requires a reverse complement function to be called on DNA strings of length 1 through around length 30. Line profiling programs indicate that my functions spend a ...
- 3,201
13
votes
8
answers
17k
views
Fast way to count number of reads and number of bases in a fastq file?
I am looking for a tool, preferably written in C or C++, that can quickly and efficiently count the number of reads and the number of bases in a compressed fastq file. I am currently doing this using <...
- 10.6k
21
votes
12
answers
2k
views
Random access on a FASTQ file
I would like to select a random record from a large set of n unaligned sequencing reads in log(n) time complexity (big O ...
- 2,286
13
votes
4
answers
792
views
How do I efficiently subset a very large line-based file?
This has come up repeatedly recently: I have a very large text file (in the order of several GiB) and I need to perform line-based subsetting for around 10,000 lines. There exist solutions for ...
- 4,895
22
votes
10
answers
7k
views
What is the fastest way to calculate the number of unknown nucleotides in FASTA / FASTQ files?
I used to work with publicly available genomic references, where basic statistics are usually available and if they are not, you have to compute them only once so there is no reason to worry about ...
- 5,652