6 votes

Fast processing of fastq data

Knowing a high-performance language will make a huge difference here. See the example below in C. I haven't tested, but it should be easy to modify for your purpose. C++, Rust, Go, Nim and Julia can ...
user172818's user avatar
  • 6,485
6 votes
Accepted

"Sequence Duplication Levels" module still fails after pre-processing Illumina data

To answer your direct question, there are a few reasons why there might be high levels of sequence duplication. From the FastQC help: The underlying assumption of this module is of a diverse ...
Daniel Standage's user avatar
6 votes

"Sequence Duplication Levels" module still fails after pre-processing Illumina data

FastQC assumes that all samples are for whole genome sequencing and will flag them as failed if they differ too much from that assumption. This will, for example, cause essentially all RNA-seq, ChIP-...
Devon Ryan's user avatar
  • 19.6k
5 votes

Is there a read mapper that allows input in SAM format instead of FASTA/FASTQ?

With bwa-mem or minimap2, the recommended way is samtools fastq -T BC,RX name-grouped.sam | bwa mem -C -p - This passes the BC...
user172818's user avatar
  • 6,485
5 votes

Is there a read mapper that allows input in SAM format instead of FASTA/FASTQ?

The common solution for scRNA-seq is to put cell barcodes and such in read headers and then post-process things with UMItools. But regarding your actual question, STAR can accept SAM/BAM as input ...
Devon Ryan's user avatar
  • 19.6k
4 votes

Is there a command line tool to split a SAM/BAM file by CB (cell barcode) tag?

I wrote a python script based on pysam that is free for anyone to use: ...
winni2k's user avatar
  • 2,236
4 votes
Accepted

Edit all the fasta headers using awk

EDIT after OP's update: Try this Perl one-liner: perl -pe 'BEGIN { $i = 1 } $i += s{>([^_]+)_.*_}{>RNA${i}#${1}/}' input_file > output_file Here, the ...
Timur Shtatland's user avatar
4 votes
Accepted

Replacing column 1 in file1 with matching ID's in file 2

I was not sure about the exact output (columns) you desired. The following code should inner merge both data frames. It writes a copy of 'model_list' column as a first column. You could remove any ...
Supertech's user avatar
  • 596
4 votes

Where can I find examples of published bioinformatics pipeline packages?

I think the nextflow core (https://nf-co.re) pipelines should satisfy most of your requirements. These pipelines are written in nextflow (https://nextflow.io) which allows scaling from personal to hpc ...
Niklas's user avatar
  • 106
4 votes

Where can I find examples of published bioinformatics pipeline packages?

The GATK team maintains a GitHub repository with their pipelines: https://github.com/gatk-workflows. They use the Cromwell Workflow Management System, an open source tool created by the Broad ...
terdon's user avatar
  • 9,662
3 votes

Edit all the fasta headers using awk

Your own answer doesn't seem to do what you ask as it doesn't add the sequential identifier to each sequence. Maybe try this? ...
dariober's user avatar
  • 689
3 votes

Appropriate value to replace outliers in clinical studies

In my experience, it is more common to use all clinical data as-is for clinical studies. And if data is missing, either omit the sample or omit the variable with missing data. If your classifier can'...
Reilstein's user avatar
  • 367
3 votes

How to remove degenerate sequence (N4) at the 5' and 3' of the reads?

From the link you posted it looks as though they used UMI tools as per the section which says 'were removed and added to the name of the read as a unique molecular identifier (UMI) using UMI tools'. ...
Graeme's user avatar
  • 181
3 votes
Accepted

What are the right parameters to trim a small RNA transcriptome with trimmomatic?

The command where you trim with adapters and by quality is perfectly fine. That FastQC isn't perfectly happy is expected. It's a tool made for whole genome sequencing QC, you SHOULD see a number of "...
Devon Ryan's user avatar
  • 19.6k
3 votes
Accepted

Inspection of gene expression in scRNA-seq data

In an ideal world, ribosomal RNA (as seen in your top hit) should be excluded from samples prior to sequencing. Where this is not possible (i.e. in the data that have been presented to you), it would ...
gringer's user avatar
  • 13.8k
3 votes
Accepted

Scaling by linear regression against the number of reads

I don't know if this question has been solved already, but what they try to do is equalize the depth of sequencing for each cell. Therefore, they scale for the total number of reads. If you regress ...
DCZ's user avatar
  • 200
2 votes

What is a standard approach to binarize microarray gene expression data?

Based on the info you provide, ArrayBin R package provides you the necessary tools: binarize.array() from ArrayBin, allowing: Implementation of an adaptive ...
aechchiki's user avatar
  • 2,676
2 votes
Accepted

How hard is it to clean and QC gene expression microarray data?

I am not sure how much you know about bioinformatics already, can you use R? For a bioinformatician looking at QC for microarrays should not be a big deal, at least for me it would take maybe a day (...
benn's user avatar
  • 3,571
2 votes

Edit all the fasta headers using awk

sed -e 's:^\(.*\)_.*_\(.*\)$:\1/\2:' < input > output
BioInfo's user avatar
  • 374
2 votes

Filter rows of VCF file for Match=EXACT?

You can filter records in Python for example using pyvcf.VcfFrame.filter_flagany() method I wrote: ...
Seung-been Steven Lee's user avatar
2 votes
Accepted

replacing periods with hyphen

In A2M format, upper case letters represent matches, lower case letters represent inserts, dashes represent deletions, and dots (or spaces outside the identifier lines) represent gaps aligned to ...
Stas Malavin's user avatar
2 votes

Replacing column 1 in file1 with matching ID's in file 2

The basic strategy of merge 'left' (which I think is default) is the correct thing to do. However, I would be concerned by lack of a key between the two data sets ...
M__'s user avatar
  • 11.9k
2 votes

Integrating bulk RNA-Seq data with different sequencing depths and from different sources

Per-sample coverage / count differences are expected in all differential expression analysis, and correction for that is built into DESeq2. You shouldn't need to do any additional correction with ...
gringer's user avatar
  • 13.8k
2 votes

How can I delete these lines in my fastq file?

This is not the thing I do these days but ... gunzip -c file.fastq.gz | sed "45500973,45500974d" > new.fastq You were using ...
M__'s user avatar
  • 11.9k
1 vote

Dictionary and index of vcf for base recalibration step

I have also encountered this error before in cases where there was something wrong with the VCF file I was supplying. Double check your command to ensure that you are feeding it the file you believed ...
Michael Weinstein's user avatar
1 vote
Accepted

How to Read SCF file in Python?

It was done using sangerseqR with rpy2 in python ...
alex3465's user avatar
  • 151
1 vote

Filter rows of VCF file for Match=EXACT?

Hard to say without seeing VCF rows, but generally speaking with text files you can use grep, if I understand the question: ...
Maximilian Press's user avatar
1 vote
Accepted

How to extract column from a matrix matching the another file (sample file)

Assuming your "master" files looks like this: ...
dc37's user avatar
  • 1,021
1 vote
Accepted

Print specific columns in a matrix on the basis of sample id's in the header

Example in R. If you have a file of column names, names.txt, such as: mpg cyl disp and a data.frame like ...
user438383's user avatar
  • 1,679

Only top scored, non community-wiki answers of a minimum length are eligible