6
votes
Fast processing of fastq data
Knowing a high-performance language will make a huge difference here. See the example below in C. I haven't tested, but it should be easy to modify for your purpose.
C++, Rust, Go, Nim and Julia can ...
- 6,285
6
votes
Accepted
"Sequence Duplication Levels" module still fails after pre-processing Illumina data
To answer your direct question, there are a few reasons why there might be high levels of sequence duplication. From the FastQC help:
The underlying assumption of this module is of a diverse ...
- 5,010
6
votes
"Sequence Duplication Levels" module still fails after pre-processing Illumina data
FastQC assumes that all samples are for whole genome sequencing and will flag them as failed if they differ too much from that assumption. This will, for example, cause essentially all RNA-seq, ChIP-...
- 19.4k
5
votes
Is there a read mapper that allows input in SAM format instead of FASTA/FASTQ?
The common solution for scRNA-seq is to put cell barcodes and such in read headers and then post-process things with UMItools.
But regarding your actual question, STAR can accept SAM/BAM as input ...
- 19.4k
4
votes
Is there a read mapper that allows input in SAM format instead of FASTA/FASTQ?
With bwa-mem or minimap2, the recommended way is
samtools fastq -T BC,RX name-grouped.sam | bwa mem -C -p -
This passes the BC...
- 6,285
4
votes
Accepted
Edit all the fasta headers using awk
EDIT after OP's update:
Try this Perl one-liner:
perl -pe 'BEGIN { $i = 1 } $i += s{>([^_]+)_.*_}{>RNA${i}#${1}/}' input_file > output_file
Here, the ...
- 755
4
votes
Is there a command line tool to split a SAM/BAM file by CB (cell barcode) tag?
I wrote a python script based on pysam that is free for anyone to use:
...
- 2,166
4
votes
Accepted
Replacing column 1 in file1 with matching ID's in file 2
I was not sure about the exact output (columns) you desired. The following code should inner merge both data frames. It writes a copy of 'model_list' column as a first column. You could remove any ...
- 534
4
votes
Where can I find examples of published bioinformatics pipeline packages?
I think the nextflow core (https://nf-co.re) pipelines should satisfy most of your requirements. These pipelines are written in nextflow (https://nextflow.io) which allows scaling from personal to hpc ...
- 51
4
votes
Where can I find examples of published bioinformatics pipeline packages?
The GATK team maintains a GitHub repository with their pipelines: https://github.com/gatk-workflows. They use the Cromwell Workflow Management System, an open source tool created by the Broad ...
- 9,352
3
votes
Edit all the fasta headers using awk
Your own answer doesn't seem to do what you ask as it doesn't add the sequential identifier to each sequence. Maybe try this?
...
- 689
3
votes
Appropriate value to replace outliers in clinical studies
In my experience, it is more common to use all clinical data as-is for clinical studies. And if data is missing, either omit the sample or omit the variable with missing data.
If your classifier can'...
- 367
3
votes
How to remove degenerate sequence (N4) at the 5' and 3' of the reads?
From the link you posted it looks as though they used UMI tools as per the section which says
'were removed and added to the name of the read as a unique molecular identifier (UMI) using UMI tools'. ...
- 181
3
votes
Accepted
What are the right parameters to trim a small RNA transcriptome with trimmomatic?
The command where you trim with adapters and by quality is perfectly fine. That FastQC isn't perfectly happy is expected. It's a tool made for whole genome sequencing QC, you SHOULD see a number of "...
- 19.4k
3
votes
Accepted
Inspection of gene expression in scRNA-seq data
In an ideal world, ribosomal RNA (as seen in your top hit) should be excluded from samples prior to sequencing. Where this is not possible (i.e. in the data that have been presented to you), it would ...
- 13k
3
votes
Accepted
Scaling by linear regression against the number of reads
I don't know if this question has been solved already, but what they try to do is equalize the depth of sequencing for each cell. Therefore, they scale for the total number of reads. If you regress ...
- 200
2
votes
What is a standard approach to binarize microarray gene expression data?
Based on the info you provide, ArrayBin R package provides you the necessary tools:
binarize.array() from ArrayBin, allowing:
Implementation of an adaptive ...
- 2,656
2
votes
Accepted
How hard is it to clean and QC gene expression microarray data?
I am not sure how much you know about bioinformatics already, can you use R? For a bioinformatician looking at QC for microarrays should not be a big deal, at least for me it would take maybe a day (...
- 3,561
2
votes
Filter rows of VCF file for Match=EXACT?
You can filter records in Python for example using pyvcf.VcfFrame.filter_flagany() method I wrote:
...
2
votes
2
votes
Accepted
replacing periods with hyphen
In A2M format, upper case letters represent matches, lower case letters represent inserts, dashes represent deletions, and dots (or spaces outside the identifier lines) represent gaps aligned to ...
- 56
2
votes
Replacing column 1 in file1 with matching ID's in file 2
The basic strategy of merge 'left' (which I think is default) is the correct thing to do. However, I would be concerned by lack of a key between the two data sets ...
M__♦
- 10.2k
2
votes
How can I delete these lines in my fastq file?
This is not the thing I do these days but ...
gunzip -c file.fastq.gz | sed "45500973,45500974d" > new.fastq
You were using ...
M__♦
- 10.2k
1
vote
Reading in external single cell data
It looks like it is 10X data and I strongly suggest a dedicated single cell analysis package such as Seurat. You will most likely end up using one of these anyway for data analysis.
Read10X() can be a ...
- 3,672
1
vote
How to run trimmomatic in HPC
Depending on the cluster management tool, you might have received e-mails when the "job" begins and ends. If so, you can check the "Exit status" of the job. For example, in the case of our HPC the ...
- 3,672
1
vote
Dictionary and index of vcf for base recalibration step
I have also encountered this error before in cases where there was something wrong with the VCF file I was supplying.
Double check your command to ensure that you are feeding it the file you believed ...
1
vote
Accepted
1
vote
Appropriate value to replace outliers in clinical studies
Removing outliers is common practice in statistical modeling and perfectly acceptable. However, with regards 1.5 IQR I am far from certain about this approach. Normally, if you want to be conservative ...
M__♦
- 10.2k
1
vote
Filter rows of VCF file for Match=EXACT?
Hard to say without seeing VCF rows, but generally speaking with text files you can use grep, if I understand the question:
...
- 3,409
Only top scored, non community-wiki answers of a minimum length are eligible