9 votes

Delete all 4 lines of a fastq read from a fastq file using read ID

Don't do it: Your FASTQ file is either malformed or your FASTQ record spans more than four lines, which is allowed in FASTQ. For a detailed description of what can go wrong in FASTQ parsing see for ...
winni2k's user avatar
  • 2,256
6 votes

Fast processing of fastq data

Knowing a high-performance language will make a huge difference here. See the example below in C. I haven't tested, but it should be easy to modify for your purpose. C++, Rust, Go, Nim and Julia can ...
user172818's user avatar
  • 6,525
6 votes
Accepted

Concatenate multiple fasta files into one file per unique entry

You can do this simply enough in awk: ...
terdon's user avatar
  • 9,846
6 votes

Delete all 4 lines of a fastq read from a fastq file using read ID

If you are 100% sure the read only has 4 lines (they can have more), you can use this sed command: ...
terdon's user avatar
  • 9,846
5 votes
Accepted

How do I re-name the headers of my Fasta file?

If awk is good as well: ...
finswimmer's user avatar
  • 1,342
4 votes
Accepted

How can I easily get the read size distribution of reads mapping on a certain set of regions?

This can be done rather simply by combining this samtools based answer and bioawk: ...
bli's user avatar
  • 3,130
4 votes
Accepted

Intersecting two different files with one "master" file based on different columns

An awk solution ...
glenn jackman's user avatar
4 votes
Accepted

Remove repeated ALT allele from the REF field of a vcf-like file

I am making a couple of assumptions here: the delimiter is tab the reference allele is always present in the ALT column the reference allele in the ALT column is always in the first position With ...
vkkodali's user avatar
  • 1,266
4 votes

Identify non-coding regions from a genome annotation

Getting the non coding regions of a protein coding transcript, sounds like you are looking for UTR. UTR has its own feature in the gtf file. So you can do this: <...
finswimmer's user avatar
  • 1,342
3 votes

How do I re-name the headers of my Fasta file?

It seems like you want to do four things: Remove all text up to and including the first |. Now that that has been removed, take the string between what is now (...
terdon's user avatar
  • 9,846
3 votes

Identify non-coding regions from a genome annotation

This isn't a problem that's easily solved with awk. It's not like you're extracting a feature that's annotated in the GTF file. Instead, you want the empty space between annotated features. A few ...
Daniel Standage's user avatar
3 votes
Accepted

Parsing gtf file for transcript ID and transcript name

I don't see how your awk command would work since you're using whitespace as a field delimiter. In any case, you can use a much shorter perl command for this: <...
terdon's user avatar
  • 9,846
3 votes

Delete all 4 lines of a fastq read from a fastq file using read ID

When I can I like to do file processing line-by-line in the spirit of UNIX tools. You can read 4 lines from a Fastq file into 4 tab-separated values on a single line using ...
Daniel Standage's user avatar
3 votes
Accepted

Build a Pubmed query given long gene list

After getting it out on paper (so to speak) I was able to accomplish what I wished with bash: ...
Kohl Kinning's user avatar
  • 1,149
3 votes

Using column 2 of one file to match with two columns of another file, and append

Pipe a modified form of the second file to BEDOPS bedmap and the first file, then pipe that result to cut out desired columns: ...
Alex Reynolds's user avatar
3 votes
Accepted

separate genes with expression values

What do you want to do with NA values? The below scripts ignore them (h3ab74's answer will print them to positiveExpression) ...
Greg's user avatar
  • 301
3 votes
Accepted

How can I obtain a list of all NCBI gene ID's along with their full name, symbol, and also known as symbols?

All of these data are in the gene_info.gz file on the NCBI FTP site: https://ftp.ncbi.nlm.nih.gov/gene/DATA/ Specifically, you may want to look at the ...
vkkodali's user avatar
  • 1,266
3 votes
Accepted

eQTL data extraction from database JSON file

Here a jq solution if you can use it: ...
Arnaud Valmary's user avatar
3 votes
Accepted

How can I change all text in a .tsv file to 1, except the first three fields?

Not an expert, but : awk 'BEGIN { FS = OFS = "\t" } { for(i=4; i<=6; i++) if($i ~ /^ *$/) {$i = 0} else {$i = 1} }; 1' Since you have a very specific ...
Sam's user avatar
  • 149
2 votes

replacing SNPs with missing calls with a specific string

A possible approach in plain python: ...
bli's user avatar
  • 3,130
2 votes
Accepted

Splitting characters of a column in repeated lines and reducing them to one line

OK. This is a bit convoluted, but it works. There might be simpler solutions... First we'll write a function that takes a data frame and a character position (ie; first or second position in the ...
heathobrien's user avatar
  • 1,816
2 votes

Counting a specific consecutive character with its occurrence position and length

I couldn't come up with a one liner but have a two liner instead... To get the start positions of the N repeat: ...
benn's user avatar
  • 3,571
2 votes
Accepted

How to separate a microbial strain of MTB resistant and susceptible drugwise?

Since this involves combining multiple conditions, it is easier (and certainly more readable if you ever need to come back to it later) to write a script to do this. For example, in perl: ...
terdon's user avatar
  • 9,846
2 votes

Identify non-coding regions from a genome annotation

If you want all transcripts from that gtf file whose type isn't "protein_coding", you can use almost the same command, just change the == ("is") to ...
terdon's user avatar
  • 9,846
2 votes

How to merge two files on the first 3 columns?

You can use the same approach as for two fields, just add the third: awk 'NR==FNR{a[$1,$2,$3]=$4;next} ($1,$2,$3) in a{print $0, a[$1,$2,$3]}' ret lcls
terdon's user avatar
  • 9,846
2 votes
Accepted

how to merge two fasta based on full fasta header and remove the duplicate

You can use the FastaToTbl and TblToFasta scripts I have posted here for this: ...
terdon's user avatar
  • 9,846
2 votes

Remove repeated ALT allele from the REF field of a vcf-like file

Another awk approach. This should work on a normal VCF file (it will print the headers) as well as your example and only changes the ALT field if its first character is the same as the 2nd field: <...
terdon's user avatar
  • 9,846
1 vote

Parsing a GenBank file with multiple gene entries

You can do this very easily with awk: ...
terdon's user avatar
  • 9,846
1 vote
Accepted

Parsing a GenBank file with multiple gene entries

You can simply use grep for this purpose as shown below, grep -v /translation bio.txt | grep -B100000000 /db_xref= > output_file.txt Just make sure that you ...
Ammar Sabir Cheema's user avatar
1 vote
Accepted

Modifying .vcf files

I am pretty sure that there is a reason that the first nucleotide of the ALT is the same as that of REF in ...
haci's user avatar
  • 4,032

Only top scored, non community-wiki answers of a minimum length are eligible