Skip to main content
added 58 characters in body
Source Link
Nikita Vlasenko
  • 2.6k
  • 3
  • 27
  • 38

So, I guess the file is corrupted and the issue is with some library that is generating the file. However, reinstalling manually other libraries do not help.

So, I guess the file is corrupted and the issue is with some library that is generating the file.

So, I guess the file is corrupted and the issue is with some library that is generating the file. However, reinstalling manually other libraries do not help.

added 260 characters in body
Source Link
Nikita Vlasenko
  • 2.6k
  • 3
  • 27
  • 38

Edit

After running manually gzip -t on a file:

-bash-4.2$ gzip -t old-lec_1_r1.fq.gz 
gzip: old-lec_1_r1.fq.gz: unexpected end of file

So, I guess the file is corrupted and the issue is with some library that is generating the file.

Edit

After running manually gzip -t on a file:

-bash-4.2$ gzip -t old-lec_1_r1.fq.gz 
gzip: old-lec_1_r1.fq.gz: unexpected end of file

So, I guess the file is corrupted and the issue is with some library that is generating the file.

added commands that were run on the cluster
Source Link
Nikita Vlasenko
  • 2.6k
  • 3
  • 27
  • 38

Edit The command I am running is the following one:

python "$path/salmon_rna_seq.py" -L 'IU' -E "$path/jyvo_experiment-
metadata.yaml" -T 'gencode_mouse_m13' -C "$path/salmon_rna_seq.yaml" -I 
"$path_to_fastq/$name-lec_1_r1.fq.gz" "$path_to_fastq/$name-
lec_1_r2.fq.gz" -O "$path/res/$name-lec_1" -S "$name-lec_1" -P '6' -X 'True'

The command is run by Pypiper: https://github.com/epigen/pypiper Inside of salmon_rna_seq.py I am also running commands to treat the data, the salmon part looks the following way:

def salmon(self):
    cmd = "salmon quant -i " + self.indexed_transcriptome
    cmd += " -l " + self.lib_type

    if self.trim:
        cmd += " -1 " + self.sample_files['r1']['fastq']['trimmed_paired']
        cmd += " -2 " + self.sample_files['r2']['fastq']['trimmed_paired']
    else:
        cmd += " -1 " + self.sample_files['r1']['fastq']['chastity']
        cmd += " -2 " + self.sample_files['r2']['fastq']['chastity']

    cmd += " -o " + self.sample_dirs['salmon']['base']
    cmd += " --numBootstraps=30"
self.pipe_manager.run(cmd, self.sample_files['salmon'])

Commands that the Pypiper runs are the following ones (_commands.sh Pypiper file):

zcat -c $PATH/res/old-lec_1/fastq/merged/old-lec_1_r1.fq.gz | grep -A 3 
'^@.*[^:]*:N:[^:]*:' | grep -v '^\-\-$' | gzip -c > $PATH/res/old-lec_1/fastq
/chastity/old-lec_1_r1.fq.gz

zcat -c $PATH/res/old-lec_1/fastq/merged/old-lec_1_r2.fq.gz | grep -A 3 
'^@.*[^:]*:N:[^:]*:' | grep -v '^\-\-$' | gzip -c > $PATH/res/old-lec_1/fastq
/chastity/old-lec_1_r2.fq.gz

java -jar $HOME/anaconda2/share/trimmomatic-0.36-5
/trimmomatic.jar PE -threads 6 -phred33 $PATH/res/old-lec_1/fastq/chastity/old-
lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r2.fq.gz $PATH/res/old-
lec_1/fastq/trimmed/old-lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/trimmed/unpaired
/old-lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r2.fq.gz $PATH/res/old-
lec_1/fastq/trimmed/unpaired/old-lec_1_r2.fq.gz ILLUMINACLIP:$HOME/anaconda2/share/trimmomatic-0.36-5/adapters
/Small_List_Of_Adapters.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15
MINLEN:36

fastqc $PATH/res/old-lec_1/fastq/merged/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/merged

fastqc $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/chastity

fastqc $PATH/res/old-lec_1/fastq/merged/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/merged

fastqc $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/chastity

fastqc $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/trimmed

fastqc $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/trimmed

salmon quant -i $HOME/Genomes/mouse/salmon
/gencode.vM13.transcripts.fa.sidx -l IU -1 $PATH/res/old-lec_1/fastq/trimmed
/old-lec_1_r1.fq.gz -2 $PATH/res/old-lec_1/fastq/trimmed/old-
lec_1_r2.fq.gz -o $PATH/res/old-lec_1/quantification --numBootstraps=30

Edit The command I am running is the following one:

python "$path/salmon_rna_seq.py" -L 'IU' -E "$path/jyvo_experiment-
metadata.yaml" -T 'gencode_mouse_m13' -C "$path/salmon_rna_seq.yaml" -I 
"$path_to_fastq/$name-lec_1_r1.fq.gz" "$path_to_fastq/$name-
lec_1_r2.fq.gz" -O "$path/res/$name-lec_1" -S "$name-lec_1" -P '6' -X 'True'

The command is run by Pypiper: https://github.com/epigen/pypiper Inside of salmon_rna_seq.py I am also running commands to treat the data, the salmon part looks the following way:

def salmon(self):
    cmd = "salmon quant -i " + self.indexed_transcriptome
    cmd += " -l " + self.lib_type

    if self.trim:
        cmd += " -1 " + self.sample_files['r1']['fastq']['trimmed_paired']
        cmd += " -2 " + self.sample_files['r2']['fastq']['trimmed_paired']
    else:
        cmd += " -1 " + self.sample_files['r1']['fastq']['chastity']
        cmd += " -2 " + self.sample_files['r2']['fastq']['chastity']

    cmd += " -o " + self.sample_dirs['salmon']['base']
    cmd += " --numBootstraps=30"
self.pipe_manager.run(cmd, self.sample_files['salmon'])

Commands that the Pypiper runs are the following ones (_commands.sh Pypiper file):

zcat -c $PATH/res/old-lec_1/fastq/merged/old-lec_1_r1.fq.gz | grep -A 3 
'^@.*[^:]*:N:[^:]*:' | grep -v '^\-\-$' | gzip -c > $PATH/res/old-lec_1/fastq
/chastity/old-lec_1_r1.fq.gz

zcat -c $PATH/res/old-lec_1/fastq/merged/old-lec_1_r2.fq.gz | grep -A 3 
'^@.*[^:]*:N:[^:]*:' | grep -v '^\-\-$' | gzip -c > $PATH/res/old-lec_1/fastq
/chastity/old-lec_1_r2.fq.gz

java -jar $HOME/anaconda2/share/trimmomatic-0.36-5
/trimmomatic.jar PE -threads 6 -phred33 $PATH/res/old-lec_1/fastq/chastity/old-
lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r2.fq.gz $PATH/res/old-
lec_1/fastq/trimmed/old-lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/trimmed/unpaired
/old-lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r2.fq.gz $PATH/res/old-
lec_1/fastq/trimmed/unpaired/old-lec_1_r2.fq.gz ILLUMINACLIP:$HOME/anaconda2/share/trimmomatic-0.36-5/adapters
/Small_List_Of_Adapters.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15
MINLEN:36

fastqc $PATH/res/old-lec_1/fastq/merged/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/merged

fastqc $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/chastity

fastqc $PATH/res/old-lec_1/fastq/merged/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/merged

fastqc $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/chastity

fastqc $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r1.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/trimmed

fastqc $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r2.fq.gz --outdir $PATH/res/old-
lec_1/fastqc/trimmed

salmon quant -i $HOME/Genomes/mouse/salmon
/gencode.vM13.transcripts.fa.sidx -l IU -1 $PATH/res/old-lec_1/fastq/trimmed
/old-lec_1_r1.fq.gz -2 $PATH/res/old-lec_1/fastq/trimmed/old-
lec_1_r2.fq.gz -o $PATH/res/old-lec_1/quantification --numBootstraps=30
added tags
Link
Loading
updated with the new things that I tried to fix the issue
Source Link
Nikita Vlasenko
  • 2.6k
  • 3
  • 27
  • 38
Loading
Source Link
Nikita Vlasenko
  • 2.6k
  • 3
  • 27
  • 38
Loading