2
$\begingroup$

I am a beginner in snakemake and I'm trying to develop a working flow.

Imagine I have multiple folders and in each folder I have a tumour and normal sample files:

SRC120/ 
      SRC120-T1.fastq
      SRC120-N1.fastq

SRC121/ 
      SRC121-T1.fastq
      SRC121-N1.fastq

and so on (in total I have more than 100 SRC folders).

I am trying to make a snakemake rule to make a symlink of these fastq files.

What I have written so far is:

import os

SAMPLES=os.listdir("/analysis/Anna/snakemake-demo/data/samples_fastq-test1/")                                               

def getFastqFilesForTissu(wildcards):
    fastqs = list()
    # Beware no other files than fastqs should be there
    for s in os.listdir("/analysis/Anna/snakemake-demo/data/samples_fastq-test1/"+wildcards.sample):
        fastqs.append(os.path.join("/analysis/Anna/snakemake-demo/data/samples_fastq-test1/",wildcards.sample,s))
    return fastqs


rule all:                                                                        
    input:                                                                       
        expand("results/{sample}/{sample}.fastq", sample=SAMPLES)                   


rule foo:
    input:
        fastq_files = getFastqFilesForTissu 
    output:
        "results/{sample}/{sample}.fastq"
    shell:
        'ln -s {input} {output}'

This works if I have one file in each folder and when the file name matches with the folder's name (i.e., when there is no T1 or N1 suffix).

Is there any way I could modify my rule to get it working for all samples in SRC directories? Or a better way to handle this issue?

$\endgroup$
4
  • 1
    $\begingroup$ What is the output you expect? What values do you want your {input} and {output} to have for a given example? This looks like a simple shell operation, do you really need to use snakemake? $\endgroup$
    – terdon
    Sep 15, 2023 at 7:46
  • $\begingroup$ This is just a very small part of a big workflow. the output is symlinked fastq files in the results folder $\endgroup$
    – Anna1364
    Sep 15, 2023 at 16:54
  • $\begingroup$ What symlinked files? What name should the symlink have and what should its target be? The more detail you give, the better the chances someone will be able to answer. Please edit your question and add the expected result so we know what you need. $\endgroup$
    – terdon
    Sep 15, 2023 at 16:57
  • $\begingroup$ All of these information is in the code if you read through the snakemake code. There is no need for any edition. $\endgroup$
    – Anna1364
    Sep 15, 2023 at 19:16

1 Answer 1

2
$\begingroup$

I suggest using package glob, which supports unix-style pattern search. For instance finding all .py files in my repo:

import glob 
all_python_files = glob.glob(r'c:\personal\coderepo\**\*.py', recursive=True)

Which outputs full paths:

'c:\personal\coderepo\SimpleHTR\data\checkDirs.py',
'c:\personal\coderepo\....

I typically like to input the glob.glob to Pandas dataframes:

all_files = glob.glob(f'{base_folder}/*')
all_files = pd.DataFrame(all_files, columns=['full_path'])
all_files['my_filename'] = all_files['full_path'].apply(lambda x: os.path.basename(x))

#do whatever filtering is required:
all_files = all_files[all_files['my_file_name'].str.contains('regex_here')]

Then perform what ever I need to do in loop:

for idx, _ in all_files.iterrows():
    do_something(_['full_path'])
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.