0
$\begingroup$

So I have around 10,000 FASTA files of Influenza A virus. These files contains sequences of each of the 8 segments of the viral genome and I want to separate these files into different locations based on the content of these FASTA files. In each FASTA file, first line has the segment number. Eg.

>KM368312.1 Influenza A virus (A/swine/Shandong/01/2009(H1N1)) segment 3 polymerase PA (PA) and PA-X protein (PA-X) genes, complete cds

Okay imagine you have 10,000 text files in one folder and there are 8 types of these text files based on the content of them. Eg. file1.txt has segment 1 written in it, file2.txt has segment 2 written in it upto segment 8. So I want to group the files containing segment 1 together in one folder, segment 2 in other folder together and so on.

$\endgroup$
11
  • $\begingroup$ I edited your question to make it more readable. Check if I interpreted it correctly and change if not. $\endgroup$ – Stephan Oct 29 '20 at 9:08
  • $\begingroup$ Thank you, It's my first time using stackoverflow, so sorry about that. $\endgroup$ – Neeleshwar Pandey Oct 29 '20 at 9:10
  • $\begingroup$ My genetics knowledge is quite basic. Which is the part you want to sort by? (I assume you have several folders with that name where the corresponding files should go?) $\endgroup$ – Stephan Oct 29 '20 at 9:10
  • $\begingroup$ We can treat these fasta files as basic text files containing sequences. So i have 10,000 text files and i want to sort them into different folders based on the segment sequence these files have. Eg. One text file will have segment 1 sequence only, and so on. In total there are 8 segments, so 8 folders. I hope i am clear now. $\endgroup$ – Neeleshwar Pandey Oct 29 '20 at 9:13
  • $\begingroup$ What's the "segment sequence" part? KM368312.1? $\endgroup$ – Stephan Oct 29 '20 at 9:14
2
$\begingroup$

That's quite easy with cmd:

@echo off
for /l %%i in (1,1,8) do (
  for /f "delims=" %%f in ('findstr  /mc:" segment %%i " *.txt') do (
    ECHO move "%%f" "segment %%i\"
  )
)

findstr /mc:"string" *.txt searches each matching file for the string and outputs the filename, if the string was found. The for /f loop around catches those filenames, so you are able to move them. The for /l loop just iterates from 1 to 8.

See for /? and findstr /? for more information about those commands.

You probably want to adapt the filemask *.txt to your needs.

Note: I "disarmed" the move command for troubleshooting. Remove the ECHO when you are sure it does what you want.

$\endgroup$
1
  • $\begingroup$ I know this is tagged cmd but it’s very unlikely that OP is actually using Windows when working with FASTA files. $\endgroup$ – Konrad Rudolph Oct 29 '20 at 10:07
1
$\begingroup$

The general idea is:

  1. iterate over all files
    1. read the first line of each file, extract the segment bit
    2. move the file based on that

In Python, this can be done in many ways. Here’s one way:

import os
import re


def find_segment(id):
    return re.search(r'segment (\d)', id).group(1)


for segment in range(8):
    os.mkdir(f'segment-{segment + 1}')

for entry in os.scandir():
    if not entry.is_file():
        continue

    with open(entry.path, 'r') as file:
        segment = find_segment(next(file))
    os.rename(entry.path, f'segment-{segment}/{entry.path}')

Note the absence of error handling — this code assumes that every operation succeeds; empty files, or files without “segment X” in their FASTA ID, will trigger an error, as will usual IO failures (missing permissions, etc.).


But many people wouldn’t use Python to solve this but a shell scripting language. For example, here’s something equivalent in Bash (untested):

mkdir segment-{1..8}

for file in *.fa *.fasta; do
    segment=$(sed 's/.*segment \([[:digit:]]\).*/\1/; q' "$file")
    if [[ "$segment" != '' ]]; then
        mv "$file" "segment-$segment"
    done
done
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.