Replace a string in a column of a text file based on matching string listed in another file using Linux

Question

I have multiple sets of files, so I grepped the list from a column in a separate file such as file1.txt. I want to change all other files by using this file1.txt based on a matching string.

For example, I am pasting three files here

File 1:

chr - seq1 NZ_JAHWGH010000010.1 0 60562 green_a4
chr - seq3 NZ_JAHWGH010000012.1 0 466573 green_a4
chr - seq5 NZ_JAHWGH010000013.1 0 125526 green_a4
chr - seq39 NZ_CAJGBF010000006.1 0 211154 green_a4
chr - seq40 NZ_CAJGBF010000007.1 0 199119 green_a4
chr - seq41 NZ_CAJGBF010000008.1 0 190934 green_a4
chr - seq42 NZ_CAJGBF010000009.1 0 178251 green_a4

I extracted the required information using grep; the command is given below:

grep "NZ_JAHWGH" file.txt |awk '{print $3}' >file1.txt
grep "NZ_CAJGBF" file.txt |awk '{print $3}' >file2.txt

Now I wanted to use these file1.txt and file2.txt to change another file. Output will be like if list in file1.txt matches then all matched lists should be substituted as seq1 similarly using the second file2.txt. I want to substitute all matching patterns as seq2.

For example the file is given here to be changed by using information from the previous file:

seq3 19301 20914 fill_color=green_a0
seq1 16726 18337 fill_color=green_a0
seq5 167934 169650 fill_color=green_a0
seq42 200621 202367 fill_color=red_a0
seq41 160164 161903 fill_color=green_a0
seq40 31356 33104 fill_color=green_a0

And final output after the change should be as follows:

seq1 19301 20914 fill_color=green_a0
seq1 16726 18337 fill_color=green_a0
seq1 167934 169650 fill_color=green_a0
seq2 200621 202367 fill_color=red_a0
seq2 160164 161903 fill_color=green_a0
seq2 31356 33104 fill_color=green_a0

I tried using this command:

for i in f2.txt; do
    grep -wf f1.txt $i |
    awk '{$1="seq1";print}'
    grep -wf other.txt $i |
    awk '{$1="seq2";print}'
done

2nd attempt:

grep -wf f1.txt f2.txt |
sed -e 's/seq[0-9]*/seq1/g'

Is this correct? If it is correct is there any more efficient way to do it? Also it is only printing to the console. I want the output and also changes in the main file. Kindly improve my scripts.

I appreciate yout time and help.

Answer 1

Here's one way using awk. This avoids the creation of the intermediary files by matching the alphabetic characters in the fourth column. If the prefix has not been seen, it is added to an array along with a counter, which is only incremented when the prefix has not yet been seen. We then store the seq ids and the replacement strings in another array. Once we've finished reading the first file, we can then query the first column of the second file in the arguments list. If the seq id in the first column exists in the array of seq ids, we can print the replacement string and remaining columns.

With the following in a file called script.awk:

BEGIN {

   counter = 1
}

FNR==NR {
    if( match($4, /^NZ_[A-Z]+/) ) {
        prefix = substr($4, RSTART, RLENGTH)

        if( !(prefix in seen) ) {
            seen[prefix] = counter++
        }

        seq_ids[$3] = "seq" seen[prefix]
    }

    next
}

$1 in seq_ids {

    print seq_ids[$1], $2, $3, $4
}

Run using:

awk -f script.awk file1 file2

Results:

seq1 19301 20914 fill_color=green_a0
seq1 16726 18337 fill_color=green_a0
seq1 167934 169650 fill_color=green_a0
seq2 200621 202367 fill_color=red_a0
seq2 160164 161903 fill_color=green_a0
seq2 31356 33104 fill_color=green_a0