1
$\begingroup$

I have a files which has columns representing genes,sample numbers and sample names (55). The genes are displayed by occurrence in the samples (highest to lowest-55 and 2).I wish to calculate the genes which may co-occur in samples. I tried doing this with awk 'n=x[$1,$2,$3,$4,$5,$6]{print n"\n"$0;} {x[$1,$2,$3,$4,$5,$6]=$0;}' adding from 1 to 55 columns, but it doesn't work.

An example here:

Nfib    55  A1.S1   A2.S2   A3.S3   A4.S4   A6.S6   A7.S7   A8.S8   A9.S9   A9.S9*  A10.S10 A11.S11 A12.S12 IZ_BH_2.1   IZ_BH_2.2   IZ_BH_2.3   IZ_BH_2.4   IZ_BH_2.5   IZ_BH_2.6   IZ_BH_2.7   IZ_BH_2.8   IZ_BH_2.9   IZ_BH_2.10  IZ_BH_2.11  IZ_BH_2.12  IZ_BH_3.1   IZ_BH_3.2   IZ_BH_3.3   IZ_BH_3.4   IZ_BH_3.5   IZ_BH_3.6   IZ_BH_3.7   IZ_BH_3.8   IZ_BH_3.9   IZ_BH_3.10  IZ_BH_3.11  IZ_BH_3.11* IZ_BH_3.12  IZ_BH_4.1   IZ_BH_4.2   IZ_BH_4.4   IZ_BH_4.5   IZ_BH_4.6   IZ_BH_4.7   IZ_BH_4.8   IZ_BH_4.9   IZ_BH_4.10  IZ_BH_4.11  IZ_BH_5.1   IZ_BH_5.2   IZ_BH_5.3   IZ_BH_5.4   IZ_BH_5.5   IZ_BH_5.6   IZ_BH_5.7   IZ_BH_5.8


Lyn 41  A1.S1   A4.S4   A7.S7   A8.S8   A9.S9   A9.S9*  A10.S10 A11.S11 IZ_BH_2.1   IZ_BH_2.2   IZ_BH_2.3   IZ_BH_2.4   IZ_BH_2.5   IZ_BH_2.6   IZ_BH_2.7   IZ_BH_2.9   IZ_BH_2.11  IZ_BH_2.12  IZ_BH_3.2   IZ_BH_3.3   IZ_BH_3.4   IZ_BH_3.5   IZ_BH_3.6   IZ_BH_3.8   IZ_BH_3.9   IZ_BH_3.10  IZ_BH_3.11  IZ_BH_4.1   IZ_BH_4.2   IZ_BH_4.3   IZ_BH_4.4   IZ_BH_4.7   IZ_BH_4.8   IZ_BH_4.11  IZ_BH_5.1   IZ_BH_5.3   IZ_BH_5.4   IZ_BH_5.5   IZ_BH_5.6   IZ_BH_5.7   IZ_BH_5.8


Cdk5rap2    52  A1.S1   A2.S2   A3.S3   A6.S6   A7.S7   A8.S8   A9.S9   A9.S9*  A11.S11 A12.S12 IZ_BH_2.1   IZ_BH_2.2   IZ_BH_2.3   IZ_BH_2.4   IZ_BH_2.5   IZ_BH_2.6   IZ_BH_2.7   IZ_BH_2.8   IZ_BH_2.9   IZ_BH_2.10  IZ_BH_2.11  IZ_BH_2.12  IZ_BH_3.1   IZ_BH_3.2   IZ_BH_3.3   IZ_BH_3.4   IZ_BH_3.5   IZ_BH_3.6   IZ_BH_3.7   IZ_BH_3.8   IZ_BH_3.9   IZ_BH_3.10  IZ_BH_3.11  IZ_BH_3.11* IZ_BH_3.12  IZ_BH_4.1   IZ_BH_4.2   IZ_BH_4.3   IZ_BH_4.4   IZ_BH_4.6   IZ_BH_4.8   IZ_BH_4.9   IZ_BH_4.10  IZ_BH_4.11  IZ_BH_5.1   IZ_BH_5.2   IZ_BH_5.3   IZ_BH_5.4   IZ_BH_5.5   IZ_BH_5.6   IZ_BH_5.7   IZ_BH_5.8

The first name is of the gene, followed by number of samples and sample names.

for these 3 genes these are the commonly occuring samples:

A1.S1, A7.S7, A8.S8, A9.S9*, A9.S9, A11.S11, IZ_BH_2.1, IZ_BH_2.2, IZ_BH_2.3,IZ_BH_2.4, IZ_BH_2.5,IZ_BH_2.6,IZ_BH_2.7, IZ_BH_2.9, IZ_BH_2.11,IZ_BH_2.12, IZ_BH_3.2,IZ_BH_3.3,IZ_BH_3.4, IZ_BH_3.5, IZ_BH_3.6,IZ_BH_3.8,
IZ_BH_3.9,IZ_BH_3.10,IZ_BH_3.11,IZ_BH_4.1,IZ_BH_4.2,IZ_BH_4.4,IZ_BH_4.8,IZ_BH_4.11,IZ_BH_5.1,IZ_BH_5.3,IZ_BH_5.4,IZ_BH_5.5,IZ_BH_5.6,IZ_BH_5.7,IZ_BH_5.8

Kindly help.

$\endgroup$
2
  • $\begingroup$ Show a small worked example plz $\endgroup$ Commented Feb 13, 2021 at 22:46
  • $\begingroup$ I have added an example $\endgroup$
    – Amit
    Commented Feb 14, 2021 at 13:23

1 Answer 1

1
$\begingroup$

I think what makes this especially hard is the way your data is formatted. The problem would be way easier if you had a matrix of presence / absence instead. Each row would be a gene, each column a sample, and the values would be 0 (absent) or 1 (present).

Awk becomes a bit inconvenient for this task, so I'll use python instead. To get such a matrix from your file, you could do the following:

import numpy as np
import pandas as pd

# Get total number of genes / samples
all_samples = set()
all_genes = set()
with open('genes.txt', 'r') as genes:
  for line in genes:
    line = line.split('\t')
    all_genes.add(line[0])
    all_samples.update(line[2:])
all_samples = list(all_samples)
all_genes = list(all_genes)

# Initialize presence matrix with correct dimensions
pres_mat = np.zeros((len(all_genes), len(all_samples)), dtype=int)

# Map each sample/gene to a row/col
row_map = {gene: i for i, gene in enumerate(all_genes)}
col_map = {sample: i for i, sample in enumerate(all_samples)}

# Fill matrix incrementally
with open('genes.txt', 'r') as genes:
  for line in genes:
    line = line.split('\t')
    curr_gene = line[0]
    row = row_map[curr_gene]
    for sample in line[2:]:
      col = col_map[sample]
      pres_mat[row, col] = 1

# Add names and write to file
df = pd.DataFrame(pres_mat, columns=all_samples)
df.insert(0, 'gene', all_genes)
df.to_csv('gene_presence.tsv, sep='\t')

You should get something that looks like:

       gene  IZ_BH_5.5  A8.S8  IZ_BH_2.1  A9.S9*  IZ_BH_5.3  A9.S9  ...  IZ_BH_3.11  IZ_BH_4.7  IZ_BH_4.9  IZ_BH_2.8  IZ_BH_3.5  IZ_BH_2.6  IZ_BH_4.3
0       Lyn          1      1          1       1          1      1  ...           1          1          0          0          1          1          1
1      Nfib          1      1          1       1          1      1  ...           1          1          1          1          1          1          0
2  Cdk5rap2          1      1          1       1          1      1  ...           1          0          1          1          1          1          1

With this format, it is now easy to compute the most frequent samples. E.g. df.sum(axis=0) to get the number of occurence of each sample for these 3 genes. It would also allow you to do different things, such as computing the pairwise similarity between samples.

$\endgroup$
1
  • $\begingroup$ Thanks a lot @cmdoret $\endgroup$
    – Amit
    Commented Feb 17, 2021 at 10:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.