NLP and vectorisation is used to trap non-linearity which is highly applicable to sentences written in English. Vectorisation of k-mers for an overlapping sliding window is weird because it's trying to assess the functionality of non-coding sequence, i.e. "silent" and non-cds mutations will be the dominant mutation class, but it would identify localised co-mutations, i.e. within a k-mer.
The only application I can think is stuff like identifying promoters or mutations in promoters, i.e. non-coding regions with a known function and can be trained against appropriate independent experimental observations, e.g. gene expression. If the k-mers were large it might work for RNA secondary structure because it would trap compensatory mutations. Vectorisation for individual nucleotides, i.e. dumbie variables, would not be successful at trapping compensatory mutations.
In summary it has niche application.
BERT is a famous large language model BTW.
If k-mer =3 that's about non-linearity of proteins outside an RNN algorithm, but it depends how the sliding window is deployed. It depends on the step size of the sliding window.
To try and answer this better, proteins fold back on themself triggering "co-evolution" or more accurately "covariance" between non-linear sites. This defines the triplet codon:
- in machine learning this is needed because it will struggle to find it by itself
- it will help it identify co-evolving mutations at non-linear locations across the gene
It might also identify selection events better.
Summary If you've sat down and thought the equation the RNN is attempting to unravel its mind-boggling. More worrisome does it really do it? It's really complicated to simply discuss, but when you give it key information the ML/DL gets there quicker. The authors will be aware of that in their training and testing.
Honestly, we're at the limits of reasonable application of ML/DL without going into RasNet type infrastructure. My system is different the idea is the same - give the training model the key biological information, it gets to the answer quicker, a lot quicker.