1
$\begingroup$

The website https://maayanlab.cloud/sigcom-lincs/#/SignatureSearch/UpDown can be used to "Identify reversers and mimickers from over 1 million signatures by entering up and down gene sets .. " I am looking to get these "reversers" and "mimickers", aka chemical perturbagens.

So on this website, I paste my list of a selection of Up-regulated genes, in their respective box, and the same for my list of down-regulated genes. These genes are from an RNA sequencing experiment.

After the website has done its thing, I expand the search for chemical perturbagens (LINCS L1000 Chemical Perturbations (2021)). One of the data tables I can download contain something called a Gene Rank Matrix. This table consist of one row for each gene and columns of the chemical perturbagens found. The values are integers.

A small subsection of the data table would look like this: enter image description here

My questions are what these integers (12, 144, 411, 641) represent. Are they how much the perturbagens (In this case REP.B025_YAPC_24H_K01_resiquimod_2.22uM) will affect the genes?

The paper describing this website searches (https://academic.oup.com/nar/article/50/W1/W697/6582159) mentions "The expression tables provide the computed CD differential gene expression coefficients for each signature, while the rank matrices provide the integer gene ranks for each signature as determined by the coefficients"

and

"Matrices that represent the gene rankings of the L1000 signatures are stored as HashMaps by the data API in random access memory (RAM). HashMaps are data structures designed to efficiently search for key-value pairs, and this implementation drastically cuts the fetch times of the gene rankings by UUIDs, thus improving the speed of the search"

Does this mean that the scores found wont have any meaning in regards to how well the perturbagens found will affect the genes? Or can some meaning be derived from this?

Thanks in advance!

$\endgroup$
0

1 Answer 1

1
$\begingroup$

Summary answer Although I'm unhappy about how the paper is written and possibly the methodology, the following sentence is likely (almost certainly) to be the answer.

The coefficient (aka weight) is how important that gene is for the classification of that signature. Thats it! :-)

Essentially, there are loads of genes involved in identifying which genes represents which signature. Not all those genes are equally informative. The positive or negative skews in that calculation, called a "classification", are identified through their weights (aka coefficients). Thats simply the answer. The bigger the coefficient the more important that gene is in the classification, i.e. its associated with the signature.

There is no need to be more complicated than that. If you are interested in a critique of that paper, which I had to do to get to the answer and thereby understand the analytics of that paper, this is presented below.



Critique of Evangelista et al. 2022: Nucleic Acids 50:W697–W709

Note 1 I don't have time to read the paper in absolute detail, particularly the appendices, where some of the information needed may have be placed.

Note 2 In the answer below, a "expression coefficient" is what I've called an "expression weight" its the same thing.

Weightings and analytical concerns I do understand weightings in deep learning, particularly machine learning. I read the paper and the ANN (artificial neural network) was very basic. I don't believe it had multiple layers, because it would have said so and it was a very odd description for a deep learning algorithm. It would also be odd to have a single layer in deep learning, very odd. The loss function is also not described at all which is extremely important in deep learning (it will not work otherwise). Essentially, what the authors call "deep learning" appears to a machine learning neural network using an Adams gradient descent. Adams gradient descent is a standard approach in deep-learning admittedly. Thereafter in the paper the details were very sketchy, maybe they are in the appendix (should be checked). Any deep-learning, machine-learning requires a demonstration of the robustness of the training and this appears absent. The two basic stats are accuracy and AUC-ROC, which were definitely absent. Universal stats are used to quickly understand how good the algorithm is in comparison to other deep-,machine- learning algorithms for other data sets.

Analytical overview Generally I would assume the authors have performed a supervised learning algorithm, but this may not be correct (below). If it is supervised learning they have performed they would have identified their 'signatures' against a very large data set, then trained the different types of signature against that data and tested it. The learning could be terrible - but it's just not stated at all. I'm surprised because Nucleic Acids is at the pinnacle of algorithm publications.

Summary. The authors performed deep-learning (which looked like a neural network machine learning algorithm). The description is at best poor, but further information could be present in the appendix. Therefore its not absolutely clear what a "weight" would represent because there are two possibilities.

Weights possibility: unsupervised learning The one thing that might have happened is they've performed an unsupervised learning algorithm and the weights are a sort of "maximum variance". This very trendy. That would explain the lack of accuracy and AUC-ROC type stuff. IF this is true and I don't believe it's explained - I could be wrong - then the weights represent maximum information. What does that mean? Good question, its context specific, its the gene which is most important in sectioning up the data - e.g. housekeeping gene which is required 'all over the place', not just part of the cell cycle is an example. Basically, in unsupervised weighted output high ranking weights are a mix of really boring and really brilliant stuff. So you just go through it with biological understanding, is this gene boring yeah/neigh, is this one - wait wow! unexpected ... That is actually fashionable analytics :-)

Weights possibility: supervised learning Is already answered at the beginning. This is almost certainly the correct answer - it would be the orthodox approach - and we just accept the robustness checks for the algorithm and possibly the algorithm itself were simply insufficient.

Why isn't the paper clear (at best)? What I assume has happened is the team have brilliant frontend and genomics team backend. The analyst in the middle is unlikely to be a specialist and thats why it's poorly described, or possibly performed. I assume if the reviewers were of similar composition then that's how its got through Nucleic Acids. Thus, I would personally treat the output with caution because deep learning (machine-learning) will be the essential bottleneck between their backend development and your data. Maybe "bottleneck" is not quite an accurate description, because its speeding up the classification, so maybe a "gate-keeper" is a better description.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.