# Negative binomial modeling of RNA-Seq data

A common way to model RNA-seq data is using a negative binomial distribution, where each sample-gene pair is modeled by a different negative binomial distribution with mean $\mu_{ij}$ where $i$ and $j$ are indices for genes and samples, respectively (see article). My question is, given we only have a single observation ($X_{ij}$) from each sample-gene pair, how come we can learn the $\mu_{ij}$ from that single observation? We need multiple observations to estimate a mean, right? Maybe I am missing something critical.

• The negative binomial distribution is not fitted for a single gene-sample, but along a sample – llrs Feb 26 '18 at 8:05
• “how come we can learn the $\mu_{ij}$ from that single observation?” — we can’t. This fundamentally doesn’t work. – Konrad Rudolph Feb 26 '18 at 10:36
• @Llopis thanks, but if they are fitting for each gene separately, where is the sample-specific information coming from such that they can learn a different $\mu_{ij}$ for each pair? – user5054 Feb 26 '18 at 13:43
• I wasn't sure of my first comments and I'm not with this one but, the value for each gene can't be estimated for a single sample. But it is estimated from all the samples. See the first section of the methods in the linked paper. – llrs Feb 26 '18 at 14:55
• @TomKelly It is, in fact, extremely common. The two most commonly used packages for differential expression use it (edgeR, DESeq). – Konrad Rudolph Jun 18 '19 at 11:34

The $\mu_{ij}$ in that section of the DESeq manuscript is the expected value of sample $j$ in gene $i$ given its group association (with expected value $\mu_{ij}$). This is why $K_{ij}$ (this is presumably what you meant by $X_{ij}$) is a function of $\mu_{ij}$, rather than $\mu_{ij}$ itself. I think you're just swapping $K_{ij}$ and $\mu_{ij}$ in your head.