Covariance
In probability theory and statistics, the covariance between two real-valued random variables X and Y, with expected values E(X) = μ and E(Y) = ν is defined as:
- <math>\operatorname{cov}(X, Y) = \operatorname{E}((X - \mu) (Y - \nu)),<math>
where E is the expectation operator.
This is equivalent to the following formula which is commonly used in actual calculations:
- <math>\operatorname{cov}(X, Y) = \operatorname{E}(X Y) - \mu \nu<math>
If X and Y are independent, then their covariance is zero. This follows because under independence, E(X·Y) = E(X)·E(Y) = <math>\mu \nu<math>. The converse, however, is not true: it is possible that X and Y are not independent, yet their covariance is zero. Random variables whose covariance is zero are called uncorrelated.
If X and Y are real-valued random variables and c is a constant ("constant", in this context, means non-random), then the following facts are a consequence of the definition of covariance:
- <math>\operatorname{cov}(X, X) = \operatorname{var}(X)<math>
- <math>\operatorname{cov}(X, Y) = \operatorname{cov}(Y, X)<math>
- <math>\operatorname{cov}(cX, Y) = c\, \operatorname{cov}(X, Y)<math>
- <math>\operatorname{cov}\left(\sum_i{X_i}, \sum_j{Y_j}\right) = \sum_i{\sum_j{\operatorname{cov}\left(X_i, Y_j\right)}}<math>
For column-vector valued random variables X and Y with respective expected values μ and ν, and n and m scalar components respectively, the covariance is defined to be the n×m matrix
- <math>\operatorname{cov}(X, Y) = \operatorname{E}((X-\mu)(Y-\nu)^\top).<math>
For vector-valued random variables, cov(X, Y) and cov(Y, X) are each other's transposes.
The covariance is sometimes called a measure of "linear dependence" between the two random variables. That phrase does not mean the same thing that it means in a more formal linear algebraic setting (see linear dependence), although that meaning is not unrelated. The correlation is a closely related concept used to measure the degree of linear dependence between two variables.
