3.6 Estimation of Autocovariance, Autocorrelation, and Cross-Correlation

Thus far, we have dealt with the theoretical values for various forms of variance, covariance, and correlation based on the assumption that we know the underlying process from which our observations have been drawn. In real life, we will usually be given the time series and then attempt to infer aspects such as the underlying process from statistical estimators. In this section we will develop techniques analogous to how sample covariance and sample correlation are used to estimate the true covariance and correlation in data science and statistics.

Estimate Definitions¶

Sample Autocovariance¶

The standard formula used in sources such as Shumway & Stoffer (2025) and for sample autocovariance is

where $\bar{x}$ is the sample mean and the “hat” notation indicates that $\hat{\gamma}$ is an estimate of the true population value. Note that we assume the same mean for $x_t$ and $x_{t+h}$.

Sample Autocorrelation¶

Autocorrelation is estimated by defining the sample autocorrelation

Sample Cross-Covariance¶

The sample cross-covariance is defined as

and the sample cross-correlation as

Properties of Estimators¶

Positive Semidefiniteness¶

You may have noticed something strange about Eq. (1). For $n$ observations, we only have $n-h$ pairs to sum over; for example when $n=100$ and $h=2$ there are 98 pairs consisting of $(x_0,x_2), (x_1,x_3),\ldots,(x_{97}, x_{99})$. Why then, do we divide Eq. (1) by $n$ rather than $n-h$?^[1]

The answer relates to the necessity that the autocovariance matrix (and by extension autocorrelation matrix) be positive semidefinite in order to avoid the possibility of generating negative variances. To see that the form used in Eq. (1) guarantees this property, let us sketch the proof from Brockwell & Davis (2016)^[2]. Let us define the matrix $\mathbf{X}$ using the demeaned time series such that

where the elements of $X$ are the vectors $[\ldots, x_0-\bar{x}, x_1-\bar{x}, x_2-\bar{x}, \ldots]$ appropriately padded with zeros. Put slightly differently, we have now factored $\boldsymbol{\Gamma}$ such that

Since we have expressed $\boldsymbol{\Gamma}$ as the square of $\mathbf{X}$, for any vector $\mathbf{v}$ we have

The central point to understand is that we could only factor $\boldsymbol{\Gamma}$ because we could pull out $\frac{1}{n}$ from every entry. If we separately weighted each $\gamma(h)$ in $\boldsymbol{\Gamma}$ by $n-h$, we would no longer be able to express the autocovariance matrix $\boldsymbol{\Gamma}$ as the product of two other matrices. Consequently, we would have no guarantee that $\boldsymbol{\Gamma}$ was positive semidefinite, and could end up with negative variances.

Bias and Consistency¶

Note that neither dividing by $n$ nor $n-h$ creates an unbiased estimator, i.e. in neither case is it true that

though when dividing by $n-h$ the bias will be smaller.

The estimates are both, however, consistent in that in as $n\rightarrow\infty$

As observed in Wasserman (2004), most modern statistics texts consider consistency to be sufficient and are less concerned with estimators being unbiased.

Standard Error of Autocorrelation and Cross-Correlation¶

A natural question to ask is how we might determine if an autocorrelation value is statistically significant. We previously touched on the fact that programs such as statsmodels estimate statistical significance automatically. But we have not yet explained how significance is estimated.

It turns out that in order to definitively calculate statistical significance, we would need to know the underlying process that generated our data—exactly the question we are trying to answer in the first place. Instead, let’s ask a slightly different question: What values should we expect for the autocorrelation if our time series is pure white noise?

Under the assumption of white noise^[3], we can expect the sample autocorrelation values to be normally distributed with mean 0 and variance $\frac{1}{n}$

where $n$ is the total number of observations (Box et al. (2008)). Equivalently, the standard error of $\hat{\rho}(h)$ is given by

For a white noise process, we expect around $95\%$ of all sample autocorrelation values to fall within $\pm\frac{2}{\sqrt{n}}$. By default, statsmodels usually uses Eq. (13) at a $95\%$ confidence interval to estimate statistical significance to create figures such as Figure 2^[4]. It can be shown that this formula also holds for estimating the sample cross-correlation $\hat{\rho}_{x,y}(h)$ (Shumway & Stoffer (2025)).