Yule Walker

Derivation of the Yule-Walker Equations

Yule-Walker equations is to assume that the signal lags multiplied by the autoregressive (AR) model coefficients equals the noise because this follows directly from the definition of an autoregressive process.

1. Autoregressive Model Assumption

An autoregressive process of order \( p \) (AR(\( p \))) is defined as:

\[ x_t = \sum_{i=1}^{p} \phi_i x_{t-i} + w_t \]

where:

• \( x_t \) is the time series at time \( t \),
• \( \phi_i \) are the AR model coefficients,
• \( w_t \) is white noise (uncorrelated with zero mean and constant variance),
• \( x_{t-i} \) are past values (lags) of the signal.

This equation states that each value in the time series is a linear combination of past values plus some noise.

2. Moving to the Autocovariance Function

By multiplying both sides by \( x_{t-k} \) (for some lag \( k \)) and taking expectations, we obtain:

\[ E[x_{t-k} x_t] = E \left[ x_{t-k} \sum_{i=1}^{p} \phi_i x_{t-i} \right] + E[x_{t-k} w_t] \]

Since \( w_t \) is white noise and is uncorrelated with past values (\( x_{t-k} \)), we get:

\[ E[x_{t-k} w_t] = 0 \]

which simplifies the equation to:

\[ \gamma_k = \sum_{i=1}^{p} \phi_i \gamma_{k-i} \]

where \( \gamma_k = E[x_t x_{t-k}] \) is the autocovariance function of \( x_t \).

3. Forming the Yule-Walker Equations

By writing these equations for \( k = 1, 2, ..., p \), we obtain a system of linear equations in terms of the autocovariance function \( \gamma_k \) and the AR coefficients \( \phi_i \). This system of equations is known as the Yule-Walker equations:

\[ \begin{bmatrix} \gamma_0 & \gamma_1 & \gamma_2 & \dots & \gamma_{p-1} \\ \gamma_1 & \gamma_0 & \gamma_1 & \dots & \gamma_{p-2} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \gamma_{p-1} & \gamma_{p-2} & \gamma_{p-3} & \dots & \gamma_0 \end{bmatrix} \begin{bmatrix} \phi_1 \\ \phi_2 \\ \vdots \\ \phi_p \end{bmatrix} = \begin{bmatrix} \gamma_1 \\ \gamma_2 \\ \vdots \\ \gamma_p \end{bmatrix} \]

Key Takeaways

• The assumption that signal lags multiplied by AR coefficients equals noise follows from the AR process definition.
• This leads to an equation where we multiply by \( x_{t-k} \) and take expectations, eliminating the noise term due to its uncorrelation.
• The resulting system of equations links the autocovariance function to the AR coefficients, forming the Yule-Walker equations, which allow estimation of AR parameters from the observed data.

This method is widely used in time series analysis and linear prediction, especially in estimating the parameters of an AR process efficiently.