Exam Questions - Block 5: Linear model

[SebastianVeuskens]

Linear model (last page of the notes)→matter of formulas

[SebastianVeuskens]

Linear regression model

Assumptions

1. Homoschedastic and independent error:

2. Connection between observation, covariants and error:

Likelihood calculations

We get the following likelihood for the observation, conditional on the variance and the coefficients.

With

and

The important part is the step from row 3 to row 4. By choosing beta_hat this way and adding and substracting Xbeta_hat at the same time inside the paranthesis of (y-Xbeta), it becomes ((y-Xbeta_hat)+(Xbeta_hat-Xbeta)). This leads finally to just the two terms left in row 4 in the exponent.

Posterior

Choose a non-informative prior (other choices are possible but more complicated):

Then the posterior distribution is:

Connection between Bayesian and Frequentist

In the Bayesian linear model framework, we get a distribution for the coefficients (in this case a t-distribution). In contrast, in the frequentist linear model framework, a maximum-likelihood estimate is obtained for the parameters.
However, the location and thus the mode of the coefficient's t-distribution is beta_hat. This is the same as the maximum-likelihood estimate obtained from the frequentist approach.
Thus, both approaches lead to the same point-estimate for the coefficients.