Standard Error of Coefficients in Regression

Standard Error of Coefficients in Regression

1- Regress (Dependent Y) to (Independent) X₁, X₂, X₃…..

2-   Sum of Residual Square =  (Y_i (predicted) - Y_i )²

3-   Mean Square Error = Sum of Residuals / ( N - K - 1)

N = Number of observations, K = Number of Regressor as above

4-   Regress by taking (Dependent) X₁ and (Independent) X₂, X₃, X₄………..

For standard error of slope of X_1.

a-   Sum of Residual Square (SS Residual) =  (X_i (predicted) - X_i )²

b-   Standard Error of slope of X1  = Sqrt (( (3) / (a))

5-   T-Statistic - Standard Error / Coefficient.

6-   P - Value in Excel = =TDIST( ( absolute value of t-statistic),degree of freedom, tails(2))

Repeating steps 4-6 for statistics of X2- Dependent) X₂ and (Independent) X₁

+

-       Variance-

-       Slope-

Standard Error of Multiple Regression Matrix Form

Y = b₀ + b₁*X₁ + b₂*X₂

[2]

-        Note- It is possible to have variables that are dependent but uncorrelated, since correlation only measures linear dependence. A nice thing about normally distributed RV’s is that they are a convenient special case: if they are uncorrelated, they are also independent.

-        Coefficients can be significant despite of low R² as R² do not consider intercept.

-        For equation – Y = b₀ + b₁*X₁ + b₂*X₂

We have-

Squaring both sides-

---

[1] SS Residuals is residuals of regression with X as dependent and Z as independent.

[2] S²_{y, 12} is mean square residual of regression with independent x2 and dependent x1.

   


https://drive.google.com/file/d/0Bx3mfFH5R-y3c0pOQkc5VkJNVlk/view?usp=sharing