📊 OLS from First Principles — Matrix‑Form Trainer

  • Generate 4 points from y = b0 + b1 x + e.
  • Explore the SSE bowl and its tangent at your guess.
  • Derive the matrix closed form $\hat{\mathbf{b}}$ and the sampling variance $\mathrm{Var}(\hat{\mathbf{b}})$, then write small functions that use each other in sequence.

Dataset (n=4)


🧠 Derivation Lab (Matrix form)

🔬 Visual mini‑tables (colored): how each product is built

2×2 inverse rule (colored)


🧮 Practice by line item

b to use for items that need it

1) $ (\mathbf{X}'\mathbf{X})^{-1} $ (returns k×k matrix → enter row‑major k*k numbers)

2) $ \mathbf{X}'\mathbf{y} $ (k×1 vector)

3) $ \hat{\mathbf b} = (\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}'\mathbf{y} $ (k×1 vector)

4) $ \mathbf{e} = \mathbf{y} - \mathbf{X}\mathbf{b} $ (returns n×1 vector)

5) $ \hat{\sigma}^2 = \tfrac{1}{n-k}\,\mathbf{e}'\mathbf{e} $ (scalar)

6) $ \mathrm{Var}(\hat{\mathbf{b}}) = \hat{\sigma}^2 (\mathbf{X}'\mathbf{X})^{-1} $ (k×k matrix → enter row‑major)

5a) $ \mathrm{se}(\hat{\mathbf b}) = \sqrt{\operatorname{diag}(\mathrm{Var}(\hat{\mathbf b}))} $ (k×1 vector)

5b) $ R^2 = 1 - \tfrac{\mathbf e'\mathbf e}{(\mathbf y - \bar y)'(\mathbf y - \bar y)} $ (scalar)


📐 Closed‑form OLS: $\hat{\mathbf{b}} = (\mathbf{X}'\mathbf{X})^{-1} \mathbf{X}'\mathbf{y}$

Write the formula (any notation). Examples: inv(X'X) X'y or inv(X.T @ X) @ X.T @ y.

Contribs to X'X (row = obs)

Sums for X'X

X'X (2×2)

Contribs to X'y (row = obs)

Sums for X'y

X'y (2×1)



🧩 Write the Python code (functions)

Implement each function. Click Run test to check. If shapes don't match, you'll see a dimension hint.

Hint: use n, k = X.shape.

Hint: $R^2 = 1 - \tfrac{\text{SSE}}{\text{TSS}}$, with $\text{SSE}=\mathbf{e}'\mathbf{e}$ and $\text{TSS}=(\mathbf{y}-\bar y)'(\mathbf{y}-\bar y)$.


🧩 Write the code — R functions

Implement each R function. Click Run test to check. If R is not installed, you'll see a friendly note.

Hint (R): n <- nrow(X); k <- ncol(X).


📦 Assemble your code file

📦 Assemble your R script