# Simple Linear Regression

Last Update: February 21, 2022

Simple linear regression is used to model linear relationship between two variables $y$ and $x$ . Dependent variable $y$ is the explained one which is also known as target or response feature. Independent variable $x$ is the explanatory one which is also known as predictor feature.

When doing simple linear regression, we can start by drawing a scatter chart with variables $y$ and $x$ on the vertical and horizontal axis, respectively. Then, we can draw a line which describes linear relationship between variables $y$ and $x$. This line represents model fitting with formula $\hat{y}_{i}&space;=&space;\hat{\beta}_{0}&space;+&space;\hat{\beta}_{1}&space;x_{i}&space;\;&space;(1)$. Notice that we are using ˄ or hat character in formula notation because they are estimates. Regression fitted values $\hat{y}_{i}$ are the estimated $y_{i}$ values. Estimated constant or intercept coefficient $\hat{\beta}_{0}$ is the $\hat{y}$ value when $x=0$ or the $\hat{y}$ value where line crosses vertical axis. Estimated slope coefficient $\hat{\beta}_{1}$ is the estimated change in $\hat{y}$ when $x$ changes in one unit.

Model fitting can be done using ordinary least squares method with formula $min&space;\;&space;\sum_{i=1}^{n}&space;\hat{e}_{i}^{2}&space;\;&space;(2)$. This method minimizes the sum of squared estimated regression residuals $\hat{e}_{i}$. Estimated regression residuals $\hat{e}_{i}$ with formula $\hat{e}_{i}&space;=&space;y_{i}&space;-&space;\hat{y}_{i}&space;\;&space;(3)$ are the differences between actual $y_{i}$ and fitted $\hat{y}_{i}$ values.

Below, we find an example of scatter chart with simple linear regression of house price explained by its lot size [1].

Courses

My online courses are hosted at Teachable website.

For more details on this concept, you can view my Linear Regression Courses.

References

[1] Data Description: Sales prices of houses sold in the city of Windsor, Canada, during July, August and September, 1987.

Original Source: Anglin, P., and Gencay, R. (1996). Semiparametric Estimation of a Hedonic Price Function. Journal of Applied Econometrics, 11, 633–648.

Source: AER R package HousePrices object. Christian Kleiber and Achim Zeileis. (2008). Applied Econometrics with R. Springer-Verlag, New York.

+