Skip to content

Multiple Linear Regression

Last Update: February 21, 2022

Multiple linear regression is used to model linear relationship between one dependent or explained variable y and two or more independent or explanatory variables x_{1},...,x_{p}. Variable y is also known as target or response feature and variables x_{1},...,x_{p} are also known as predictor features.

As example, we can fit a three-variable multiple linear regression model with formula \hat{y}_{i}=\hat{\beta}_{0}+\hat{\beta}_{1}x_{1i}+\hat{\beta}_{2}x_{2i}\;(1). Notice that we are using ˄ or hat character in formula notation because they are estimates. Regression fitted values \hat{y}_{i} are the estimated y_{i} values. Estimated constant coefficient \hat{\beta}_{0} is the \hat{y} value when x_{1}=0 and x_{2}=0. Estimated partial regression coefficient \hat{\beta}_{1} is the estimated change in y when x_{1} changes in one unit while holding x_{2} constant. Similarly, estimated partial regression coefficient \hat{\beta}_{2} is the estimated change in y when x_{2} changes in one unit while holding x_{1} constant.

Model fitting can be done using ordinary least squares method with formula min\sum_{i=1}^{n}\hat{e}_{i}^{2}\;(2). This method minimizes the sum of squared regression residuals \hat{e}_{i}. Regression residuals \hat{e}_{i} with formula \hat{e}_{i}=y_{i}-\hat{y}_{i}\;(3) are the estimated differences between actual y_{i} and fitted \hat{y}_{i} values.

Below, we find an example of estimated coefficients from multiple linear regression of house price explained by its lot size and number of bedrooms [1].

Table 1. Microsoft Excel® estimated coefficients from multiple linear regression of house price explained by its lot size and number of bedrooms.

Courses

My online courses are hosted at Teachable website.

For more details on this concept, you can view my Linear Regression Courses.

References

[1] Data Description: Sales prices of houses sold in the city of Windsor, Canada, during July, August and September, 1987.

Original Source: Anglin, P., and Gencay, R. (1996). Semiparametric Estimation of a Hedonic Price Function. Journal of Applied Econometrics, 11, 633–648.

Source: AER R Package HousePrices Object. Christian Kleiber and Achim Zeileis. (2008). Applied Econometrics with R. Springer-Verlag, New York.

My online courses are closed for enrollment.
+