en.m.wikipedia.org
Residual sum of squares
This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.
Find sources: "Residual sum of squares" – news · newspapers · books · scholar · JSTOR(April 2013) (Learn how and when to remove this template message)
In statistics, the residual sum of squares (RSS), also known as the sum of squared residuals (SSR) or the sum of squared estimate of errors (SSE), is the sum of the squares of residuals (deviations predicted from actual empirical values of data). It is a measure of the discrepancy between the data and an estimation model, such as a linear regression. A small RSS indicates a tight fit of the model to the data. It is used as an optimality criterion in parameter selection and model selection.
In general, total sum of squares = explained sum of squares + residual sum of squares. For a proof of this in the multivariate ordinary least squares (OLS) case, see partitioning in the general OLS model.
One explanatory variable
In a model with a single explanatory variable, RSS is given by:
where yi is the ith value of the variable to be predicted, xi is the ith value of the explanatory variable, and
is the predicted value of yi (also termed
). In a standard linear simple regression model,
, where and are coefficients, y and x are the regressand and the regressor, respectively, and ε is the error term. The sum of squares of residuals is the sum of squares of
; that is
where
is the estimated value of the constant term and
is the estimated value of the slope coefficient .
Matrix expression for the OLS residual sum of squares
The general regression model with n observations and k explanators, the first of which is a constant unit vector whose coefficient is the regression intercept, is
where y is an n × 1 vector of dependent variable observations, each column of the n × k matrix X is a vector of observations on one of the k explanators, is a k × 1 vector of true coefficients, and e is an n× 1 vector of the true underlying errors. The ordinary least squares estimator for is
The residual vector =
; so the residual sum of squares is:
,
(equivalent to the square of the norm of residuals). In full:
,
where H is the hat matrix, or the projection matrix in linear regression.
Relation with Pearson's product-moment correlation
The least-squares regression line is given by
,
where
and
, where
and
Therefore,
where
The Pearson product-moment correlation is given by
therefore,
See also
References
^ Archdeacon, Thomas J. (1994). Correlation and regression analysis : a historian's guide. University of Wisconsin Press. pp. 161–162. ISBN 0-299-13650-7. OCLC 27266095.
Draper, N.R.; Smith, H. (1998). Applied Regression Analysis (3rd ed.). John Wiley. ISBN 0-471-17082-8.
Last edited on 17 May 2021, at 22:40
Content is available under CC BY-SA 3.0 unless otherwise noted.