Explained sum of squares
From Wikipedia, the free encyclopedia
This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these template messages)
This article needs attention from an expert in statistics. (September 2009)
This article includes a list of general references, but it remains largely unverified because it lacks sufficient corresponding inline citations.(December 2010)
In statistics, the explained sum of squares (ESS), alternatively known as the model sum of squares or sum of squares due to regression (SSR – not to be confused with the residual sum of squares (RSS) or sum of squares of errors), is a quantity used in describing how well a model, often a regression model, represents the data being modelled. In particular, the explained sum of squares measures how much variation there is in the modelled values and this is compared to the total sum of squares (TSS), which measures how much variation there is in the observed data, and to the residual sum of squares, which measures the variation in the error between the observed data and modelled values.
Contents
1Definition
2Partitioning in simple linear regression
2.1Simple derivation
3Partitioning in the general ordinary least squares model
4See also
5Notes
6References
Definition​[​edit​]
The explained sum of squares (ESS) is the sum of the squares of the deviations of the predicted values from the mean value of a response variable, in a standard regression model — for example, yi = a + b1x1i + b2x2i + ... + εi, where yi is the ith observation of the response variable, xji is the i th observation of the j th explanatory variable, a and bj are coefficients​, i indexes the observations from 1 to n, and εi is the i th value of the error term. In general, the greater the ESS, the better the estimated model performs.
If and
are the estimated coefficients​, then
is the i th predicted value of the response variable. The ESS is then:
where
the value estimated by the regression line .[1]
In some cases (see below): total sum of squares (TSS) = explained sum of squares (ESS) + residual sum of squares (RSS).
Partitioning in simple linear regression​[​edit​]
The following equality, stating that the total sum of squares (TSS) equals the residual sum of squares (=SSE : the sum of squared errors of prediction) plus the explained sum of squares (SSR :the sum of squares due to regression or explained sum of squares), is generally true in simple linear regression:
Simple derivation​[​edit​]
Square both sides and sum over all i:
Here is how the last term above is zero from simple linear regression​[2]
So,
Therefore,
Partitioning in the general ordinary least squares model​[​edit​]
The general regression model with n observations and k explanators, the first of which is a constant unit vector whose coefficient is the regression intercept, is
where y is an n × 1 vector of dependent variable observations, each column of the n × k matrix X is a vector of observations on one of the k explanators, is a k × 1 vector of true coefficients, and e is an n × 1 vector of the true underlying errors. The ordinary least squares estimator for is
The residual vector is
, so the residual sum of squares
is, after simplification,
Denote as the constant vector all of whose elements are the sample mean
of the dependent variable values in the vector y. Then the total sum of squares is
The explained sum of squares, defined as the sum of squared deviations of the predicted values from the observed mean of y, is
Using
in this, and simplifying to obtain
, gives the result that TSS = ESS + RSS if and only if
. The left side of this is
times the sum of the elements of y, and the right side is
times the sum of the elements of , so the condition is that the sum of the elements of y equals the sum of the elements of , or equivalently that the sum of the prediction errors (residuals)
is zero. This can be seen to be true by noting the well-known OLS property that the k × 1 vector
: since the first column of X is a vector of ones, the first element of this vector
is the sum of the residuals and is equal to zero. This proves that the condition holds for the result that TSS = ESS + RSS.
In linear algebra terms, we have
,
,
. The proof can be simplified by noting that
. The proof is as follows:
Thus,
which again gives the result that TSS = ESS + RSS, since
.
See also​[​edit​]
Notes​[​edit​]
  1. ^ "Sum of Squares - Definition, Formulas, Regression Analysis". Corporate Finance Institute. Retrieved 2020-06-11​.
  2. ^ Mendenhall, William (2009). Introduction to Probability and Statistics (13th ed.). Belmont, CA: Brooks/Cole. p. 507. ISBN 9780495389538​.
References​[​edit​]
Categories​: Least squares
Search
This page was last edited on 20 July 2021, at 03:55 (UTC).
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.
Privacy policy
About Wikipedia
Disclaimers
Contact Wikipedia
Mobile view
Developers
Statistics
Cookie statement
TalkContributionsCreate accountLog in
ArticleTalk
ReadEditView history
Visit the main pageMain pageContentsCurrent eventsRandom articleAbout WikipediaContact usDonateHelpLearn to editCommunity portalRecent changesUpload fileWhat links hereRelated changesUpload fileSpecial pagesPermanent linkPage informationCite this pageWikidata itemDownload as PDFPrintable versionDeutschItalianoEdit links