🕷️ Crawler Inspector

[Back](https://setosa.io/ev/) # Ordinary Least Squares Regression ### Explained Visually [Tweet](https://twitter.com/share) By [Victor Powell](http://twitter.com/vicapow) and [Lewis Lehe](http://twitter.com/lewislehe) ### This explanation is interactive\! *Dials* allow you to adjust scalar values. *Points* that have a gray circle around them are draggable. Statistical regression is basically a way to predict unknown quantities from a batch of existing data. For example, suppose we start out knowing the height and hand size of a bunch of individuals in a "sample population," and that we want to figure out a way to predict hand size from height for individuals not in the sample. By applying OLS, we'll get an equation that takes hand size---the 'independent' variable---as an input, and gives height---the 'dependent' variable---as an output. Below, OLS is done behind-the-scenes to produce the regression equation. The constants in the regression---called 'betas'---are what OLS spits out. Here, beta\_1 is an intercept; it tells what height would be even for a hand size of zero. And beta\_2 is the coefficient on hand size; it tells how much taller we should expect someone to be for a given increment in their hand size. Drag the sample data to see the betas change. 0 20 40 60 80 100 hand size 0 20 40 60 80 100 height Beta 1 - The y-intercept of the regression line. Beta 2 - The slope of the regression line. 5\.00 \+ 0\.81 \* hand size = height At some point, you probably asked your parents, "Where do betas come from?" Let's raise the curtain on how OLS finds its betas. Error is the difference between prediction and reality: the vertical distance between a real data point and the regression line. OLS is concerned with the *squares* of the errors. It tries to find the line going through the sample data that minimizes the sum of the squared errors. Below, the squared errors are represented as squares, and your job is to choose betas (the slope and intercept of the regression line) so that the total area of all the squares (the sum of the squared errors) is as small as possible. That's OLS\! 0\.00 \+ 1\.00 \* hand size = height 0 20 40 60 80 100 x 0 20 40 60 80 100 y Now, real scientists and even sociologists rarely do regression with just one independent variable, but OLS works exactly the same with more. Below is OLS with two independent variables. Instead of the errors being relative to a line, though, they're now relative to a plane in 3D space. So now the job of OLS is to find the equation for that plane. The slice of the plane through each axis is shown in the first two figures. 0 20 40 60 80 100 x1 0 20 40 60 80 100 y 0 20 40 60 80 100 x2 0 20 40 60 80 100 By playing with the dots, you can see that, when there are multiple variables involved, the true relationships can be very counterintuitive. That's why we have statistics: to make us unsure about things. Below, see if you can choose the betas to minimize the sum of squared errors. There are many other prediction techniques much more complicated than OLS, like logistic regression, weighted least-squares regression, robust regression and the growing family of non-parametric methods. 50\.00 \+ 0\.00 \* hand size \+ 0\.00 \* hand size \= height 0 20 40 60 80 100 x1 0 20 40 60 80 100 y 0 20 40 60 80 100 x2 0 20 40 60 80 100 Special thanks to [Ian Johnson](http://twitter.com/enjalot) for reviewing an earlier version of this explorable explanation and suggesting the idea of using GIFs to explain how the controls work. For more explanations, visit the Explained Visually [project homepage.](https://setosa.io/ev/) Or subscribe to our mailing list. Please enable JavaScript to view the [comments powered by Disqus.](http://disqus.com/?ref_noscript)

### This explanation is interactive\! *Dials* allow you to adjust scalar values. *Points* that have a gray circle around them are draggable. Statistical regression is basically a way to predict unknown quantities from a batch of existing data. For example, suppose we start out knowing the height and hand size of a bunch of individuals in a "sample population," and that we want to figure out a way to predict hand size from height for individuals not in the sample. By applying OLS, we'll get an equation that takes hand size---the 'independent' variable---as an input, and gives height---the 'dependent' variable---as an output. Below, OLS is done behind-the-scenes to produce the regression equation. The constants in the regression---called 'betas'---are what OLS spits out. Here, beta\_1 is an intercept; it tells what height would be even for a hand size of zero. And beta\_2 is the coefficient on hand size; it tells how much taller we should expect someone to be for a given increment in their hand size. Drag the sample data to see the betas change. 0 20 40 60 80 100 hand size 0 20 40 60 80 100 height Beta 1 - The y-intercept of the regression line. Beta 2 - The slope of the regression line. 5\.00 \+ 0\.81 \* hand size = height At some point, you probably asked your parents, "Where do betas come from?" Let's raise the curtain on how OLS finds its betas. Error is the difference between prediction and reality: the vertical distance between a real data point and the regression line. OLS is concerned with the *squares* of the errors. It tries to find the line going through the sample data that minimizes the sum of the squared errors. Below, the squared errors are represented as squares, and your job is to choose betas (the slope and intercept of the regression line) so that the total area of all the squares (the sum of the squared errors) is as small as possible. That's OLS\! 0\.00 \+ 1\.00 \* hand size = height 0 20 40 60 80 100 x 0 20 40 60 80 100 y Now, real scientists and even sociologists rarely do regression with just one independent variable, but OLS works exactly the same with more. Below is OLS with two independent variables. Instead of the errors being relative to a line, though, they're now relative to a plane in 3D space. So now the job of OLS is to find the equation for that plane. The slice of the plane through each axis is shown in the first two figures. 0 20 40 60 80 100 x1 0 20 40 60 80 100 y 0 20 40 60 80 100 x2 0 20 40 60 80 100 By playing with the dots, you can see that, when there are multiple variables involved, the true relationships can be very counterintuitive. That's why we have statistics: to make us unsure about things. Below, see if you can choose the betas to minimize the sum of squared errors. There are many other prediction techniques much more complicated than OLS, like logistic regression, weighted least-squares regression, robust regression and the growing family of non-parametric methods. 50\.00 \+ 0\.00 \* hand size \+ 0\.00 \* hand size \= height 0 20 40 60 80 100 x1 0 20 40 60 80 100 y 0 20 40 60 80 100 x2 0 20 40 60 80 100 Special thanks to [Ian Johnson](http://twitter.com/enjalot) for reviewing an earlier version of this explorable explanation and suggesting the idea of using GIFs to explain how the controls work.