REGRESSION

It’s a supervised task used on numeric variables with the objective of minimize the error of the prediction using the other variables for the prediction

LINEAR REGRESSION

given a data set $X$ with $N$ rows and $D$ columns:

• $x_i$ is a $D$ dimensional data element response vector $y$ with $N$ values $y_i$
• $w$ is a $D$-dimensional vector of coefficients that needs to be learned

so the relation between the $yi$ element and the $xi$ elements is modeled

$$y_i=w^T\ast x_i\forall i\in [\mathrm{1...}N]$$

so the forecast is given by

$$y^f=X\ast w^T$$

QUALITY INDICATORS

• Mean of the observed data

$$y^{avg}=\frac{1}{N}\ast \sum _{i}yi$$

• Sum of squared residuals $S{S}_{res}$

$$S{S}_{res}=\sum _i(yi-y{i}^f{)}^2$$

• Total sum of squares $S{S}_{tot}$

$$S{S}_{tot}=\sum _i(yi-y{i}^{avg}{)}^2$$

• Coefficient of determination

$$R^2=1-\frac{S{S}_{res}}{S{S}_{tot}}$$

the Coefficient of determination compares the chosen model with that of a horizontal straight line

if the model does not follow the trend of the data the $R^2$ value can be also negative

when the number of feature is high overfitting is possible

POLYNOMIAL REGRESSION

the target is influenced by a single feature and the relationship can’t be describe by a straight line

PREVIOUS NEXT