Regressions

Updated at 2017-11-13 14:39

Regression analysis is a statistical process for estimating the relationships among variables.

In regression machine learning problems, the desired output is a continuous number e.g. age of a person.

Simple Linear Regression: a linear predictor function with one input variable e.g. (y = x^2).
Multiple Linear Regression: a linear predictor function with more than one input variable.
Ordinal Regression: model output is an ordinal variables thus have an order but unknown distance between themselves e.g. "good", "ok", "poor".
Nonlinear Regression: the predictor function is nonlinear such as exponentiation^2. There may be multiple local minima to optimize.
General Linear Model: a matrix formula (y = xb + u) which has input matrix x, output matrix y, model coefficiency matrix (b) and error matrix u. b works like neural network weights and u like bias, finding the right matrix values for those will allow creating predictions.
Generalized Linear Model (GLM): allows for outputs that have error distribution models other than a normal distribution by allowing the linear model to be related to the response variable via a link function.

Logistic Regressions

Logistic Regression predictions are categorical.
Binary Logistic Regression predictions are either 0 or 1.
Multinomial Logistic Regression predictions can be more than 2 possible discrete outcomes.
Ordered Logistic Regression predictions are ordinal e.g. "poor", "fair" and "good".