Machine learning tasks can be grouped into three categories:

1. Supervised learning (SL): Training dataset consisting of both example inputs and desired outputs.
2. Unsupervised learning (UL): Training dataset consisting only of the example inputs and the program must learn the desired outputs itself.
3. Reinforcement learning (RL): Program interacts with a dynamic environment and tries to attain a certain goal e.g. driving a car. The correct input/output pairs are never presented.

Semi-supervised learning is an overlapping subcategory utilizing unlabeled data for the most of the training while having a small proportion of labeled data.

Supervised learning predicts; Unsupervised learning transforms. Prediction is telling the label or probabilities of different labels for given sample. Transform turns sample into different representation, likelihood or cluster identifier.

Machine learning can be broken down into problem categories by output:

• Classification: the output is discrete e.g. gender of a person.
• Regression: the output is continuous value e.g. age of a person. Classification can always be represented as regression, but not the other way around.
• Clustering: a set of inputs is to be divided into groups but the groups are not known beforehand like in classification.
• Density Estimation: finding the distribution of inputs in some space.
• Dimensionality Reduction: the target is to simplify the features set per sample.

Supervised Learning

Supervised learning means that our dataset consisting of both features and labels. Target is to create a program that predicts label of an unlabeled sample.

We have a dataset of images with people; each image also has
that person's gender and age. We use this data to train a program to
predict persons gender and age.

The most important thing with all supervised learning approaches is label quality. If training data labels are perfect, the model will learn to predict it very well.

Unsupervised Learning

Unsupervised learning means that our dataset only has features. Program has to figure out how to label them itself; that is to say, program needs to figure out how are the samples different. Unsupervised learning is used when you have a lot of data but they don't know how to create proper labels.

Given an audio clip with person talking over music, separate the two tracks.

Given a video, isolate a moving object and categorize in relation to other
moving objects which have been seen.

Reinforcement Learning

Reinforcement learning is inspired by behaviorist psychology, concerned with how software ought to take actions in an environment so as to maximize reward or minimize punishment. Reinforcement learning is used when the data is scarce or when you can't clearly define the ideal end state.

Sources

• Wikipedia: Machine Learning
• The Master Algorithm, Pedro Domingos