Training and Loss

Training a model just means reducing loss in a process called empirical risk minimization. Loss is a number indicating how bad your model predicted when compared to a single known example.

Squared Loss

Also L2 loss

= the square of the difference between the label and the prediction
= (observation - prediction(x))2
= (y - y')2

Also Mean squared error (MSE)

$$MSE = 1/N \sum(y-prediction(x))^2$$

Where: (x,y) is an example in which x is the set of features and y is the label prediction(x) is a function of the weights and bias in combination with the set of features x D is a data set containing many labeled examples, which are (x,y) pairs N is the number of examples in D