Overfitting and Underfitting
The system will only be capable of learning if the training data contains enough relevant features and not too many irrelevant ones. A critical part of the success of a Machine Learning project is coming up with a good set of features to train on. This process is called feature engineering, which involves Feature selection (selecting the most useful features to train on among existing features) and Feature extraction (combining existing features to produce a more useful one).
Overfitting
Overfitting means that the model performs well on the training data, but it does not generalize well. Complex models can detect subtle patterns in the data, but if the training set is noisy, or if it is too small, then the model is likely to detect patterns in the noise itself. Obviously these patterns will not generalize to new instances.
Constraining a model to make it simpler and reduce the risk of overfitting is called regularization. The amount of regularization to apply during learning can be controlled by a hyperparameter. A hyperparameter is a parameter of a learning algorithm. As such, it is not affected by the learning algorithm itself; it must be set prior to training and remains constant during training. Tuning hyperparameters is an important part of building a Machine Learning system.
Overfitting happens when the model is too complex relative to the amount and noisiness of the training data. There are possible solutions: simplify the model by selecting one with fewer parameters by reducing the number of attributes in the training data, or by constraining the model; gather more training data and reduce the noise in the training data.
Underfitting
Underfitting is the opposite of overfitting: it occurs when the model is too simple to learn the underlying structure of the data.
There are the main options for fixing this problem: select a more powerful model, with more parameters; feed better features to the learning algorithm (feature engineering); reduce the constraints on the model.
