feature matrix = matrix where rows correspond to features, and each column is observations of a feature
feature extraction = projecting features to significantly fewer dimensions keeping high-quality models
feature selection = selecting high-quality, informative features and dropping less useful features
Three types of feature selection methods:
- Filter methods select the best features by examining their statistical properties.
- Wrapper methods use trial and error to find the subset of features that produces models with the highest quality predictions.
- Embedded methods select the best feature subset as part of, as an extension of, a learning algorithm's training process.
Model Selection includes Hyperparameter Tuning, or Hyperparameter Optimization.
validation = observations (features and targets) are split into two sets, called the training set and the test set