Its the term given to the iterative process of building good features for a better model. Its the process that makes relevant features (using formulas and relations between others).
When do we use it: When we have a refined and optimised model.
What does it involve
- Create new features from existing ones (e.g., ratios, interactions).
- Transform features to better capture non-linear relationships.
- Reduce dimensionality if necessary (e.g., PCA).
Example: Predicting house prices. Raw features might be square footage, number of bedrooms, and location. Feature engineering could involve: Combining square footage and bedrooms into a “living space” feature.