Its the term given to the iterative process of building good features for a better model. Its the process that makes relevant features (using formulas and relations between others).
We use it when we have a refined and optimised model.
What does it involve
- Create new features from existing ones (e.g., ratios, interactions).
- Transform features to better capture non-linear relationships.
- Dimensionality Reduction if necessary.
The main techniques of feature engineering:
- are selection (picking subset),
- learning (picking the best),
- extraction and combination(combining).
Example: Predicting house prices. Raw features might be square footage, number of bedrooms, and location. Feature engineering could involve: Combining square footage and bedrooms into a “living space” feature.
Example:
- Decompose datetime information into separate features for date and time to capture their respective predictive powers.
Interaction terms: Feature Engineering:
- Definition and Purpose: Interaction terms are new features created by combining existing ones to capture the interaction effects between them, improving model accuracy.
- Common Methods: Multiplication (e.g., square footage * number of bedrooms) and division (e.g., price per square foot) are common ways to create interaction terms.
- Benefits: They help uncover complex patterns, tackle non-linearities, and enhance the model’s ability to learn how features influence each other.
- Application: Use domain knowledge to identify meaningful interactions and start with simple, pairwise interactions to avoid overfitting.