[FIXED] how does a recursive feature elimination decide to support a feature?

Issue

what is the base algorithm used in RFE and how does RFE decides which feature to support and which feature not to support?

Solution

Before the explanation let is fix some facts:

1. RFE requires a base algorithm that assigns weights to features. This could be any algorithm that provides feature importance or coefficients, such as linear regression, support vector machines, or decision trees.

2. Features are ranked based on their importance scores. Features with lower importance are considered less essential.

3. Remove the least important feature(s)

Then, the specific details of how RFE decides which feature to support or eliminate depend on the underlying algorithm used for modeling. For example:

• Linear Models (e.g., Linear Regression): RFE might eliminate features with the smallest coefficients, as they contribute less to the linear combination.

• Tree-based Models (e.g., Decision Trees, Random Forests): RFE might use feature importances provided by the decision tree or random forest algorithm. Features with lower importances could be removed.

Answered By - Daniel Perez Efremova