DropCorrelatedFeatures#
- class feature_engine.selection.DropCorrelatedFeatures(variables=None, method='pearson', threshold=0.8, missing_values='ignore', confirm_variables=False)[source]#
DropCorrelatedFeatures() finds and removes correlated features. Correlation is calculated with
pandas.corr(). Features are removed on first found first removed basis, without any further insight.DropCorrelatedFeatures() works only with numerical variables. Categorical variables will need to be encoded to numerical or will be excluded from the analysis.
More details in the User Guide.
- Parameters
- variables: str or list, default=None
The list of variables to evaluate. If None, the transformer will evaluate all numerical features in the dataset.
- method: string or callable, default=’pearson’
Can take ‘pearson’, ‘spearman’, ‘kendall’ or callable. It refers to the correlation method to be used to identify the correlated features.
‘pearson’: standard correlation coefficient
‘kendall’: Kendall Tau correlation coefficient
‘spearman’: Spearman rank correlation
callable: callable with input two 1d ndarrays and returning a float.
For more details on this parameter visit the
pandas.corr()documentation.- threshold: float, default=0.8
The correlation threshold above which a feature will be deemed correlated with another one and removed from the dataset.
- missing_values: str, default=ignore
Whether the missing values should be raised as error or ignored when determining correlation. Takes values ‘raise’ and ‘ignore’.
- confirm_variables: bool, default=False
If set to True, variables that are not present in the input dataframe will be removed from the list of variables. Only used when passing a variable list to the parameter
variables. See parameter variables for more details.
- Attributes
- features_to_drop_:
Set with the correlated features that will be dropped.
- correlated_feature_sets_:
Groups of correlated features. Each list is a group of correlated features.
- variables_:
The variables that will be considered for the feature selection procedure.
- feature_names_in_:
List with the names of features seen during
fit.- n_features_in_:
The number of features in the train set used in fit.
See also
pandas.corrfeature_engine.selection.SmartCorrelationSelection
Notes
If you want to select from each group of correlated features those that are perhaps more predictive or more complete, check Feature-engine’s SmartCorrelationSelection.
Methods
fit:
Find correlated features.
fit_transform:
Fit to data, then transform it.
get_feature_names_out:
Get output feature names for transformation.
get_params:
Get parameters for this estimator.
set_params:
Set the parameters of this estimator.
transform:
Remove correlated features.
- fit(X, y=None)[source]#
Find the correlated features.
- Parameters
- Xpandas dataframe of shape = [n_samples, n_features]
The training dataset.
- ypandas series. Default = None
y is not needed in this transformer. You can pass y or None.
- fit_transform(X, y=None, **fit_params)[source]#
Fit to data, then transform it.
Fits transformer to
Xandywith optional parametersfit_paramsand returns a transformed version ofX.- Parameters
- Xarray-like of shape (n_samples, n_features)
Input samples.
- yarray-like of shape (n_samples,) or (n_samples, n_outputs), default=None
Target values (None for unsupervised transformations).
- **fit_paramsdict
Additional fit parameters.
- Returns
- X_newndarray array of shape (n_samples, n_features_new)
Transformed array.
- get_feature_names_out(input_features=None)[source]#
Get output feature names for transformation.
- input_features: None
This parameter exists only for compatibility with the Scikit-learn pipeline, but has no functionality. You can pass a list of feature names or None.
- Returns
- feature_names_out: list
The feature names.
- :rtype:py:class:
~typing.List
- get_params(deep=True)[source]#
Get parameters for this estimator.
- Parameters
- deepbool, default=True
If True, will return the parameters for this estimator and contained subobjects that are estimators.
- Returns
- paramsdict
Parameter names mapped to their values.
- set_params(**params)[source]#
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as
Pipeline). The latter have parameters of the form<component>__<parameter>so that it’s possible to update each component of a nested object.- Parameters
- **paramsdict
Estimator parameters.
- Returns
- selfestimator instance
Estimator instance.
- transform(X)[source]#
Return dataframe with selected features.
- Parameters
- X: pandas dataframe of shape = [n_samples, n_features].
The input dataframe.
- Returns
- X_new: pandas dataframe of shape = [n_samples, n_selected_features]
Pandas dataframe with the selected features.
- :rtype:py:class:
~pandas.core.frame.DataFrame