ProbeFeatureSelection#
- class feature_engine.selection.ProbeFeatureSelection(estimator, variables=None, scoring='roc_auc', n_probes=1, distribution='normal', cv=5, random_state=0, confirm_variables=False)[source]#
ProbeFeatureSelection() generates one or more probe features based on the user-selected distribution. The distribution options are ‘normal’, ‘binomial’, ‘uniform’, or ‘all’. ‘all’ creates at least one distribution for each of the three aforementioned distributions.
Using cross validation, the class fits a Scikit-learn estimator to the provided dataset’s variables and the probe features.
The class derives the feature importance for each variable and probe feature. In the case of there being more than one probe feature, ProbeFeatureSelection() calculates the average feature importance of all the probe features.
The variables that have a feature importance less than the feature importance or average feature importance of the probe feature(s) are dropped from the dataset.
More details in the User Guide.
- Parameters
- estimator: object
A Scikit-learn estimator for regression or classification. The estimator must have either a
feature_importances
or acoef_
attribute after fitting.- variables: str or list, default=None
The list of variables to evaluate. If None, the transformer will evaluate all numerical features in the dataset.
- scoring: str, default=’roc_auc’
Metric to evaluate the performance of the estimator. Comes from sklearn.metrics. See the model evaluation documentation for more options: https://scikit-learn.org/stable/modules/model_evaluation.html
- n_probes: int, default=1
Number of probe features to be created. If distribution is ‘all’, n_probes must be a multiple of 3.
- distribution: str, default=’normal’
The distribution used to create the probe features. The options are ‘normal’, ‘binomial’, ‘uniform’, and ‘all’. ‘all’ creates at least 1 or more probe features comprised of each distribution type, i.e., normal, binomial, and uniform. The remaining options create
n_probes
features of the selected distribution.- cv: int, cross-validation generator or an iterable, default=3
Determines the cross-validation splitting strategy. Possible inputs for cv are:
None, to use cross_validate’s default 5-fold cross validation
int, to specify the number of folds in a (Stratified)KFold,
CV splitter: (https://scikit-learn.org/stable/glossary.html#term-CV-splitter)
An iterable yielding (train, test) splits as arrays of indices.
For int/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used. These splitters are instantiated with
shuffle=False
so the splits will be the same across calls. For more details check Scikit-learn’scross_validate
’s documentation.
- Attributes
- probe_features_:
A dataframe comprised of the pseudo-randomly generated features based on the selected distribution.
- feature_importances_:
Pandas Series with the feature importance.
- features_to_drop_:
List with the features that will be removed.
- variables_:
The variables that will be considered for the feature selection procedure.
- feature_names_in_:
List with the names of features seen during
fit
.- n_features_in_:
The number of features in the train set used in fit.
References
- 1
Stoppiglia, et al. “Ranking a Random Feature for Variable and Feature Selection”. JMLR: 1399-1414, 2003 https://jmlr.org/papers/volume3/stoppiglia03a/stoppiglia03a.pdf
Examples
>>> from sklearn.datasets import load_breast_cancer >>> from sklearn.linear_model import LogisticRegression >>> from feature_engine.selection import ProbeFeatureSelection >>> X, y = load_breast_cancer(return_X_y=True, as_frame=True) >>> sel = ProbeFeatureSelection( >>> estimator=LogisticRegression(), >>> scoring="roc_auc", >>> n_probes=3, >>> distribution="normal", >>> cv=3, >>> random_state=150, >>> ) >>> X_tr = sel.fit_transform(X, y) print(X.shape, X_tr.shape)
Methods
fit:
Find the important features.
fit_transform:
Fit to data, then transform it.
get_feature_names_out:
Get output feature names for transformation.
get_params:
Get parameters for this estimator.
set_params:
Set the parameters of this estimator.
get_support:
Get a mask, or integer index, of the features selected.
transform:
Reduce X to the selected features.
- fit(X, y)[source]#
Find the important features.
- Parameters
- X: pandas dataframe of shape = [n_samples, n_features]
- y: array-like of shape (n_samples)
Target variable. Required to train the estimator.
- fit_transform(X, y=None, **fit_params)[source]#
Fit to data, then transform it.
Fits transformer to
X
andy
with optional parametersfit_params
and returns a transformed version ofX
.- Parameters
- Xarray-like of shape (n_samples, n_features)
Input samples.
- yarray-like of shape (n_samples,) or (n_samples, n_outputs), default=None
Target values (None for unsupervised transformations).
- **fit_paramsdict
Additional fit parameters.
- Returns
- X_newndarray array of shape (n_samples, n_features_new)
Transformed array.
- get_feature_names_out(input_features=None)[source]#
Get output feature names for transformation. In other words, returns the variable names of transformed dataframe.
- Parameters
- input_featuresarray or list, default=None
This parameter exits only for compatibility with the Scikit-learn pipeline.
If
None
, thenfeature_names_in_
is used as feature names in.If an array or list, then
input_features
must matchfeature_names_in_
.
- Returns
- feature_names_out: list
Transformed feature names.
- get_metadata_routing()[source]#
Get metadata routing of this object.
Please check User Guide on how the routing mechanism works.
- Returns
- routingMetadataRequest
A
MetadataRequest
encapsulating routing information.
- get_params(deep=True)[source]#
Get parameters for this estimator.
- Parameters
- deepbool, default=True
If True, will return the parameters for this estimator and contained subobjects that are estimators.
- Returns
- paramsdict
Parameter names mapped to their values.
- get_support(indices=False)[source]#
Get a mask, or integer index, of the features selected.
- Parameters
- indicesbool, default=False
If True, the return value will be an array of integers, rather than a boolean mask.
- Returns
- supportarray
An index that selects the retained features from a feature vector. If
indices
is False, this is a boolean array of shape [# input features], in which an element is True if its corresponding feature is selected for retention. Ifindices
is True, this is an integer array of shape [# output features] whose values are indices into the input feature vector.
- set_params(**params)[source]#
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as
Pipeline
). The latter have parameters of the form<component>__<parameter>
so that it’s possible to update each component of a nested object.- Parameters
- **paramsdict
Estimator parameters.
- Returns
- selfestimator instance
Estimator instance.