Feature Filtering

Feature filtering using a mlr3filters::Filter object, see the mlr3filters package.

If a Filter can only operate on a subset of columns based on column type, then only these features are considered and filtered. nfeat and frac will count for the features of the type that the Filter can operate on; this means e.g. that setting nfeat to 0 will only remove features of the type that the Filter can work with.

Format

R6Class object inheriting from PipeOpTaskPreprocSimple/PipeOpTaskPreproc/PipeOp.

Construction

PipeOpFilter$new(filter, id = filter$id, param_vals = list())

filter :: Filter
Filter used for feature filtering. This argument is always cloned; to access the Filter inside PipeOpFilter by-reference, use $filter.
id :: character(1) Identifier of the resulting object, defaulting to the id of the Filter being used.
param_vals :: named list
List of hyperparameter settings, overwriting the hyperparameter settings that would otherwise be set during construction. Default list().

Input and Output Channels

Input and output channels are inherited from PipeOpTaskPreproc.

The output is the input Task with features removed that were filtered out.

State

The $state is a named list with the $state elements inherited from PipeOpTaskPreproc, as well as:

scores :: named numeric
Scores calculated for all features of the training Task which are being used as cutoff for feature filtering. If frac or nfeat is given, the underlying Filter may choose to not calculate scores for all features that are given. This only includes features on which the Filter can operate; e.g. if the Filter can only operate on numeric features, then scores for factorial features will not be given.
features :: character
Names of features that are being kept. Features of types that the Filter can not operate on are always being kept.

Parameters

The parameters are the parameters inherited from the PipeOpTaskPreproc, as well as the parameters of the Filter used by this object. Besides, parameters introduced are:

filter.nfeat :: numeric(1)
Number of features to select. Mutually exclusive with frac, cutoff, and permuted.
filter.frac :: numeric(1)
Fraction of features to keep. Mutually exclusive with nfeat, cutoff, and permuted.
filter.cutoff :: numeric(1)
Minimum value of filter heuristic for which to keep features. Mutually exclusive with nfeat, frac, and permuted.
filter.permuted :: integer(1)
If this parameter is set, a random permutation of each feature is added to the task before applying the filter. All features selected before the permuted-th permuted features is selected are kept. This is similar to the approach in Wu (2007) and Thomas (2017). Mutually exclusive with nfeat, frac, and cutoff.

Note that at least one of filter.nfeat, filter.frac, filter.cutoff, and filter.permuted must be given.

Internals

This does not use the $.select_cols feature of PipeOpTaskPreproc to select only features compatible with the Filter; instead the whole Task is used by private$.get_state() and subset internally.

Fields

Fields inherited from PipeOpTaskPreproc, as well as:

filter :: Filter
Filter that is being used for feature filtering. Do not use this slot to get to the feature filtering scores after training; instead, use $state$scores. Read-only.

Methods

Methods inherited from PipeOpTaskPreprocSimple/PipeOpTaskPreproc/PipeOp.

References

Wu Y, Boos DD, Stefanski LA (2007). “Controlling Variable Selection by the Addition of Pseudovariables.” Journal of the American Statistical Association, 102(477), 235–243. doi:10.1198/016214506000000843 .

Thomas J, Hepp T, Mayr A, Bischl B (2017). “Probing for Sparse and Fast Variable Selection with Model-Based Boosting.” Computational and Mathematical Methods in Medicine, 2017, 1–8. doi:10.1155/2017/1421409 .

Examples

library("mlr3")
library("mlr3filters")

# setup PipeOpFilter to keep the 5 most important
# features of the spam task w.r.t. their AUC
task = tsk("spam")
filter = flt("auc")
po = po("filter", filter = filter)
po$param_set
#> <ParamSetCollection(5)>
#>                 id    class lower upper nlevels        default  value
#>             <char>   <char> <num> <num>   <num>         <list> <list>
#> 1:    filter.nfeat ParamInt     0   Inf     Inf <NoDefault[0]> [NULL]
#> 2:     filter.frac ParamDbl     0     1     Inf <NoDefault[0]> [NULL]
#> 3:   filter.cutoff ParamDbl  -Inf   Inf     Inf <NoDefault[0]> [NULL]
#> 4: filter.permuted ParamInt     1   Inf     Inf <NoDefault[0]> [NULL]
#> 5:  affect_columns ParamUty    NA    NA     Inf  <Selector[1]> [NULL]
po$param_set$values$filter.nfeat = 5

# filter the task
filtered_task = po$train(list(task))[[1]]

# filtered task + extracted AUC scores
filtered_task$feature_names
#> [1] "capitalAve"      "capitalLong"     "charDollar"      "charExclamation"
#> [5] "your"           
head(po$state$scores, 10)
#> charExclamation     capitalLong      capitalAve            your      charDollar 
#>       0.3290461       0.3041626       0.2882004       0.2801659       0.2721394 
#>    capitalTotal            free             our             you          remove 
#>       0.2622801       0.2327285       0.2109325       0.2104681       0.2031303 

# feature selection embedded in a 3-fold cross validation
# keep 30% of features based on their AUC score
task = tsk("spam")
gr = po("filter", filter = flt("auc"), filter.frac = 0.5) %>>%
  po("learner", lrn("classif.rpart"))
learner = GraphLearner$new(gr)
rr = resample(task, learner, rsmp("holdout"), store_models = TRUE)
rr$learners[[1]]$model$auc$scores
#>   charExclamation       capitalLong              your        capitalAve 
#>      0.3238152005      0.3042322800      0.2875033175      0.2871065413 
#>        charDollar      capitalTotal              free               you 
#>      0.2764362681      0.2662379684      0.2320238685      0.2143728547 
#>               our            remove             money               all 
#>      0.2098176687      0.2046412205      0.1818790031      0.1816593829 
#>                hp            num000          business              over 
#>      0.1748898581      0.1585216214      0.1501428748      0.1415560087 
#>          internet            george              mail               hpl 
#>      0.1348769419      0.1342355533      0.1334216533      0.1326369475 
#>             email           receive           address             order 
#>      0.1325690488      0.1315892724      0.1237574755      0.1168579037 
#>              make           num1999          charHash            credit 
#>      0.1089995665      0.1046412205      0.0998692894      0.0988800028 
#>              will            people         addresses              labs 
#>      0.0958661400      0.0913733235      0.0742654995      0.0736150430 
#>               edu            num650             num85               lab 
#>      0.0700172954      0.0699748310      0.0686686100      0.0592269277 
#>        technology           meeting            telnet              data 
#>      0.0532025196      0.0529178757      0.0527411621      0.0435513907 
#>                pm            report           project charSquarebracket 
#>      0.0391912753      0.0380785767      0.0378828426      0.0351343820 
#>            num857            num415          original        conference 
#>      0.0343571959      0.0325759050      0.0297613150      0.0283529141 
#>                cs     charSemicolon                re              font 
#>      0.0268617962      0.0264183534      0.0260759846      0.0222261050 
#>  charRoundbracket            direct             num3d             table 
#>      0.0219286334      0.0131568704      0.0093397236      0.0053856736 
#>             parts 
#>      0.0007809459