[FIXED] sklearn subset fitted pipeline - reuse for transform

Issue

I have constructed a pipeline with several steps which takes some time to fit. For debugging I would like to be able to inspect subsets of that pipeline (e.g. {pipe step 1-3}.transform(X)).

I know that I can use Pipe(pipe.named_steps[:3]) to extract a subset and construct a new pipeline from it. Unfortunately I have to refit the pipeline before calling transform on it.

Is there a way to avoid the refit?

Solution

You can access subparts of a Pipeline object by indexing it like a normal list, e.g. pipe[:3]. This will return a new, yet unfitted Pipeline instance. Interestingly though, its components are fitted.

In consequence, a check with scikit-learn's check_is_fitted function would raise an error. However, you can nonetheless call pipe[:3].transform(X) which will still work if you have fit the whole pipeline before.

Answered By - afsharov