scikit-learn Onboarding LabChapter V — Pipeline: chaining estimators✓ 0/5Contents
Chapter V

Pipeline: chaining estimators

Pipeline is scikit-learn's core composability primitive: it wires a list of (name, estimator) steps into one object that fits/transforms every step but the last, then fits (or predicts/scores/transforms) the final step. It never inspects step types with isinstance — it duck-types via hasattr, so any object exposing fit/transform (or fit/predict) slots in as a black box, which is why Pipeline composes with virtually any scikit-learn-compatible estimator.

Figure I · pipelineHow Pipeline drives its steps
4 stages
Press play — or step through the pipeline stage by stage.
    def _validate_steps(self):        if not self.steps:            raise ValueError("The pipeline is empty. Please add steps.")        names, estimators = zip(*self.steps)        # validate names        self._validate_names(names)        # validate estimators        self._check_estimators_are_instances(estimators)        transformers = estimators[:-1]        estimator = estimators[-1]        for t in transformers:            if t is None or t == "passthrough":                continue            if not (hasattr(t, "fit") or hasattr(t, "fit_transform")) or not hasattr(                t, "transform"            ):                raise TypeError(                    "All intermediate steps should be "                    "transformers and implement fit and transform "                    "or be the string 'passthrough' "                    "'%s' (type %s) doesn't" % (t, type(t))                )        # We allow last estimator to be None as an identity transformation        if (            estimator is not None            and estimator != "passthrough"            and not hasattr(estimator, "fit")        ):            raise TypeError(                "Last step of Pipeline should implement fit "                "or be the string 'passthrough'. "                "'%s' (type %s) doesn't" % (estimator, type(estimator))            )
sklearn/pipeline.py · lines 299–335
Figure II · Pipeline never checks isinstance(t, TransformerMixin). It only asks hasattr(t, 'fit'/'fit_transform') and hasattr(t, 'transform') for intermediate steps, and hasattr(estimator, 'fit') for the last one. Any object shaped like an estimator/transformer is accepted — this is the black-box contract that lets Pipeline compose with custom, third-party, or hand-rolled estimators.
walkthrough

Inside Pipeline.fit(): chain transformers, then fit the last step

sklearn/pipeline.py · lines 589–664
lines 589–592

self._fit(X, y, ...) sequentially fit_transforms every step except the last, returning Xt — the data after all preprocessing.

lines 595–603

If the final estimator isn't 'passthrough', it only ever gets .fit(Xt, y, ...) — never .transform(). The final step is treated purely as a predictor/consumer, not a transformer, even if it happens to have a transform method.

lines 605–606

When the last step is 'passthrough', fitting it is a no-op — the pipeline still finished fitting all real transformers, it just has no trainable final stage.

        routed_params = self._check_method_params(method="fit", props=params)        Xt = self._fit(            X, y, routed_params, raw_params=params, callback_ctx=callback_ctx        )        with _print_elapsed_time("Pipeline", self._log_message(len(self.steps) - 1)):            subcontext = callback_ctx.subcontext(task_name="fit-final-estimator")            if self._final_estimator != "passthrough":                with subcontext.propagate_callback_context(self._final_estimator):                    subcontext.call_on_fit_task_begin(estimator=self, X=Xt, y=y)                    last_step_params = self._get_metadata_for_step(                        step_idx=len(self) - 1,                        step_params=routed_params[self.steps[-1][0]],                        all_params=params,                    )                    self._final_estimator.fit(Xt, y, **last_step_params["fit"])                    subcontext.call_on_fit_task_end(estimator=self, X=Xt, y=y)            else:                subcontext.call_on_fit_task_begin(estimator=self, X=Xt, y=y)                subcontext.call_on_fit_task_end(estimator=self, X=Xt, y=y)
step 1 of 3
        check_is_fitted(self)        Xt = X        if not _routing_enabled():            for _, name, transform in self._iter(with_final=False):                Xt = transform.transform(Xt)            return self.steps[-1][1].predict(Xt, **params)        # metadata routing enabled        routed_params = process_routing(self, "predict", **params)        for _, name, transform in self._iter(with_final=False):            Xt = transform.transform(Xt, **routed_params[name].transform)        return self.steps[-1][1].predict(Xt, **routed_params[self.steps[-1][0]].predict)
sklearn/pipeline.py · lines 758–813
Figure III · predict() loops over every step but the last calling .transform(), then calls .predict() only on self.steps[-1][1] — the exact same shape as fit(): all the work up front is transform, only the tail method differs (predict here, transform/score elsewhere).
checkpoint

Check your understanding

Answered in place — nothing is graded, everything is explained. 0 / 3 passed

What must every intermediate step in a Pipeline implement that the final step is not strictly required to?

How does Pipeline decide whether an object is allowed to be an intermediate step?

Why does pipe.predict_proba() only appear as a callable method when the final estimator supports it?