scikit-learn Onboarding LabChapter VI — model_selection: cross-validation and hyperparameter search✓ 0/5Contents
Chapter VI

model_selection: cross-validation and hyperparameter search

GridSearchCV and RandomizedSearchCV are meta-estimators: they never touch the internals of the wrapped estimator, only its get_params/set_params/clone contract. BaseSearchCV.fit() clones the estimator once as a template, then re-clones and re-configures it via set_params for every (candidate, fold) pair so fits never share mutable state, before finally cloning and re-fitting on the full data with the winning parameters. This module traces that clone-per-fit discipline end to end and shows why it lets a single generic algorithm search over literally any estimator.

Figure I · pipelineHow BaseSearchCV.fit() turns a param grid into best_estimator_
5 stages
Press play — or step through the pipeline stage by stage.
walkthrough

Every (candidate, split) pair gets its own clone

sklearn/model_selection/_search.py · lines 1083–1101
lines 1084–1085

clone(base_estimator) is called once per delayed task — every candidate/fold combination gets its own fresh, unfitted instance instead of reusing or mutating a shared one, which is what makes running these fits in parallel safe.

line 1090

parameters is one candidate's hyperparameter dict (e.g. {'estimator__C': 10}); it rides along to _fit_and_score, which is where it actually gets applied to the clone.

lines 1097–1100

candidate_split_pairs is the cross product of every candidate parameter dict with every CV split, so this comprehension is literally n_candidates * n_splits independent fit-and-score jobs.

                out = parallel(                    delayed(_fit_and_score)(                        clone(base_estimator),                        X,                        y,                        train=train,                        test=test,                        parameters=parameters,                        split_progress=(split_idx, n_splits),                        candidate_progress=(cand_idx, n_candidates),                        **fit_and_score_kwargs,                        caller=self,                        callback_ctx=evaluation_ctx,                    )                    for (                        ((cand_idx, parameters), (split_idx, (train, test))),                        evaluation_ctx,                    ) in zip(candidate_split_pairs, evaluation_contexts)                )
step 1 of 3
    if parameters is not None:        # here we clone the parameters, since sometimes the parameters        # themselves might be estimators, e.g. when we search over different        # estimators in a pipeline.        # ref: https://github.com/scikit-learn/scikit-learn/pull/26786        estimator = estimator.set_params(**clone(parameters, safe=False))
sklearn/model_selection/_validation.py · lines 827–832
Figure II · This is the actual line where the base-API contract gets exercised: the cloned estimator has its candidate's hyperparameters applied with the same plain set_params every estimator implements. GridSearchCV never needs estimator-specific code because it only ever calls clone/get_params/set_params/fit — the parameters themselves are cloned too, since a 'parameter' can itself be a nested estimator (e.g. tuning which classifier a Pipeline step uses).
        if self.refit:            # here we clone the estimator as well as the parameters, since            # sometimes the parameters themselves might be estimators, e.g.            # when we search over different estimators in a pipeline.            # ref: https://github.com/scikit-learn/scikit-learn/pull/26786            self.best_estimator_ = clone(base_estimator).set_params(                **clone(self.best_params_, safe=False)            )
sklearn/model_selection/_search.py · lines 1164–1171
Figure III · The winning hyperparameters are applied exactly the way every candidate's were during the search: clone the original template, set_params with the (cloned) best_params_, then fit — only this time on the whole dataset rather than a single fold. best_estimator_ is what predict/score/transform delegate to afterward.
checkpoint

Check your understanding

Answered in place — nothing is graded, everything is explained. 0 / 3 passed

Why does BaseSearchCV.fit() call clone(base_estimator) separately for every (candidate, split) pair instead of reusing one shared estimator instance across all of them?

Inside _fit_and_score, which method is used to apply a candidate's hyperparameter dict onto the cloned estimator before fitting?

When refit=True, what data does best_estimator_ end up being fit on?