=================================================================================

Bayesian optimization takes into account past evaluations when choosing the hyperparameter set to evaluate next. By choosing its parameter combinations in an informed way, it enables itself to focus on those areas of the parameter space that it believes will bring the most promising validation scores. Therefore it:
          i) enables itself to focus on those areas of the parameter space that it believes will bring the most promising validation scores.
          ii) requires less iterations to get to the optimal set of hyperparameter values.
          iii) limits the number of times a model needs to be trained for validation.

Figure 3750 shows application of Bayesian optimization. It illustrates how Bayesian optimization can be applied in hyperparameter tuning, using the scikit-optimize library, for a machine learning model. It demonstrates the use of the scikit-optimize library to search for optimal hyperparameters of a Random Forest classifier.

Figure 3750. Application of Bayesian optimization (Code).

Here, the parameter search space is defined by:

          param_space = {
          "n_estimators": (10, 200),
          "max_depth": (1, 50),
          "min_samples_split": (2, 10),
          "min_samples_leaf": (1, 10),
          }

This specifies the search space for hyperparameters. For example, "n_estimators" can vary between 10 and 200.

============================================