当前位置：网站首页>随机森林、LGBM基于贝叶斯优化调参
随机森林、LGBM基于贝叶斯优化调参
2022-07-24 05:24:00 【羽星_s】
前言
- 本文基于孕妇吸烟与胎儿问题中数据集与前期处理
- 针对随机森林与LGBM模型网格搜索效率低，使用贝叶斯调参提高效率
- 有关于贝叶斯优化包相关参数说明详解可以看GitHub地址
- 将处理好的数据用
dill包进行封装，大家在尝试运行时，需要安装dill包 - 数据文件下载地址
数据导入
- 基于
jupyter notebook的魔术命令，如果不在jupyter notebook中运行，请将%号去掉
# 如果已经安装过dill包可以不要下面的魔术命令
%pip install dill
import dill
# 如果执行这步出错，请检查错误信息
# 是否有包未安装，比如bayes_opt包或lightgbm包
dill.load_session('C:/Users/lenovo/Desktop/data.pkl')
# 若未报错，则数据导入成功
安装贝叶斯优化包
- 基于
jupyter notebook的魔术命令，如果不在jupyter notebook中运行，请将%号去掉
%pip install bayesian-optimization
随机森林贝叶斯调参
导入包
# 贝叶斯调参优化随机森林
from sklearn.model_selection import cross_val_score
from bayes_opt import BayesianOptimization
构造黑盒函数
- 构造黑盒函数，即需要优化的目标，这里我选取的是10折交叉检验的分数值。当然你也可以在
RandomForestRegressor()后使用.fit(train_x, train_y).score(test_x, test_y)将模型在测试集上表现作为最优化目标 - 因为
bayes_opt库只支持最大值，所以最后的输出如果是求最小值，那么需要在前面加上负号，以转为最大值。这里使用neg_mean_squared_error作为最大化目标 - 由于
bayes优化只能优化连续超参数，因此要加上int()转为离散超参数
# 函数中包含需要调整的参数
def rf_cv(n_estimators, min_samples_split, max_features, max_depth, min_samples_leaf, max_leaf_nodes):
# 交叉检验，得到的评分为贝叶斯调参优化目标
val = cross_val_score(
# 由于bayes优化只能优化连续超参数，因此要加上int()转为离散超参数
RandomForestRegressor(n_estimators=int(n_estimators),
min_samples_split=int(min_samples_split),
min_samples_leaf = int(min_samples_leaf),
max_leaf_nodes = int(max_leaf_nodes),
max_features=int(max_features)
max_depth=int(max_depth),
random_state=42,
oob_score = 'True'),
X_train, y_train, scoring='neg_mean_squared_error', cv=10
).mean()
return val
确定域空间
- 确定参数搜索的范围，并打印迭代过程
# 规定各参数搜索范围
rf_bo = BayesianOptimization(rf_cv,
{
'n_estimators': (800, 1500),
'min_samples_split': (2, 20),
'max_features': (1, 6),
'max_depth': (3, 10),
'min_samples_leaf' : (2,20),
'max_leaf_nodes':(10,40)})
rf_bo.maximize()
输出：
| iter | target | max_depth | max_fe... | max_le... | min_sa... | min_sa... | n_esti... |
-------------------------------------------------------------------------------------------------
| 1 | -246.0 | 4.341 | 3.478 | 27.05 | 2.973 | 16.09 | 882.3 |
| 2 | -264.1 | 4.023 | 1.761 | 31.94 | 19.03 | 5.529 | 1.338e+0 |
| 3 | -245.4 | 5.505 | 4.8 | 23.88 | 4.304 | 12.89 | 1.147e+0 |
| 4 | -247.3 | 9.276 | 2.907 | 33.51 | 7.78 | 6.09 | 1.492e+0 |
| 5 | -257.1 | 6.966 | 1.406 | 25.3 | 14.24 | 11.24 | 925.4 |
| 6 | -245.3 | 5.946 | 4.281 | 26.73 | 5.724 | 13.45 | 1.147e+0 |
| 7 | -250.0 | 3.0 | 6.0 | 29.3 | 2.0 | 20.0 | 826.7 |
| 8 | -272.6 | 3.0 | 1.0 | 38.17 | 20.0 | 2.0 | 1.097e+0 |
| 9 | -248.6 | 8.698 | 5.645 | 23.75 | 2.0 | 20.0 | 1.182e+0 |
| 10 | -264.9 | 10.0 | 1.0 | 10.0 | 20.0 | 2.0 | 858.3 |
| 11 | -272.4 | 3.0 | 1.0 | 17.23 | 20.0 | 2.0 | 1.165e+0 |
| 12 | -247.0 | 4.871 | 5.969 | 31.59 | 2.493 | 16.71 | 1.134e+0 |
| 13 | -253.0 | 10.0 | 6.0 | 40.0 | 2.0 | 20.0 | 893.9 |
| 14 | -247.9 | 8.938 | 5.776 | 10.13 | 3.715 | 19.55 | 1.129e+0 |
| 15 | -246.0 | 8.643 | 4.938 | 16.8 | 12.07 | 17.0 | 1.48e+03 |
| 16 | -272.3 | 3.046 | 1.13 | 30.6 | 20.0 | 4.148 | 1.47e+03 |
| 17 | -247.3 | 10.0 | 5.957 | 19.54 | 4.089 | 17.66 | 1.495e+0 |
| 18 | -248.7 | 6.204 | 3.098 | 10.2 | 18.7 | 11.91 | 1.495e+0 |
| 19 | -249.2 | 3.0 | 5.068 | 39.19 | 2.0 | 20.0 | 865.9 |
| 20 | -248.0 | 6.247 | 5.639 | 30.41 | 16.99 | 19.59 | 1.499e+0 |
| 21 | -252.9 | 10.0 | 6.0 | 40.0 | 2.0 | 20.0 | 1.161e+0 |
| 22 | -249.2 | 10.0 | 6.0 | 22.95 | 19.86 | 20.0 | 1.136e+0 |
| 23 | -251.1 | 10.0 | 6.0 | 29.33 | 2.0 | 20.0 | 1.207e+0 |
| 24 | -265.7 | 10.0 | 1.0 | 10.0 | 2.0 | 2.0 | 1.487e+0 |
| 25 | -248.4 | 9.63 | 5.778 | 22.09 | 2.579 | 2.131 | 1.132e+0 |
| 26 | -250.4 | 3.485 | 5.665 | 34.65 | 19.73 | 17.61 | 881.4 |
| 27 | -248.1 | 3.569 | 4.336 | 11.09 | 7.701 | 17.22 | 892.6 |
| 28 | -272.5 | 3.0 | 1.0 | 22.71 | 2.0 | 2.0 | 892.6 |
| 29 | -248.5 | 6.826 | 2.856 | 17.2 | 10.92 | 19.96 | 879.0 |
| 30 | -247.6 | 6.9 | 4.121 | 39.17 | 3.153 | 19.32 | 1.49e+03 |
=================================================================================================
- 可以看到当迭代到30次时，就已经找到了较为理想的参数
最优参
# 使用max得到最优参
rf_bo.max
输出：
{
'target': -245.31420890294567,
'params': {
'max_depth': 5.946464811467048,
'max_features': 4.280789636350939,
'max_leaf_nodes': 26.726081689294052,
'min_samples_leaf': 5.724298965332711,
'min_samples_split': 13.447191575489041,
'n_estimators': 1147.3188404305388}}
- 使用最优参(记得该取整的要取整)再次训练模型，得到相对理想的模型。
使用最优参训练模型
# 基于sklearn封装的随机森林
from sklearn.ensemble import RandomForestRegressor
# 在RandomForestRegressor()括号中填入最优参
# 这里以上面我得到的最优参为例
rf_reg = RandomForestRegressor(max_depth = 5,
max_features = 4,
max_leaf_nodes = 26,
min_samples_leaf = 5,
min_samples_split = 13,
n_estimators = 1147)
# 在训练集上训练模型
rf_reg.fit(X_train, y_train)
# 查看模型在训练集上均方误差
print('train_mes : {:.3f}'.format(mean_squared_error(y_train, rf_reg.predict(X_train))))
LGBM贝叶斯调参
- 同样的，先构造黑箱函数，然后确定域空间，唯一不同的这次我们规定随机搜索的步数和贝叶斯优化的步数。这样会增加搜索的时间，但也许能比默认步数搜索到更好的参数
LGBM模型中L1、L2正则化等参数允许为浮点类型，所以不需要使用int()进行类型转换。
def lgbm_cv(n_estimators,max_depth,num_leaves,min_child_samples,reg_alpha,reg_lambda,subsample,colsample_bytree):
val = cross_val_score(
LGBMRegressor(learning_rate = 0.001,
n_estimators=int(n_estimators),
max_depth=int(max_depth),
num_leaves = int(num_leaves),
min_child_samples = int(min_child_samples),
reg_alpha = reg_alpha, # float
reg_lambda = reg_lambda,
subsample = subsample,
colsample_bytree = colsample_bytree),
X_train, y_train, scoring='neg_mean_squared_error', cv=10).mean()
return val
lgbm_bo = BayesianOptimization(lgbm_cv,
{
'n_estimators': (1500, 3000),
'max_depth': (2, 10),
'num_leaves': (5, 20),
'min_child_samples': (3, 100),
'reg_alpha' : (0.1,1),
'reg_lambda':(0.001,1),
'subsample':(0.8,1),
'colsample_bytree':(0.8,1)})
确定迭代次数
- 迭代次数由两部分组成，随机搜索的步数和贝叶斯优化的步数，贝叶斯优化的步数要多一点，步骤越多，就越有可能找到一个好的最大值。随机探索可以通过扩大探索空间而有所帮助。
# n_iter贝叶斯优化次数,init_points随机优化次数
lgbm_bo.maximize(n_iter = 100,init_points = 50)
输出：
| iter | target | colsam... | max_depth | min_ch... | n_esti... | num_le... | reg_alpha | reg_la... | subsample |
-------------------------------------------------------------------------------------------------------------------------
| 1 | -251.5 | 0.9299 | 6.878 | 82.45 | 2.147e+0 | 11.21 | 0.3167 | 0.2107 | 0.8781 |
| 2 | -252.4 | 0.8555 | 4.824 | 58.81 | 2.055e+0 | 11.71 | 0.7608 | 0.2855 | 0.8864 |
| 3 | -249.1 | 0.96 | 7.997 | 14.63 | 2.232e+0 | 19.55 | 0.1661 | 0.6375 | 0.8603 |
| 4 | -249.4 | 0.9018 | 8.336 | 20.44 | 2.155e+0 | 18.32 | 0.6761 | 0.1784 | 0.9618 |
| 5 | -249.9 | 0.8464 | 8.894 | 76.6 | 2.907e+0 | 15.3 | 0.1962 | 0.9202 | 0.9374 |
| 6 | -256.1 | 0.9289 | 9.386 | 70.46 | 1.546e+0 | 7.014 | 0.4163 | 0.9269 | 0.8498 |
| 7 | -250.1 | 0.8198 | 5.917 | 22.09 | 1.997e+0 | 8.863 | 0.2419 | 0.08677 | 0.8001 |
| 8 | -250.1 | 0.84 | 4.637 | 25.22 | 2.121e+0 | 13.66 | 0.7642 | 0.04577 | 0.9546 |
| 9 | -248.0 | 0.8454 | 8.498 | 15.58 | 2.894e+0 | 18.63 | 0.1802 | 0.4565 | 0.8858 |
| 10 | -255.2 | 0.8839 | 7.783 | 53.08 | 1.979e+0 | 5.731 | 0.4185 | 0.5306 | 0.9554 |
| 11 | -249.2 | 0.944 | 4.216 | 88.81 | 2.929e+0 | 10.36 | 0.5145 | 0.1131 | 0.9041 |
| 12 | -247.2 | 0.8847 | 5.88 | 12.0 | 2.059e+0 | 13.3 | 0.6635 | 0.09008 | 0.9531 |
| 13 | -250.2 | 0.8651 | 3.733 | 50.73 | 2.875e+0 | 16.17 | 0.542 | 0.847 | 0.9391 |
| 14 | -251.2 | 0.8654 | 8.812 | 92.12 | 2.522e+0 | 12.46 | 0.3778 | 0.6421 | 0.9782 |
| 15 | -251.3 | 0.9663 | 6.259 | 45.49 | 2.908e+0 | 17.26 | 0.3127 | 0.04984 | 0.9103 |
| 16 | -248.5 | 0.8887 | 6.157 | 79.92 | 2.83e+03 | 7.47 | 0.7573 | 0.1109 | 0.8294 |
| 17 | -254.0 | 0.9811 | 4.113 | 83.44 | 1.704e+0 | 8.561 | 0.787 | 0.1622 | 0.9373 |
| 18 | -250.7 | 0.9525 | 6.545 | 94.2 | 2.758e+0 | 8.907 | 0.6938 | 0.3916 | 0.9476 |
| 19 | -251.0 | 0.9972 | 6.941 | 46.37 | 2.577e+0 | 12.63 | 0.7774 | 0.9983 | 0.8211 |
| 20 | -246.4 | 0.8041 | 3.084 | 3.125 | 2.505e+0 | 17.59 | 0.4361 | 0.7433 | 0.9699 |
| 21 | -249.1 | 0.8621 | 8.701 | 65.99 | 2.938e+0 | 19.25 | 0.9019 | 0.2082 | 0.8993 |
| 22 | -249.5 | 0.9085 | 3.629 | 23.46 | 2.138e+0 | 12.78 | 0.99 | 0.6435 | 0.9545 |
| 23 | -259.9 | 0.8006 | 2.359 | 33.99 | 1.649e+0 | 16.87 | 0.9538 | 0.7214 | 0.8083 |
| 24 | -253.8 | 0.9543 | 9.966 | 76.27 | 1.817e+0 | 15.48 | 0.953 | 0.4465 | 0.8373 |
| 25 | -250.0 | 0.9024 | 9.932 | 87.61 | 2.563e+0 | 17.52 | 0.6365 | 0.3728 | 0.9358 |
| 26 | -247.7 | 0.9368 | 9.499 | 26.12 | 2.618e+0 | 6.258 | 0.5139 | 0.01136 | 0.8822 |
| 27 | -256.5 | 0.8403 | 2.212 | 72.82 | 1.975e+0 | 9.344 | 0.7336 | 0.9867 | 0.986 |
| 28 | -250.4 | 0.8356 | 6.36 | 25.71 | 2.362e+0 | 12.92 | 0.9792 | 0.1116 | 0.9806 |
| 29 | -250.4 | 0.967 | 6.722 | 72.78 | 2.598e+0 | 11.43 | 0.3715 | 0.7805 | 0.9167 |
| 30 | -253.8 | 0.9365 | 2.325 | 42.29 | 2.333e+0 | 6.774 | 0.7346 | 0.5393 | 0.9785 |
| 31 | -252.4 | 0.984 | 7.318 | 44.95 | 1.856e+0 | 11.53 | 0.364 | 0.06497 | 0.9884 |
| 32 | -255.7 | 0.895 | 4.489 | 41.63 | 1.858e+0 | 5.682 | 0.1547 | 0.7007 | 0.9882 |
| 33 | -248.9 | 0.9449 | 7.396 | 32.28 | 2.355e+0 | 14.05 | 0.8676 | 0.9371 | 0.8904 |
| 34 | -251.9 | 0.8728 | 7.131 | 58.31 | 2.399e+0 | 13.46 | 0.8022 | 0.9378 | 0.9732 |
| 35 | -252.0 | 0.859 | 5.741 | 43.33 | 1.914e+0 | 11.67 | 0.4837 | 0.1606 | 0.9066 |
| 36 | -256.2 | 0.9248 | 3.282 | 44.33 | 1.549e+0 | 17.83 | 0.2039 | 0.1553 | 0.8587 |
| 37 | -250.0 | 0.8103 | 5.894 | 43.89 | 2.872e+0 | 11.2 | 0.9715 | 0.1911 | 0.8768 |
| 38 | -247.4 | 0.9543 | 4.029 | 29.43 | 2.807e+0 | 18.67 | 0.6966 | 0.3294 | 0.8734 |
| 39 | -251.7 | 0.8035 | 9.248 | 49.61 | 2.935e+0 | 19.69 | 0.4769 | 0.7963 | 0.8527 |
| 40 | -253.8 | 0.8426 | 7.232 | 73.47 | 1.953e+0 | 9.829 | 0.4052 | 0.6666 | 0.8147 |
| 41 | -253.7 | 0.8878 | 9.484 | 77.37 | 1.981e+0 | 10.31 | 0.9634 | 0.37 | 0.8724 |
| 42 | -248.7 | 0.837 | 5.001 | 5.432 | 1.895e+0 | 10.73 | 0.4941 | 0.07088 | 0.9292 |
| 43 | -252.1 | 0.8235 | 2.448 | 9.134 | 2.106e+0 | 16.9 | 0.8737 | 0.3368 | 0.8999 |
| 44 | -252.3 | 0.927 | 7.326 | 91.65 | 2.058e+0 | 11.56 | 0.7419 | 0.4703 | 0.8103 |
| 45 | -250.4 | 0.9256 | 7.025 | 64.44 | 2.401e+0 | 16.9 | 0.107 | 0.7307 | 0.8434 |
| 46 | -248.8 | 0.9423 | 3.798 | 29.89 | 2.524e+0 | 12.89 | 0.7705 | 0.741 | 0.9162 |
| 47 | -258.6 | 0.9142 | 5.767 | 76.21 | 1.505e+0 | 6.561 | 0.2224 | 0.2147 | 0.9764 |
| 48 | -252.4 | 0.9271 | 8.811 | 6.559 | 1.796e+0 | 5.845 | 0.542 | 0.8878 | 0.9614 |
| 49 | -258.8 | 0.8216 | 7.332 | 98.04 | 1.61e+03 | 5.039 | 0.4765 | 0.09902 | 0.8323 |
| 50 | -251.2 | 0.8809 | 3.795 | 97.97 | 2.588e+0 | 5.977 | 0.9903 | 0.1333 | 0.8626 |
| 51 | -252.7 | 0.8713 | 3.811 | 55.85 | 2.138e+0 | 17.5 | 0.527 | 0.4825 | 0.8298 |
| 52 | -255.0 | 0.8587 | 9.695 | 56.51 | 1.657e+0 | 14.53 | 0.6904 | 0.6378 | 0.9473 |
| 53 | -254.0 | 0.971 | 2.733 | 91.54 | 2.319e+0 | 5.299 | 0.6559 | 0.9105 | 0.9984 |
| 54 | -252.5 | 0.915 | 9.869 | 92.64 | 2.222e+0 | 10.35 | 0.9431 | 0.4694 | 0.821 |
| 55 | -251.9 | 0.9004 | 7.716 | 73.5 | 2.288e+0 | 11.8 | 0.4452 | 0.4667 | 0.9693 |
| 56 | -251.1 | 0.9678 | 4.192 | 96.36 | 2.536e+0 | 14.14 | 0.1397 | 0.04324 | 0.8681 |
| 57 | -253.8 | 0.8579 | 4.479 | 72.29 | 1.916e+0 | 17.29 | 0.8099 | 0.4709 | 0.9209 |
| 58 | -251.7 | 0.8303 | 5.407 | 92.42 | 2.351e+0 | 9.32 | 0.1488 | 0.5252 | 0.8027 |
| 59 | -250.2 | 0.8626 | 9.889 | 37.49 | 2.543e+0 | 11.68 | 0.3817 | 0.1709 | 0.8627 |
| 60 | -250.7 | 0.9972 | 8.553 | 32.94 | 1.672e+0 | 15.26 | 0.3957 | 0.1086 | 0.8383 |
| 61 | -251.9 | 0.8949 | 5.338 | 26.2 | 1.942e+0 | 7.799 | 0.8794 | 0.1337 | 0.8964 |
| 62 | -246.9 | 0.8566 | 3.851 | 16.47 | 2.076e+0 | 18.73 | 0.5352 | 0.4207 | 0.9956 |
| 63 | -245.1 | 0.9937 | 4.547 | 3.305 | 2.867e+0 | 7.185 | 0.8233 | 0.4205 | 0.9417 |
| 64 | -250.5 | 0.9768 | 8.349 | 85.95 | 2.332e+0 | 14.78 | 0.3782 | 0.8485 | 0.9663 |
| 65 | -250.4 | 0.8401 | 9.422 | 32.77 | 2.428e+0 | 18.25 | 0.7231 | 0.7265 | 0.8794 |
| 66 | -247.4 | 0.9794 | 4.493 | 8.738 | 2.042e+0 | 16.57 | 0.6579 | 0.6725 | 0.844 |
| 67 | -248.6 | 0.946 | 4.873 | 34.24 | 2.044e+0 | 16.74 | 0.3182 | 0.2692 | 0.8474 |
| 68 | -254.3 | 0.8538 | 5.687 | 79.42 | 1.824e+0 | 18.33 | 0.7764 | 0.6676 | 0.8872 |
| 69 | -255.9 | 0.8359 | 8.635 | 76.64 | 1.688e+0 | 9.01 | 0.6837 | 0.9395 | 0.9813 |
| 70 | -252.5 | 0.9696 | 9.777 | 42.32 | 2.846e+0 | 14.44 | 0.4389 | 0.1839 | 0.9697 |
| 71 | -251.0 | 0.8293 | 7.864 | 97.97 | 2.745e+0 | 15.07 | 0.4306 | 0.3661 | 0.8145 |
| 72 | -254.7 | 0.844 | 2.044 | 82.31 | 2.124e+0 | 12.8 | 0.9643 | 0.9277 | 0.8329 |
| 73 | -248.6 | 0.9647 | 4.654 | 79.34 | 2.879e+0 | 14.25 | 0.3899 | 0.4517 | 0.8848 |
| 74 | -253.5 | 0.9759 | 9.409 | 34.56 | 1.831e+0 | 6.315 | 0.2175 | 0.9694 | 0.9046 |
| 75 | -249.8 | 0.9993 | 3.751 | 53.76 | 2.705e+0 | 18.52 | 0.9932 | 0.3771 | 0.9974 |
| 76 | -248.2 | 0.8568 | 7.918 | 26.47 | 2.832e+0 | 9.567 | 0.3081 | 0.2001 | 0.9637 |
| 77 | -250.6 | 0.9161 | 9.572 | 21.78 | 2.08e+03 | 17.8 | 0.659 | 0.2496 | 0.8105 |
| 78 | -256.7 | 0.8722 | 5.862 | 96.38 | 1.661e+0 | 17.3 | 0.8781 | 0.3713 | 0.8406 |
| 79 | -249.6 | 0.9668 | 3.886 | 67.11 | 2.591e+0 | 6.384 | 0.2177 | 0.1065 | 0.9837 |
| 80 | -249.0 | 0.8911 | 5.993 | 29.05 | 2.946e+0 | 13.13 | 0.902 | 0.1512 | 0.9985 |
| 81 | -249.2 | 0.8635 | 3.71 | 43.96 | 2.767e+0 | 18.02 | 0.9057 | 0.6716 | 0.8505 |
| 82 | -253.0 | 0.9688 | 3.429 | 79.2 | 2.066e+0 | 6.202 | 0.9512 | 0.6868 | 0.8608 |
| 83 | -250.9 | 0.952 | 4.57 | 75.44 | 2.293e+0 | 17.13 | 0.6878 | 0.9675 | 0.8407 |
| 84 | -254.5 | 0.9119 | 3.122 | 82.4 | 1.808e+0 | 12.66 | 0.9125 | 0.5803 | 0.9666 |
| 85 | -257.6 | 0.9048 | 3.93 | 95.5 | 1.624e+0 | 10.92 | 0.6581 | 0.2187 | 0.9817 |
| 86 | -257.4 | 0.8564 | 2.41 | 22.95 | 1.807e+0 | 8.449 | 0.3729 | 0.761 | 0.8282 |
| 87 | -248.5 | 0.8595 | 4.704 | 5.026 | 1.914e+0 | 17.77 | 0.5784 | 0.7281 | 0.8922 |
| 88 | -248.6 | 0.9925 | 2.062 | 13.15 | 2.681e+0 | 12.09 | 0.1668 | 0.6771 | 0.8767 |
| 89 | -247.2 | 0.9495 | 8.553 | 6.885 | 2.643e+0 | 14.56 | 0.8508 | 0.2082 | 0.9777 |
| 90 | -249.9 | 0.8889 | 5.049 | 75.7 | 2.95e+03 | 19.67 | 0.5517 | 0.6659 | 0.9488 |
| 91 | -252.1 | 0.8089 | 5.886 | 56.19 | 2.138e+0 | 7.085 | 0.1107 | 0.08426 | 0.8332 |
| 92 | -250.7 | 0.8242 | 8.274 | 80.49 | 2.32e+03 | 6.095 | 0.5902 | 0.6058 | 0.9501 |
| 93 | -257.2 | 0.9934 | 2.024 | 65.59 | 1.808e+0 | 8.934 | 0.1617 | 0.9746 | 0.8435 |
| 94 | -250.5 | 0.8985 | 2.545 | 84.7 | 2.795e+0 | 19.0 | 0.1595 | 0.3708 | 0.8454 |
| 95 | -250.3 | 0.8932 | 8.764 | 47.66 | 2.538e+0 | 19.69 | 0.5193 | 0.4982 | 0.8129 |
| 96 | -251.0 | 0.9709 | 7.095 | 48.09 | 2.861e+0 | 11.9 | 0.3469 | 0.7833 | 0.9071 |
| 97 | -252.4 | 0.9348 | 8.922 | 77.9 | 2.059e+0 | 15.69 | 0.8596 | 0.5381 | 0.9972 |
| 98 | -253.6 | 0.9101 | 2.922 | 11.67 | 1.967e+0 | 15.83 | 0.9068 | 0.7492 | 0.8623 |
| 99 | -250.2 | 0.8583 | 8.64 | 89.44 | 2.598e+0 | 18.2 | 0.1822 | 0.4276 | 0.8993 |
| 100 | -246.6 | 0.8264 | 5.264 | 10.81 | 2.47e+03 | 7.567 | 0.6361 | 0.3497 | 0.8002 |
| 101 | -250.1 | 0.8995 | 3.704 | 52.65 | 2.985e+0 | 19.72 | 0.11 | 0.1371 | 0.8817 |
| 102 | -248.8 | 0.8594 | 5.012 | 4.972 | 2.131e+0 | 8.508 | 0.3815 | 0.945 | 0.9192 |
| 103 | -247.6 | 0.9669 | 3.601 | 29.7 | 2.932e+0 | 18.49 | 0.3248 | 0.7928 | 0.8384 |
| 104 | -256.3 | 0.9432 | 9.293 | 54.1 | 1.761e+0 | 5.085 | 0.3883 | 0.5722 | 0.9298 |
| 105 | -257.2 | 0.9938 | 2.443 | 84.29 | 1.823e+0 | 18.0 | 0.3708 | 0.7673 | 0.9651 |
| 106 | -252.1 | 0.9587 | 8.62 | 42.49 | 2.116e+0 | 19.23 | 0.7527 | 0.4659 | 0.9647 |
| 107 | -253.4 | 0.8556 | 9.788 | 62.2 | 1.972e+0 | 19.38 | 0.9473 | 0.1454 | 0.8905 |
| 108 | -253.2 | 0.8099 | 7.696 | 50.18 | 2.162e+0 | 15.46 | 0.6554 | 0.827 | 0.8181 |
| 109 | -251.1 | 0.8175 | 8.761 | 13.57 | 1.69e+03 | 8.422 | 0.8813 | 0.9929 | 0.9977 |
| 110 | -251.1 | 0.8626 | 6.849 | 92.65 | 2.571e+0 | 8.158 | 0.758 | 0.8469 | 0.9977 |
| 111 | -252.6 | 0.8674 | 7.393 | 27.39 | 1.724e+0 | 18.88 | 0.7886 | 0.4438 | 0.8879 |
| 112 | -251.1 | 0.9263 | 4.146 | 78.4 | 2.257e+0 | 14.98 | 0.2221 | 0.145 | 0.9518 |
| 113 | -247.5 | 0.8534 | 6.048 | 10.58 | 2.222e+0 | 8.573 | 0.4608 | 0.3796 | 0.99 |
| 114 | -246.1 | 0.8489 | 3.597 | 6.778 | 2.394e+0 | 9.335 | 0.5718 | 0.1133 | 0.8493 |
| 115 | -250.9 | 0.9191 | 7.406 | 27.09 | 2.777e+0 | 19.52 | 0.6142 | 0.7267 | 0.8041 |
| 116 | -248.9 | 0.8929 | 7.28 | 38.2 | 2.463e+0 | 8.195 | 0.4663 | 0.3756 | 0.8455 |
| 117 | -252.0 | 0.9764 | 8.82 | 12.13 | 1.569e+0 | 7.788 | 0.7776 | 0.3066 | 0.8528 |
| 118 | -254.7 | 0.9845 | 6.606 | 88.3 | 1.692e+0 | 15.52 | 0.1237 | 0.8686 | 0.9593 |
| 119 | -249.9 | 0.9242 | 4.519 | 52.74 | 2.537e+0 | 19.79 | 0.4742 | 0.5048 | 0.9595 |
| 120 | -254.5 | 0.8294 | 4.186 | 32.16 | 1.826e+0 | 6.485 | 0.4382 | 0.6719 | 0.9082 |
| 121 | -257.4 | 0.8519 | 7.839 | 87.22 | 1.519e+0 | 13.29 | 0.3876 | 0.01006 | 0.9211 |
| 122 | -253.1 | 0.8557 | 4.863 | 99.88 | 2.196e+0 | 6.027 | 0.2864 | 0.08695 | 0.8477 |
| 123 | -255.3 | 0.9123 | 3.487 | 58.82 | 1.743e+0 | 16.01 | 0.981 | 0.2148 | 0.9191 |
| 124 | -252.9 | 0.8671 | 6.105 | 34.59 | 1.822e+0 | 7.959 | 0.5157 | 0.4601 | 0.9288 |
| 125 | -251.6 | 0.823 | 4.232 | 4.69 | 1.63e+03 | 17.58 | 0.3137 | 0.896 | 0.8394 |
| 126 | -251.1 | 0.9605 | 9.27 | 3.885 | 1.594e+0 | 9.155 | 0.3988 | 0.04495 | 0.8383 |
| 127 | -251.1 | 0.8553 | 8.43 | 92.19 | 2.549e+0 | 15.68 | 0.4312 | 0.3145 | 0.8269 |
| 128 | -253.7 | 0.8167 | 6.511 | 63.97 | 1.761e+0 | 18.79 | 0.4067 | 0.4545 | 0.8913 |
| 129 | -250.5 | 0.9509 | 3.603 | 86.18 | 2.351e+0 | 6.903 | 0.858 | 0.1375 | 0.887 |
| 130 | -251.3 | 0.8577 | 7.442 | 58.83 | 2.718e+0 | 18.86 | 0.3148 | 0.3542 | 0.8545 |
| 131 | -252.5 | 0.8522 | 5.962 | 44.5 | 1.682e+0 | 17.12 | 0.1616 | 0.242 | 0.8918 |
| 132 | -256.4 | 0.9798 | 4.611 | 95.35 | 1.608e+0 | 9.765 | 0.6692 | 0.7236 | 0.8428 |
| 133 | -251.3 | 0.9747 | 8.909 | 39.93 | 2.322e+0 | 13.09 | 0.7278 | 0.2057 | 0.9541 |
| 134 | -258.2 | 0.898 | 2.558 | 77.46 | 1.824e+0 | 17.6 | 0.184 | 0.882 | 0.8166 |
| 135 | -250.7 | 0.9765 | 8.294 | 36.04 | 1.986e+0 | 15.15 | 0.1793 | 0.0547 | 0.8536 |
| 136 | -256.1 | 0.8067 | 8.756 | 78.74 | 1.653e+0 | 9.253 | 0.4187 | 0.356 | 0.8482 |
| 137 | -255.0 | 0.9928 | 6.42 | 93.78 | 1.784e+0 | 18.29 | 0.3528 | 0.7969 | 0.9533 |
| 138 | -250.7 | 0.9377 | 9.27 | 93.17 | 2.997e+0 | 13.43 | 0.2546 | 0.3083 | 0.8694 |
| 139 | -259.6 | 0.8726 | 8.204 | 78.49 | 1.566e+0 | 5.512 | 0.6164 | 0.7091 | 0.8871 |
| 140 | -253.0 | 0.9226 | 2.858 | 49.18 | 2.565e+0 | 14.22 | 0.7601 | 0.5902 | 0.8829 |
| 141 | -255.2 | 0.8894 | 2.576 | 54.63 | 2.112e+0 | 16.73 | 0.9314 | 0.9692 | 0.9501 |
| 142 | -251.9 | 0.8662 | 2.44 | 42.66 | 2.677e+0 | 7.367 | 0.5093 | 0.6031 | 0.8568 |
| 143 | -246.4 | 0.8408 | 5.624 | 9.557 | 2.655e+0 | 17.48 | 0.6699 | 0.01308 | 0.9373 |
| 144 | -256.3 | 0.8831 | 4.472 | 72.16 | 1.638e+0 | 8.795 | 0.748 | 0.7101 | 0.8074 |
| 145 | -254.3 | 0.9842 | 5.52 | 54.41 | 1.66e+03 | 10.72 | 0.8502 | 0.954 | 0.8083 |
| 146 | -250.6 | 0.977 | 5.445 | 60.33 | 2.612e+0 | 18.05 | 0.5285 | 0.458 | 0.9856 |
| 147 | -249.9 | 0.9103 | 5.405 | 91.39 | 2.967e+0 | 12.09 | 0.6206 | 0.8223 | 0.812 |
| 148 | -252.6 | 0.9774 | 4.257 | 42.9 | 1.731e+0 | 15.42 | 0.9746 | 0.8842 | 0.8383 |
| 149 | -254.0 | 0.858 | 2.464 | 43.46 | 2.257e+0 | 13.9 | 0.3589 | 0.9985 | 0.8966 |
| 150 | -253.4 | 0.9733 | 9.152 | 55.79 | 2.118e+0 | 5.46 | 0.8561 | 0.062 | 0.8039 |
输出最优参
lgbm_bo.max
输出：
{
'target': -243.40837561015314,
'params': {
'colsample_bytree': 0.9035439141128341,
'max_depth': 3.167260637404743,
'min_child_samples': 9.324254670380586,
'n_estimators': 2816.120214269747,
'num_leaves': 17.035319354824765,
'reg_alpha': 0.33257959049618924,
'reg_lambda': 0.5556596408409464,
'subsample': 0.8898942099929366}}
- 使用最优参(记得该取整的要取整)再次训练模型，得到相对理想的模型。
写在最后
- 如果大家想更好的使用贝叶斯优化包，可以读一读优化包的GitHub说明，里面有基于经验范围的精密搜索、经验函数等一些参数的调整等，或许能提升优化器性能。
边栏推荐
- These 10 kinds of timed tasks have been learned, and you can float directly
- Leetcode sword finger offer JZ9 dual stack implementation queue
- Sed command
- DNS domain name resolution service
- 这些坑你不掌握，你还真不敢用BigDecimal
- Niuke net brush questions
- Write cookies, sessionstorage, localstorage and session at will
- Backup MySQL database with bat script under Windows
- 进程和计划任务管理
- Leetcode sword finger offer jz42 maximum sum of continuous subarrays
猜你喜欢
Polkadot | interprets how liberty plan, which subverts traditional social media, will be launched in Poka
SSH远程访问及控制
Process and planned task management
Leetcode sword finger offer JZ9 dual stack implementation queue
Use of awk
Sort by an attribute value of an object in the array
磁盘管理和文件系统
Combination of grep and regular
Maximum value of jz47 gifts (dynamic planning ideas)
【LVGL】组件的样式的设置、更改、删除API函数
随机推荐
Flink state use
【LVGL(3)】设置对象大小、位置、盒子模型、状态
ESP32超详细学习记录：NTP同步时间
Browser local storage
【LVGL(4)】对象的事件及事件冒泡
Batch implementation of key based authentication using sshpass
【LVGL】组件的样式的设置、更改、删除API函数
CentOS操作系统安全加固
MySQL Index & execution plan
PXE技术网络装机
Log collection and analysis platform
Homework in the second week
利用sshpass批量实现基于key验证
MySQL批量修改数据表编码及字符集为utf8mb4
【LVGL】【阶段总结1】
Windows下bat脚本备份MySQL数据库
go的环境搭建和起步
这些坑你不掌握，你还真不敢用BigDecimal
Leetcode does not add, subtract, multiply, divide, and calculate the number of 1 in binary
中药材的鉴别