xgboost使用、预估、模型保存、打印特征重要度

最新推荐文章于 2024-07-30 22:02:37 发布

原创最新推荐文章于 2024-07-30 22:02:37 发布 · 5.1k 阅读

本内容遵循CC 4.0 BY-SA版权协议

本文介绍如何使用XGBoost进行回归预测任务，包括模型训练、验证集上的性能评估及模型保存与加载的过程。此外，还展示了如何利用特征重要度进行特征选择。

from sklearn.ensemble import GradientBoostingClassifier
from sklearn import cross_validation, metrics
from sklearn.model_selection import cross_val_score
from sklearn.tree import DecisionTreeRegressor
import sklearn.preprocessing
from sklearn import linear_model
from sklearn.preprocessing import Imputer


X_train, X_test, y_train, y_test = cross_validation.train_test_split(X, y, test_size=0.3, random_state=198)
print(X_train.shape, X_test.shape)

xlf = xgb.XGBRegressor(max_depth=10, 
learning_rate=0.1, 
n_estimators=10, 
silent=True, 
objective='reg:linear', 
nthread=-1,
gamma=0,
min_child_weight=1, 
max_delta_step=0, 
subsample=0.85, 
colsample_bytree=0.7, 
colsample_bylevel=1, 
reg_alpha=0, 
reg_lambda=1,
scale_pos_weight=1,
seed=1440,
missing=None)
xlf.fit(X_train, y_train, eval_metric='rmse', verbose = True, eval_set = [(X_test, y_test)],early_stopping_rounds=100)

pred = xlf.predict(X_test)

保存模型

xlf.get_booster().save_model('0001.model')

载入模型

xlf_new = xgb.Booster({'nthread':4}) #init model  
xlf_new.load_model("0001.model") # load data 
//使用载入的模型时，data需要先转换，直接使用DataFarme数据会报错：AttributeError: 'DataFrame' object has no attribute 'feature_names'
data_test = xgb.DMatrix(data_test)
pred_new = xlf_new.predict(data_test)

打印特征重要度

使用f1 score
import pandas as pd  
import matplotlib.pylab as plt  
feat_imp = pd.Series(clf.booster().get_fscore()).sort_values(ascending=False)  
feat_imp.plot(kind='bar', title='Feature Importances')  
plt.ylabel('Feature Importance Score')  
plt.show()

点个赞、留个言再走吧~ 你的点赞、留言是对我最大的支持。