Bootstrap Aggregating

最新推荐文章于 2025-08-27 13:36:25 发布

原创最新推荐文章于 2025-08-27 13:36:25 发布 · 1k 阅读

0 ·

本内容遵循CC 4.0 BY-SA版权协议

收录于

当前文章被以下社区和专栏收录：

Bagging是一种用于提高分类和回归模型稳定性和准确性的元算法。它通过降低方差避免过拟合，并适用于各种模型，尤其是决策树。该方法通过从原始训练集中重复抽样创建多个训练集，然后对这些训练集上的预测结果进行平均或投票。

Wiki ref: http://en.wikipedia.org/wiki/Bootstrap_aggregating

Bootstrap aggregating (bagging) is a meta-algorithm to improve classification and regression models in terms of stability and classification accuracy. Bagging also reduces variance and helps to avoid overfitting. Although this method is usually applied to decision tree models, it can be used with any type of model. Bagging is a special case of the model averaging approach.

Given a standard training set D of size N, we generate L new training sets $D i$ also of size N' (N' < N) by sampling examples uniformly from D, and with replacement. By sampling with replacement it is likely that some examples will be repeated in each $D i$ . If $N' = N$ , then for large $N$ the set $D i$ expected to have 63.2% of the examples of D, the rest being duplicates. This kind of sample is known as a bootstrap sample. The L models are fitted using the above L bootstrap samples and combined by averaging the output (in case of regression) or voting (in case of classification). One particular interesting point about bagging is that, since the method averages several predictors, it is not useful for improving linear models.