caffe训练问题总结

原创于 2020-04-22 14:46:32 发布 · 646 阅读

0 ·

本内容遵循CC 4.0 BY-SA版权协议

深度学习同时被 2 个专栏收录

17 篇文章

订阅专栏

Caffe

13 篇文章

订阅专栏

本文详细解析了Caffe框架中常见的三个错误：检查失败datum_channels大于0、blob大小超过INT_MAX以及math_functions.cpp中的数值比较错误。针对每个问题，提供了具体的定位和修复策略，包括prototxt文件配置调整、lmdb数据生成参数修改以及源代码级的修正。

1、Check failed: datum_channels > 0 (0 vs. 0).
train.prototxt中Datatype AnnotatedData和Data互换，changing data type from “AnnotatedData” to

“Data” or “Data” to "AnnotatedData"


layer {
  name: "Data1"
  type: "AnnotatedData"
  top: "Data1"
  top: "Data2"
  transform_param {
    scale: 0.007842999882996082
    mirror: false
    mean_value: 127.0
  }
  data_param {
    source: "./train_lmdb/"
    batch_size: 64
    backend: LMDB
  }
}

2、Check failed: shape[i] <= 0x7fffffff / count_ (2688 vs. 344) blob size exceeds INT_MAX
生成lmdb训练数据的时候，需要对图片进行resize，修改create_data.sh进行修改，width=300，height=300

	
cur_dir=$(cd $( dirname ${BASH_SOURCE[0]} ) && pwd )
root_dir="/data/zhangrong/caffe/caffe-ssd/caffe"

cd $root_dir

redo=1
data_root_dir="/data/zhangrong/caffe/caffe-ssd/caffe/data/VOCdevkit"
dataset_name="MyDataSet"
mapfile="$root_dir/data/$dataset_name/labelmap_voc.prototxt"
anno_type="detection"
db="lmdb"
min_dim=0
max_dim=0
width=300
height=300

extra_cmd="--encode-type=jpg --encoded"
if [ $redo ]
then
  extra_cmd="$extra_cmd --redo"
fi
for subset in test trainval
do
  python $root_dir/scripts/create_annoset.py --anno-type=$anno_type --label-map-file=$mapfile --min-dim=$min_dim --max-dim=$max_dim --resize-width=$width --resize-height=$height --check-label $extra_cmd $data_root_dir $root_dir/data/$dataset_name/$subset.txt $data_root_dir/$dataset_name/$db/$dataset_name"_"$subset"_"$db examples/$dataset_name
done

3、math_functions.cpp:250] Check failed: a <= b <0 vs -1.19209e-007>
解决：找到math_functions.cpp，找到在路径为~/caffe/src/caffe/util的math_functions.cpp，找到第250行出错的地方，双斜杠屏蔽
找到在路径为~/caffe/src/caffe/util/sampler.cpp，在其中增加加粗黑体部分的代码

void SampleBBox(const Sampler& sampler, NormalizedBBox* sampled_bbox) {
// Get random scale.
CHECK_GE(sampler.max_scale(), sampler.min_scale());
CHECK_GT(sampler.min_scale(), 0.);
CHECK_LE(sampler.max_scale(), 1.);
float scale;
caffe_rng_uniform(1, sampler.min_scale(), sampler.max_scale(), &scale);
// Get random aspect ratio.
CHECK_GE(sampler.max_aspect_ratio(), sampler.min_aspect_ratio());
CHECK_GT(sampler.min_aspect_ratio(), 0.);
CHECK_LT(sampler.max_aspect_ratio(), FLT_MAX);
float aspect_ratio;
caffe_rng_uniform(1, sampler.min_aspect_ratio(), sampler.max_aspect_ratio(),
&aspect_ratio);
aspect_ratio = std::max(aspect_ratio, std::pow(scale, 2.));
aspect_ratio = std::min(aspect_ratio, 1 / std::pow(scale, 2.));
// Figure out bbox dimension.
float bbox_width = scale * sqrt(aspect_ratio);
float bbox_height = scale / sqrt(aspect_ratio);
if(bbox_width>=1.0){
bbox_width=1.0;
}
if(bbox_height>=1.0){
bbox_height=1.0;
}
// Figure out top left coordinates.
float w_off, h_off;
caffe_rng_uniform(1, 0.f, 1.0f - bbox_width, &w_off);
caffe_rng_uniform(1, 0.f, 1.0f - bbox_height, &h_off);
sampled_bbox->set_xmin(w_off);
sampled_bbox->set_ymin(h_off);
sampled_bbox->set_xmax(w_off + bbox_width);
sampled_bbox->set_ymax(h_off + bbox_height);
}