基于docker的python faster-rcnn caffe环境搭建+提取目标特征实验

起初是为了使用faster-rcnn的目标提取功能提取图片中的目标特征，供后续模型学习，有一个比较好的实现bottom up attention，其为py-faster-rcnn在 Visual Genome 数据集预训练的模型,官方py-faster-rcnn在COCO数据集预训练下仅识别80个class，bottom up attention可识别1600个object class以及400个属性类别。我也是基于这个项目改写来实现图像对象特征预提取的。

论文地址：https://panderson.me/up-down-attention/ （CVPR 2018）

1. 环境配置前言

caffe环境配置真的挺麻烦的。。关键是许多相关版本和当前服务器上的相差很大，去适配就更加麻烦了，我这里使用基于Docker的方法来配置，因为我一直是习惯在Docker内配置深度学习环境及开发的，使用docker开发非常清爽！！这边可以参考我以前写的两篇文章来配置docker环境：

• centos + docker搭建深度学习环境以及一些问题解决
• Docker基本使用

同时这里有位前辈曾经也记录过caffe下bottom up attention (py-faster-rcnn)环境配置，这是直接在机器上安装的。
 
下面，先预览一下可以跑得通的环境版本配置（后面有时间再研究研究cuda9.0以上的版本是不是可以使用吧，最近在赶实验实在是没时间）

• Ubuntu16.04系统
• GTX xxx 显卡
• cuda8.0+cuDNN5.1
• 无需安装opencv3！

2. 下载caffe镜像

就一行指令：docker pull bvlc/caffe:gpu

当然可以去https://hub.docker.com/search/?q=SSD%20caffe&type=image搜索想要的caffe版本以及各种docker镜像。

镜像拉下来后，我的启动指令是：

sudo docker run --runtime=nvidia -it --privileged=true -p 1xxxx:22 -v /home/xxx/tf-torch:/root/tf-torch --name env_caffe bvlc/caffe:gpu /bin/bash

这里解释一下，-p 1xxxx:22 是为了能通过1xxxx这个对外接口来直接ssh到运行了ssh服务的docker内，这对于远程开发调试来说十分有意义。-v是文件映射。

另外注意这个版本是基于python2.7的。

3. 下载bottom-up-attention代码以及编译

这是能成功执行的代码版本：https://github.com/realcactus/bottom-up-attention，用文章开头贴出的官方的版本应该也差不多，都是几年没更新代码了，不过我也没去细看，以防万一我还是放在这里。

然着项目README文件中的Installation部分来操作即可，lib中的make不会有任何问题，最后一步，Build Caffe and pycaffe这里还是会遇到问题的，也别看他写的注释了，这样来：

cd bottom-up-attention
cd caffe
// 将Makefile.config自己写一份出来
mv Makefile.config.example Makefile.config
// 这里需要注意的是，先只要把里面的
// WITH_PYTHON_LAYER := 1
// USE_NCCL := 1
// USE_CUDNN := 1
// 这三行前面的注释删掉就好，后面再有问题跟着来，直接放一个Makefile文件配置简直一脸懵逼


// 你需要安装python-opencv2，但是python2.7中安装需要注意，加上清华源，另外指定版本到3.2左右，新版本的，import时会报错。
pip install opencv-python==3.2.0.6  -i https://pypi.tuna.tsinghua.edu.cn/simple

// 接下来make
make -j8 && make pycaffe

// ok,会遇到问题，别慌，如果你用的是docker，我们应该是百分百一样的过程。
// 遇到问题，fatal error: hdf5.h: 没有那个文件或目录，这个是路径问题。参考解决：https://www.cnblogs.com/xiangfeidemengzhu/p/7058391.html
// 即locate hdf5.h，找到你的hdf5.h文件在哪，然后修改Makefile.config文件，下面那行中后半部分是我们要加上的
// INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/

// 修改Makefile文件
// LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial

// 清理编译

make clean

重新编译，八个线程

make -j8


// 接下来又会遇到问题：make pycaffe 报错：“fatal error: numpy/arrayobject.h: No such file or directory”   参考解决： https://blog.csdn.net/baidu_38172402/article/details/86669305
// 其实就是在Makefile.config中找到PYTHON_INCLUDE，看看是不是写错了，我的就是改成 
// PYTHON_INCLUDE := /usr/include/python2.7 \
                /usr/local/lib/python2.7/distpackages/numpy/core/include
// 就行，其实就是前面加了个local。

// 重新执行，等待成功：
make -j8 && make pycaffe

 
 
这里我放一份我最终的Makefile.config内容

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#	You should not set this flag if you will be reading LMDBs with any
#	possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
		-gencode arch=compute_20,code=sm_21 \
		-gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
		-gencode arch=compute_50,code=sm_50 \
		-gencode arch=compute_50,code=compute_50

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \
		/usr/local/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		# $(ANACONDA_HOME)/include/python2.7 \
		# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include

# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
USE_NCCL := 1

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

4. 修改代码进行目标特征提取

这个蛮简单的就是用generate_tsv.py文件从图像中抽取出目标特征。

4.1 数据准备

文件结构：

• data
  • MyData
    • imgs（文件夹）
    • tags.jsonl

把所有的图片放到imgs文件夹内，需要一个tags.jsonl文件，我这里列举一行内容：

{“sentence”: “这是文本内容”, “img_path”: “xxx.jpg”}

4.2 修改generate_tsv.py

load_image_ids方法中，加入一个elif：

elif split_name == "universal":
      # General extraction
      # that is, to extract the target features of a few pictures
      image_path = {}
      with open('data/MyData/tags.jsonl', 'rb') as f:
        for item in json_lines.reader(f):
          if item['img_path'] not in image_path:
            image_path[item['img_path']] = 1

      for image_p in image_path.keys():
        filepath = os.path.join('data/MyData/imgs',image_p)
        split.append((filepath, image_p))

      total_num = len(split)
      per_num = int(np.ceil(total_num / total_group))
      
      if group_id == total_num-1:
        split = split[int(group_id * per_num):]
      else:
        split = split[int(group_id * per_num):int((group_id+1)*per_num)]

然后运行文件，我的参数是：

"--cfg",
"experiments/cfgs/faster_rcnn_end2end_resnet.yml",
"--def",
"models/vg/ResNet-101/faster_rcnn_end2end_final/test.prototxt",
"--out",
"feature/MyData/MyData_resnet101_faster_rcnn_genome.tsv",
"--net",
"data/faster_rcnn_models/resnet101_faster_rcnn_final.caffemodel",
"--total_group",
"1",
"--group_id",
"0",
"--split",
"universal"