技術文章 » ubuntu16.04下安裝caffe以及錯誤彙總(轉）

ubuntu16.04下安裝caffe以及錯誤彙總(轉）

2020-08-09 14:13:16

這篇文章主要講安裝和編譯caffe過程以及遇到的問題，還有解決方法。關於如何安裝nvidia驅動，cuda，cudnn，以及opencv3 等網上已經有足夠多的教學了，可以直接在網上找。

1. 系統設定：

OS: Ubuntu16.04
python Version: python3
CUDA Version: 10.0

2. caffe安裝

2.1  安裝依賴庫：


 

  1. sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler

  2. sudo apt-get install --no-install-recommends libboost-all-dev

2.2 下載caffe原始碼

下載地址：git clone https://github.com/BVLC/caffe.git

 

3. 修改檔案

3.1 修改Makefile.config檔案

  • 進入caffe原始碼主目錄，拷貝一個Makefile.config作爲編譯設定選項
cp Makefile.config.example Makefile.config
  • 修改編譯檔案Makefile.config 

 

  1. # USE_CUDNN := 1  

  2. # USE_OPENCV := 0

  3. # OPENCV_VERSION := 3

  4. # WITH_PYTHON_LAYER := 1

  5.  

  6. # 改爲： 

  7.  

  8. USE_CUDNN := 1

  9. USE_OPENCV := 1

  10. OPENCV_VERSION := 3

  11. WITH_PYTHON_LAYER := 1


 

  1. INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include

  2. LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

  3.  

  4. 改爲：

  5.  

  6. INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial

  7. LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial

使用python3.5版本，把python2.7程式碼加註 加注釋，python3.5前的註釋去掉 


 

  1. #PYTHON_INCLUDE := /usr/include/python2.7 \

  2.                   /usr/lib/python2.7/dist-packages/numpy/core/include

  3.  

  4. PYTHON_INCLUDE := /usr/include/python3.5m \

  5.                  /usr/lib/python3.5/dist-packages/numpy/core/include

3.2 修改Makefile檔案


 

  1. NVCCFLAGS += -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)

  2. 改爲：

  3. NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)

  4.  
  5.  

  6. LIBRARIES += glog gflags protobuf boost_system boost_filesystem m

  7. 改爲：

  8. LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial

4. 編譯

make all -j8

 

遇到錯誤：

1. 編譯選項問題 

/usr/include/c++/4.9/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
 #error This file requires compiler and library support for the 

......

/usr/local/include/google/protobuf/arena.h:324:36: note:   template argument deduction/substitution failed:
Makefile:600: recipe for target '.build_release/src/caffe/proto/caffe.pb.o' failed
make: *** [.build_release/src/caffe/proto/caffe.pb.o] Error 1

解決方法：請看這位大神是如何操作的https://blog.csdn.net/pkwcxy/article/details/82351669?utm_source=blogxgwz8

在Makefile檔案中找到：

##############################
# Configure build
##############################

找到：

# Linux

新增：CXXFLAGS += -std=c++11

安裝上面的方法一開始沒報錯，但過了一段時間後還是報同樣的錯誤，再進一步如下修改，開啓Makefile檔案，給CXXFLAGS、NVCCFLAGS、LINKFLAGS都使用-std=c++11標準編譯：


 

  1. CXXFLAGS += -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS) 

  2. 改爲：

  3. CXXFLAGS += -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS) -std=c++11 

  4.  
  5.  

  6. NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)

  7. 改爲：

  8. NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)  -std=c++11

  9.  
  10.  

  11. LINKFLAGS += -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS)

  12. 改爲：

  13. LINKFLAGS += -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS) -std=c++11

 

2. CUDA版本問題

重新編譯還是有錯誤：

NVCC src/caffe/solvers/adagrad_solver.cu
nvcc fatal   : Unsupported gpu architecture 'compute_20'
NVCC src/caffe/solvers/sgd_solver.cu
nvcc fatal   : Unsupported gpu architecture 'compute_20'
Makefile:607: recipe for target '.build_release/cuda/src/caffe/solvers/rmsprop_solver.o' failed

原因是不支援GPU架構'compute_20'，在Makefile.config裏面的找到

CUDA_ARCH := -gencode arch=compute_20,code=sm_20
                  -gencode arch=compute_20,code=sm_21 \
                  -gencode arch=compute_30,code=sm_30 \
                  -gencode arch=compute_35,code=sm_35 \
                  -gencode arch=compute_50,code=sm_50 \
                  -gencode arch=compute_52,code=sm_52 \
                  -gencode arch=compute_60,code=sm_60 \
                  -gencode arch=compute_61,code=sm_61 \
                  -gencode arch=compute_61,code=compute_61

把前面compute_20相關的那2行刪除
-gencode arch=compute_20,code=sm_20
-gencode arch=compute_20,code=sm_21
 

3. g++版本問題

.build_release/lib/libcaffe.so：對‘leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)’未定義的參照
.build_release/lib/libcaffe.so：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::Status::ToString() const’未定義的參照
collect2: error: ld returned 1 exit status
Makefile:637: recipe for target '.build_release/tools/upgrade_solver_proto_text.bin' failed
make: *** [.build_release/tools/upgrade_solver_proto_text.bin] Error 1
make: *** 正在等待未完成的任務....
......

這個g++版本問題，解決方法這裏參考：https://blog.csdn.net/chenshuibiao/article/details/78734957

先檢視g++版本：

gcc --version     # 或 g++ --version

gcc (Ubuntu 4.9.3-13ubuntu2) 4.9.3
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

發現我的是4.9.3版本，需要升級gcc版本。檢視我Ubuntu系統發現裏面已經存在更高階的5版本，就不用再安裝了，直接強制軟鏈接一下即可：


 

  1. sudo ln -s /usr/bin/gcc-5 /usr/bin/gcc -f

  2. sudo ln -s /usr/bin/g++-5 /usr/bin/g++ -f

再次檢視g++版本發現已經變成5.4.0了

 g++ (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

再次編譯caffe原始碼，發現沒問題了。

4. python版本問題

如果在Makefile.config檔案裏面Python相關的設定有問題，可能會遇到以下的問題： 

CXX/LD -o .build_release/tools/upgrade_net_proto_text.bin
CXX/LD -o .build_release/examples/siamese/convert_mnist_siamese_data.bin
.build_release/lib/libcaffe.so：對‘leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)’未定義的參照
.build_release/lib/libcaffe.so：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::Status::ToString() const’未定義的參照
.build_release/tools/convert_imageset.o：在函數‘std::string* google::MakeCheckOpString<unsigned long, int>(unsigned long const&, int const&, char const*)’中：
convert_imageset.cpp:(.text._ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc]+0x50)：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/tools/convert_imageset.o：在函數‘main’中：
convert_imageset.cpp:(.text.startup+0x347)：對‘google::SetUsageMessage(std::string const&)’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::Status::ToString() const’未定義的參照
.build_release/tools/caffe.o：在函數‘std::string* google::MakeCheckOpString<cudaError, cudaError>(cudaError const&, cudaError const&, char const*)’中：
caffe.cpp:(.text._ZN6google17MakeCheckOpStringI9cudaErrorS1_EEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringI9cudaErrorS1_EEPSsRKT_RKT0_PKc]+0x50)：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/tools/caffe.o：在函數‘std::string* google::MakeCheckOpString<unsigned long, int>(unsigned long const&, int const&, char const*)’中：
caffe.cpp:(.text._ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc]+0x50)：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/tools/caffe.o：在函數‘main’中：
caffe.cpp:(.text.startup+0x53)：對‘google::SetVersionString(std::string const&)’未定義的參照
caffe.cpp:(.text.startup+0x86)：對‘google::SetUsageMessage(std::string const&)’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::Status::ToString() const’未定義的參照
.build_release/tools/extract_features.o：在函數‘std::string* google::MakeCheckOpString<int, int>(int const&, int const&, char const*)’中：
extract_features.cpp:(.text._ZN6google17MakeCheckOpStringIiiEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringIiiEEPSsRKT_RKT0_PKc]+0x50)：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/tools/extract_features.o：在函數‘std::string* google::MakeCheckOpString<unsigned long, unsigned long>(unsigned long const&, unsigned long const&, char const*)’中：
extract_features.cpp:(.text._ZN6google17MakeCheckOpStringImmEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringImmEEPSsRKT_RKT0_PKc]+0x51)：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::Status::ToString() const’未定義的參照
.build_release/tools/compute_image_mean.o：在函數‘std::string* google::MakeCheckOpString<int, int>(int const&, int const&, char const*)’中：
compute_image_mean.cpp:(.text._ZN6google17MakeCheckOpStringIiiEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringIiiEEPSsRKT_RKT0_PKc]+0x50)：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/tools/compute_image_mean.o：在函數‘std::string* google::MakeCheckOpString<unsigned long, int>(unsigned long const&, int const&, char const*)’中：
compute_image_mean.cpp:(.text._ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc]+0x50)：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/tools/compute_image_mean.o：在函數‘main’中：
compute_image_mean.cpp:(.text.startup+0x172)：對‘google::SetUsageMessage(std::string const&)’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::Status::ToString() const’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)’未定義的參照
.build_release/lib/libcaffe.so：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::Status::ToString() const’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)’未定義的參照
.build_release/lib/libcaffe.so：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::Status::ToString() const’未定義的參照
.build_release/examples/siamese/convert_mnist_siamese_data.o：在函數‘convert_dataset(char const*, char const*, char const*)’中：
convert_mnist_siamese_data.cpp:(.text+0x4cf)：對‘leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)’未定義的參照
.build_release/examples/siamese/convert_mnist_siamese_data.o：在函數‘std::string* google::MakeCheckOpString<unsigned int, int>(unsigned int const&, int const&, char const*)’中：
convert_mnist_siamese_data.cpp:(.text._ZN6google17MakeCheckOpStringIjiEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringIjiEEPSsRKT_RKT0_PKc]+0x50)：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/examples/siamese/convert_mnist_siamese_data.o：在函數‘std::string* google::MakeCheckOpString<unsigned int, unsigned int>(unsigned int const&, unsigned int const&, char const*)’中：
convert_mnist_siamese_data.cpp:(.text._ZN6google17MakeCheckOpStringIjjEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringIjjEEPSsRKT_RKT0_PKc]+0x50)：對‘google::base::CheckOpMessageBuilder::NewString()’未定義的參照
.build_release/lib/libcaffe.so：對‘leveldb::Status::ToString() const’未定義的參照
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
Makefile:637: recipe for target '.build_release/tools/compute_image_mean.bin' failed
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
make: *** [.build_release/tools/compute_image_mean.bin] Error 1
make: *** 正在等待未完成的任務....
Makefile:637: recipe for target '.build_release/tools/caffe.bin' failed
make: *** [.build_release/tools/caffe.bin] Error 1
Makefile:637: recipe for target '.build_release/tools/extract_features.bin' failed
make: *** [.build_release/tools/extract_features.bin] Error 1
Makefile:637: recipe for target '.build_release/tools/upgrade_solver_proto_text.bin' failed
make: *** [.build_release/tools/upgrade_solver_proto_text.bin] Error 1
Makefile:637: recipe for target '.build_release/tools/upgrade_net_proto_binary.bin' failed
make: *** [.build_release/tools/upgrade_net_proto_binary.bin] Error 1
Makefile:637: recipe for target '.build_release/tools/convert_imageset.bin' failed
make: *** [.build_release/tools/convert_imageset.bin] Error 1
Makefile:637: recipe for target '.build_release/tools/upgrade_net_proto_text.bin' failed
make: *** [.build_release/tools/upgrade_net_proto_text.bin] Error 1
Makefile:642: recipe for target '.build_release/examples/siamese/convert_mnist_siamese_data.bin' failed
make: *** [.build_release/examples/siamese/convert_mnist_siamese_data.bin] Error 1
 

經過自己研究發現，如果使用Python2版本就會有以上的問題，如果我改爲 python3版本，即

PYTHON_INCLUDE := /usr/include/python3.5m \
                   /usr/lib/python3.5/dist-packages/numpy/core/include

就發現沒有問題了。我係統裏面兩個版本都安裝有的，可能是預設使用python3的問題吧。

 

5. 找不到 -lboost_python3

CXX examples/cifar10/convert_cifar_data.cpp
AR -o .build_release/lib/libcaffe.a
LD -o .build_release/lib/libcaffe.so.1.0.0
/usr/bin/ld: 找不到 -lboost_python3
collect2: error: ld returned 1 exit status
Makefile:585: recipe for target '.build_release/lib/libcaffe.so.1.0.0' failed
make: *** [.build_release/lib/libcaffe.so.1.0.0] Error 1

解決方法：

在/usr/lib/x86_64-linux-gnu/ 下檢視有沒有libboost_python-py3.5.so，如果有鏈接一下就可以了。

sudo ln -s libboost_python-py35.so libboost_python3.so

如果你使用的是python3.6版本，發現在 /usr/lib/x86_64-linux-gnu/ 下沒有libboost_python-py3.6.so 的，那你就的自己編譯一個了，可以參考我的另一篇博文：https://blog.csdn.net/u012505617/article/details/88556621

 

 

 參考：

https://blog.csdn.net/sinat_27240143/article/details/82701285

https://blog.csdn.net/fanhenghui/article/details/80092131