[ERROR] 安装完Detectron后出现 cython_nms.so: undefined symbol: PyFPE_jbuf

2023年3月15日 351次阅读

上个月Install Detectron后,

检查caffe2是否安装成功：

# To check if Caffe2 GPU build was successful
# This must print a number > 0 in order to use Detectron
python2 -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())'

输出1 ，没问题

检查Detectron是否编译成功：

python2 $DETECTRON/tests/test_spatial_narrow_as_op.py

依旧通过。执行nference tutorial时也一切正常。

但是今天在执行inference tutorial时，却报错了：

python2 tools/infer_simple.py \
    --cfg configs/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml \
    --output-dir /tmp/detectron-visualizations \
    --image-ext jpg \
    --wts https://s3-us-west-2.amazonaws.com/detectron/35861858/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml.02_32_51.SgT4y1cO/output/train/coco_2014_train:coco_2014_valminusminival/generalized_rcnn/model_final.pkl \
    demo

提示：

ImportError: /home/cc/Detectron-master/lib/utils/cython_nms.so: undefined symbol: PyFPE_jbuf

于是我google了一下，很多提到出现这个问题的原因是系统上存在多个python环境导致的。我想起来，一开始由于重装系统，所以在安装Detectron时，使用的是系统自带的python2.7；而后来我因为要用tensorflow，就又重新装了anaconda2，并默认使用anaconda中的python了，我想这可能就是导致我失败的最终原因，在网上找了几种解决方法，都是坑！

第一个坑：

>>> pip uninstall numpy

　　把numpy给卸载了，然后就出现 “ no module named numpy” ,我想重新给它装回去，pip install numpy却提示我系统已经安装了numpy ，可是我却无法import !!??

　　后来还是通过 pip installl –ignore-installed numpy 重新给装回去了。

第二个坑：

>>> conda remove numpy

　　我也没注意，出提示时直接输入了yes ，结果…把装好的tensorflow和其他一些相关的包都给卸载了！！？？？ WTF

好吧，没了就没了，一会给它重新装回去就是了。但是执行inference tutorial时依旧报相同的错误！！于是就进入了下面的一个坑。

第三个坑：

　　1.删除Detecron、caffe2文件夹

>>> conda install -c caffe2 caffe2-cuda9.0-cudnn7-gcc4.8

这个更改完犊子，安装某个包时，出现一个和Permision Denied有关的Error,然后安装结束了。我进入python可以import caffe2 但是，执行inference tutorial时，出现其他的错误。

然后我就想删了，conda remove caffe2 caffe2-cuda9.0-cudnn7-gcc4.8 居然删不了，每次运行完指令，还是可以import caffe2，而且错误依旧存在。

无奈，最后我使出了杀手锏：删除Anaconda2 一切重新来过。。。

后续。。

Step1：安装Anaconda2，我事先下载了安装包，所以直接安装就行

bash ./Anaconda2-4.4.0-Linux-x86_64.sh

这里没什么好说的，正常装完后检查 python -V 显示是Anaconda2的。

Step 2: 从源码编译caffe2

有一点要注意的时，这次在编译caffe2的时候，提示cmake版本最低要求 3.2，而本机自带的是2.8，所以需要升级：

>>> cd ~

>>> wget https://cmake.org/files/v3.2/cmake-3.2.2.tar.gz

>>> tar xvf cmake-3.2.2.tar.gz

>>> cd cmake-3.2.2

>>> ./bootstrap --prefix=/usr         #这一步很关键，如果没有指定prefix，后面使用时会报错Could not find CMAKE_ROOT

>>> make

>>> sudo make install

#check:

>>> cmake --version
cmake version 3.2.2

然后执行再下载和编译caffe2

Step2: 安装Caffe2,要小心了。

>>> conda install -c caffe2 caffe2-cuda9.0-cudnn7-gcc4.8

下面是Terminal里的输出：

Fetching package metadata ........... Solving package specifications: . Package plan for installation in environment /home/cc/anaconda2: The following NEW packages will be INSTALLED: caffe2-cuda9.0-cudnn7-gcc4.8: 0.8.dev-py27h67f0068_0 caffe2 future: 0.16.0-py27_1 gflags: 2.2.0-1 glog: 0.3.5-0 leveldb: 1.20-0 libprotobuf: 3.4.0-0 lmdb: 0.9.21-0 opencv: 3.1.0-np112py27_1 protobuf: 3.4.0-py27_0 snappy: 1.1.6-0 The following packages will be UPDATED: anaconda: 4.4.0-np112py27_0             --> custom-py27_0 conda: 4.3.21-py27_0                 --> 4.3.30-py27h6ae6dc7_0 The following packages will be DOWNGRADED: jpeg: 9b-0                          --> 8d-2 libtiff: 4.0.6-3                       --> 4.0.6-2 pillow: 4.1.1-py27_0                  --> 3.4.2-py27_0 qt: 5.6.2-4                       --> 5.6.2-2 Proceed ([y]/n)?

没的说，我这里只能选y吧，等了几个小时后，都下载好了。

Proceed ([y]/n)? y gflags-2.2.0-1 100% |################################| Time: 0:00:01  99.19 kB/s glog-0.3.5-0.t 100% |################################| Time: 0:00:35  18.59 kB/s jpeg-8d-2.tar. 100% |################################| Time: 0:00:47  17.43 kB/s lmdb-0.9.21-0. 100% |################################| Time: 0:00:34  17.11 kB/s snappy-1.1.6-0 100% |################################| Time: 0:00:02  19.58 kB/s leveldb-1.20-0 100% |################################| Time: 0:00:22  15.62 kB/s libprotobuf-3. 100% |################################| Time: 0:04:55  15.33 kB/s libtiff-4.0.6- 100% |################################| Time: 0:01:38  15.48 kB/s anaconda-custo 100% |################################| Time: 0:00:00  20.59 kB/s future-0.16.0- 100% |################################| Time: 0:00:40  16.27 kB/s opencv-3.1.0-n 100% |################################| Time: 0:38:52  16.46 kB/s pillow-3.4.2-p 100% |################################| Time: 0:00:59  14.74 kB/s protobuf-3.4.0 100% |################################| Time: 0:00:24  19.81 kB/s caffe2-cuda9.0 100% |################################| Time: 0:22:54  20.96 kB/s qt-5.6.2-2.tar 100% |################################| Time: 0:26:06  29.61 kB/s qt-5.6.2-2.tar 100% |################################| Time: 0:40:15  19.21 kB/s conda-4.3.30-p 100% |################################| Time: 0:00:26  19.93 kB/s

装完后，测试下看：

>>> python2 -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure" Segmentation fault (core dumped) Failure >>> python2 -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())' WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode. WARNING:root:Debug message: libnccl.so.2: cannot open shared object file: No such file or directory Segmentation fault (core dumped)

重新打开安装页面，有如下两段话：

This does NOT include libraries that are necessary to run the tutorials, such as jupyter. See the tutorials page for the list of required packages needed to run the tutorials.

NOTE: This will install Caffe2 and all of its required dependencies into the current conda environment. 
We strongly suggest that you create a new conda environment and install Caffe2 into that. 
A conda environment is like a separate python installation and so won’t have problems with your other conda environments. You can learn more about conda environments here.

我决定进一步尝试 tutorials page里的 list of required packages needed to run the tutorials.

First download the tutorials source.

>>> git clone --recursive https://github.com/caffe2/tutorials caffe2_tutorials

Cloning into ‘caffe2_tutorials’…
remote: Counting objects: 280, done.
remote: Total 280 (delta 0), reused 0 (delta 0), pack-reused 280
Receiving objects: 100% (280/280), 38.42 MiB | 71.00 KiB/s, done.
Resolving deltas: 100% (113/113), done.
Checking connectivity… done.

To run the tutorials you will need some third-party libraries, including ipython-notebooks and matplotlib. You can install everything you’ll need with the following command.

(Anaconda users: If you’re using Anaconda, use conda install instead of pip install.) Anaconda user?好家伙，那不就是我吗？

原本是这样的：

pip install -U pip setuptools pip install \ graphviz \ hypothesis \ ipython \ jupyter \ matplotlib \ notebook \ pydot \ python-nvd3 \ pyyaml \ requests \ scikit-image \ scipy

我应该输入成：

>>> conda install setuptools Fetching package metadata ........... Solving package specifications: . Package plan for installation in environment /home/cc/anaconda2: The following NEW packages will be INSTALLED: certifi: 2018.1.18-py27_0 libgcc-ng:  7.2.0-hdf63c60_3 The following packages will be UPDATED: conda: 4.3.30-py27h6ae6dc7_0 --> 4.5.0-py27_0 conda-env:  2.6.0-0               --> 2.6.0-h36134e3_1 pycosat: 0.6.2-py27_0          --> 0.6.3-py27ha4109ae_0 setuptools: 27.2.0-py27_0         --> 39.0.1-py27_0 Proceed ([y]/n)? conda-env-2.6. 100% |################################| Time: 0:00:00   8.48 kB/s libgcc-ng-7.2. 100% |################################| Time: 0:07:00  15.20 kB/s certifi-2018.1 100% |################################| Time: 0:00:06  21.14 kB/s pycosat-0.6.3- 100% |################################| Time: 0:00:05  19.45 kB/s setuptools-39. 100% |################################| Time: 0:00:40  14.60 kB/s conda-4.5.0-py 100% |################################| Time: 0:01:12  14.74 kB/s

>>> conda install graphviz hypothesis ipython jupyter matplotlib notebook pydot python-nvd3 pyyaml requests scikit-image scipy

Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

– python-nvd3

Current channels:

– https://repo.anaconda.com/pkgs/main/linux-64
– https://repo.anaconda.com/pkgs/main/noarch
– https://repo.anaconda.com/pkgs/free/linux-64
– https://repo.anaconda.com/pkgs/free/noarch
– https://repo.anaconda.com/pkgs/r/linux-64
– https://repo.anaconda.com/pkgs/r/noarch
– https://repo.anaconda.com/pkgs/pro/linux-64
– https://repo.anaconda.com/pkgs/pro/noarch

我就把python-nvd3从安装里去掉了,接着搞:

>>> conda install graphviz hypothesis ipython jupyter matplotlib notebook pydot pyyaml requests scikit-image scipy Solving environment: done ## Package Plan ## environment location: /home/cc/anaconda2 added / updated specs: - graphviz - hypothesis - ipython - jupyter - matplotlib - notebook - pydot - pyyaml - requests - scikit-image - scipy The following packages will be downloaded: package | build ---------------------------|----------------- numpy-1.12.1               |   py27h9378851_1         3.7 MB terminado-0.8.1            |           py27_1          20 KB ca-certificates-2018.03.07 |                0         124 KB pyyaml-3.12                |   py27h2d70dd7_1         159 KB pydot-1.2.4                |           py27_0          37 KB attrs-17.4.0               |           py27_0          40 KB graphviz-2.38.0            |                2        12.0 MB hypothesis-3.53.0          |           py27_0         251 KB requests-2.18.4            |   py27hc5b0589_1          90 KB scikit-image-0.13.1        |   py27h14c3975_1        23.2 MB urllib3-1.22               |   py27ha55213b_0         153 KB backports.shutil_get_terminal_size-1.0.0|   py27h5bc021e_2           8 KB notebook-5.4.1             |           py27_0         6.6 MB libgfortran-ng-7.2.0       |       hdf63c60_3         1.2 MB jupyter-1.0.0              |           py27_4           5 KB intel-openmp-2018.0.0      |                8         620 KB ipython-5.6.0              |           py27_0        1020 KB jupyter_core-4.4.0         |   py27h345911c_0          60 KB send2trash-1.5.0           |           py27_0          16 KB openssl-1.0.2o             |       h20670df_0         3.4 MB scikit-learn-0.19.1        |   py27h445a80a_0         5.3 MB pysocks-1.6.8              |           py27_0          22 KB libstdcxx-ng-7.2.0         |       hdf63c60_3         2.5 MB scipy-1.0.1                |   py27hfc37229_0        17.7 MB numexpr-2.6.4              |   py27hd318778_0         180 KB jupyter_client-5.2.3       |           py27_0         122 KB mkl-2018.0.2               |                1       205.2 MB coverage-4.5.1             |   py27h14c3975_0         208 KB imageio-2.3.0              |           py27_0         3.3 MB ------------------------------------------------------------ Total: 287.4 MB The following NEW packages will be INSTALLED: attrs: 17.4.0-py27_0 backports.shutil_get_terminal_size: 1.0.0-py27h5bc021e_2 ca-certificates:                    2018.03.07-0 coverage: 4.5.1-py27h14c3975_0 graphviz: 2.38.0-2 hypothesis: 3.53.0-py27_0 imageio: 2.3.0-py27_0 intel-openmp:                       2018.0.0-8 libgfortran-ng:                     7.2.0-hdf63c60_3 libstdcxx-ng:                       7.2.0-hdf63c60_3 pydot: 1.2.4-py27_0 pysocks: 1.6.8-py27_0 send2trash: 1.5.0-py27_0 urllib3: 1.22-py27ha55213b_0 The following packages will be UPDATED: ipython: 5.3.0-py27_0         --> 5.6.0-py27_0 jupyter: 1.0.0-py27_3         --> 1.0.0-py27_4 jupyter_client: 5.0.1-py27_0         --> 5.2.3-py27_0 jupyter_core: 4.3.0-py27_0         --> 4.4.0-py27h345911c_0 mkl: 2017.0.1-0           --> 2018.0.2-1 notebook: 5.0.0-py27_0         --> 5.4.1-py27_0 numexpr: 2.6.2-np112py27_0    --> 2.6.4-py27hd318778_0 numpy: 1.12.1-py27_0        --> 1.12.1-py27h9378851_1 openssl: 1.0.2l-0             --> 1.0.2o-h20670df_0 pyyaml: 3.12-py27_0          --> 3.12-py27h2d70dd7_1 requests: 2.14.2-py27_0        --> 2.18.4-py27hc5b0589_1 scikit-image:                       0.13.0-np112py27_0   --> 0.13.1-py27h14c3975_1 scikit-learn:                       0.18.1-np112py27_1   --> 0.19.1-py27h445a80a_0 scipy: 0.19.0-np112py27_0   --> 1.0.1-py27hfc37229_0 terminado: 0.6-py27_0           --> 0.8.1-py27_1 Proceed ([y]/n)?