实验室服务器被人在跑,用了一个GPU导致我代码报错:
2019-03-07 10:56:42.691244: E tensorflow/stream_executor/cuda/cuda_blas.cc:459] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2019-03-07 10:56:42.694239: E tensorflow/stream_executor/cuda/cuda_blas.cc:459] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2019-03-07 10:56:42.695757: E tensorflow/stream_executor/cuda/cuda_blas.cc:459] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2019-03-07 10:56:42.729995: E tensorflow/stream_executor/cuda/cuda_blas.cc:459] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2019-03-07 10:56:42.732594: E tensorflow/stream_executor/cuda/cuda_blas.cc:459] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2019-03-07 10:56:42.734765: E tensorflow/stream_executor/cuda/cuda_blas.cc:459] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2019-03-07 10:56:42.757543: E tensorflow/stream_executor/cuda/cuda_dnn.cc:352] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR
Segmentation fault (core dumped)
然后我就在代码里面加了:
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.333)
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))
还是报错,我又加了:
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "1"
运行时使用:
CUDA_VISIBLE_DEVICES=1 python test.py
在命令行运行即可,最后成功
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.87 Driver Version: 390.87 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 108... Off | 00000000:05:00.0 On | N/A |
| 70% 87C P2 182W / 250W | 10850MiB / 11175MiB | 100% Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GTX 108... Off | 00000000:06:00.0 Off | N/A |
| 25% 61C P2 64W / 250W | 4097MiB / 11178MiB | 25% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1300 G /usr/lib/xorg/Xorg 200MiB |
| 0 2126 C ./darknet 7953MiB |
| 0 10356 C python 2577MiB |
| 0 31115 G compiz 108MiB |
| 1 3649 C python 4085MiB |
+-----------------------------------------------------------------------------+