Issue
Having trouble with CUDA + Pytorch this is the error. I reinstalled CUDA and cudnn multiple times.
Conda env is detecting GPU but its giving errors with pytorch and certain cuda libraries. I tried with Cuda 10.1 and 10.0, and cudnn version 8 and 7.6.5, Added cuda to path and everything.
However anaconda is showing cuda tool kit 9.0 is installed, whilst I clearly installed 10.0, so I am not entirely sure what's the deal with that.
=> loading model from models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth
Traceback (most recent call last):
File "hydroman2.py", line 580, in <module>
pose_model.load_state_dict(torch.load(cfg.TEST.MODEL_FILE), strict=False)
File "C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py", line 593, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py", line 773, in _legacy_load
result = unpickler.load()
File "C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py", line 729, in persistent_load
deserialized_objects[root_key] = restore_location(obj, location)
File "C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py", line 178, in default_restore_location
result = fn(storage, location)
File "C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py", line 154, in _cuda_deserialize
device = validate_cuda_device(location)
File "C:\Users\Fardin\anaconda3\envs\myenv\lib\site-packages\torch\serialization.py", line 138, in validate_cuda_device
raise RuntimeError('Attempting to deserialize object on a CUDA '
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
System info
System info:
--------------------------------------------------------------------------------
__Time Stamp__
Report started (local time) : 2021-03-19 19:59:06.957967
UTC start time : 2021-03-19 15:59:06.957967
Running time (s) : 4.003899
__Hardware Information__
Machine : AMD64
CPU Name : znver1
CPU Count : 12
Number of accessible CPUs : 12
List of accessible CPUs cores : 0 1 2 3 4 5 6 7 8 9 10 11
CFS Restrictions (CPUs worth of runtime) : None
CPU Features : 64bit adx aes avx avx2 bmi bmi2
clflushopt clzero cmov cx16 cx8
f16c fma fsgsbase fxsr lzcnt mmx
movbe mwaitx pclmul popcnt prfchw
rdrnd rdseed sahf sha sse sse2
sse3 sse4.1 sse4.2 sse4a ssse3
xsave xsavec xsaveopt xsaves
Memory Total (MB) : 16334
Memory Available (MB) : 8787
__OS Information__
Platform Name : Windows-10-10.0.19041-SP0
Platform Release : 10
OS Name : Windows
OS Version : 10.0.19041
OS Specific Version : 10 10.0.19041 SP0 Multiprocessor Free
Libc Version : ?
__Python Information__
Python Compiler : MSC v.1916 64 bit (AMD64)
Python Implementation : CPython
Python Version : 3.8.5
Python Locale : en_US.cp1252
__LLVM Information__
LLVM Version : 10.0.1
__CUDA Information__
CUDA Device Initialized : True
CUDA Driver Version : 11020
CUDA Detect Output:
Found 1 CUDA devices
id 0 b'GeForce GTX 1070' [SUPPORTED]
compute capability: 6.1
pci device id: 0
pci bus id: 6
Summary:
1/1 devices are supported
CUDA Librairies Test Output:
Finding cublas from <unknown>
named cublas.dll
trying to open library... ERROR: failed to open cublas:
Could not find module 'cublas.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding cusparse from <unknown>
named cusparse.dll
trying to open library... ERROR: failed to open cusparse:
Could not find module 'cusparse.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding cufft from <unknown>
named cufft.dll
trying to open library... ERROR: failed to open cufft:
Could not find module 'cufft.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding curand from <unknown>
named curand.dll
trying to open library... ERROR: failed to open curand:
Could not find module 'curand.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding nvvm from <unknown>
named nvvm.dll
trying to open library... ERROR: failed to open nvvm:
Could not find module 'nvvm.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding cudart from <unknown>
named cudart.dll
trying to open library... ERROR: failed to open cudart:
Could not find module 'cudart.dll' (or one of its dependencies). Try using the full path with constructor syntax.
Finding libdevice from <unknown>
searching for compute_20... ERROR: can't open libdevice for compute_20
searching for compute_30... ERROR: can't open libdevice for compute_30
searching for compute_35... ERROR: can't open libdevice for compute_35
searching for compute_50... ERROR: can't open libdevice for compute_50
__ROC information__
ROC Available : False
ROC Toolchains : None
HSA Agents Count : 0
HSA Agents:
None
HSA Discrete GPUs Count : 0
HSA Discrete GPUs : None
__SVML Information__
SVML State, config.USING_SVML : True
SVML Library Loaded : True
llvmlite Using SVML Patched LLVM : True
SVML Operational : True
__Threading Layer Information__
TBB Threading Layer Available : False
+--> Disabled due to Unknown import problem.
OpenMP Threading Layer Available : True
+-->Vendor: MS
Workqueue Threading Layer Available : True
+-->Workqueue imported successfully.
__Numba Environment Variable Information__
None found.
__Conda Information__
Conda Build : 3.20.5
Conda Env : 4.9.2
Conda Platform : win-64
Conda Python Version : 3.8.5.final.0
Conda Root Writable : True
__Installed Packages__
_pytorch_select 1.1.0 cpu anaconda
_tflow_select 2.3.0 mkl anaconda
absl-py 0.12.0 pypi_0 pypi
alabaster 0.7.12 pypi_0 pypi
appdirs 1.4.3 py36h28b3542_0 anaconda
argparse 1.4.0 pypi_0 pypi
asn1crypto 1.3.0 py36_0 anaconda
astor 0.8.1 pyh9f0ad1d_0 conda-forge
astunparse 1.6.3 pypi_0 pypi
atomicwrites 1.4.0 py_0 anaconda
attrs 19.3.0 py_0 anaconda
babel 2.9.0 pypi_0 pypi
backcall 0.2.0 py_0 anaconda
backports 1.0 py_2 anaconda
backports.weakref 1.0.post1 py36h9f0ad1d_1001 conda-forge
blas 1.0 mkl anaconda
bleach 1.5.0 py36_0 conda-forge
blinker 1.4 py_1 conda-forge
brotlipy 0.7.0 py36he774522_1000 anaconda
bzip2 1.0.8 he774522_0 anaconda
ca-certificates 2020.10.14 0 anaconda
cachetools 4.1.1 py_0 anaconda
certifi 2020.6.20 py36_0 anaconda
cffi 1.14.0 py36h7a1dbc1_0 anaconda
chardet 3.0.4 py36_1003 anaconda
click 7.1.2 pyh9f0ad1d_0 conda-forge
cloudpickle 1.4.1 py_0 anaconda
colorama 0.4.3 py_0 anaconda
contextlib2 0.6.0.post1 py_0 anaconda
cpuonly 1.0 0 pytorch
cryptography 2.9.2 py36h7a1dbc1_0 anaconda
cudatoolkit 9.0 1 anaconda
cudnn 7.6.5 cuda9.0_0 anaconda
curl 7.71.0 h2a8f88b_0 anaconda
cycler 0.10.0 py36h009560c_0 anaconda
cython 0.29.22 pypi_0 pypi
cytoolz 0.10.1 py36he774522_0 anaconda
dask-core 2.19.0 py_0 anaconda
decorator 4.4.2 py_0 anaconda
defusedxml 0.6.0 py_0 anaconda
dlib 19.20 py36h5653133_1 conda-forge
docker-py 4.2.1 py36h9f0ad1d_0 conda-forge
docker-pycreds 0.4.0 py_0 anaconda
docutils 0.16 pypi_0 pypi
easydict 1.7 pypi_0 pypi
entrypoints 0.3 py36_0 anaconda
ffmpeg 2.7.0 0 menpo
flake8 3.8.3 py_0 anaconda
flake8-polyfill 1.0.2 py36_0 anaconda
flake8-quotes 3.0.0 pyh9f0ad1d_0 conda-forge
flatbuffers 1.12 pypi_0 pypi
freetype 2.10.2 hd328e21_0 anaconda
gast 0.2.2 pypi_0 pypi
geos 3.8.1 h33f27b4_0 anaconda
gettext 0.19.8.1 hb01d8f6_1002 conda-forge
git 2.23.0 h6bb4b03_0 anaconda
glib 2.58.3 py36h04c7ab9_1004 conda-forge
google-auth 1.28.0 pypi_0 pypi
google-auth-oauthlib 0.4.3 pypi_0 pypi
google-pasta 0.2.0 pyh8c360ce_0 conda-forge
grpcio 1.32.0 pypi_0 pypi
h5py 2.10.0 py36h5e291fa_0 anaconda
hdf5 1.10.4 h7ebc959_0 anaconda
html5lib 0.9999999 py36_0 conda-forge
icc_rt 2019.0.0 h0cc432a_1 anaconda
icu 58.2 ha925a31_3 anaconda
idna 2.10 py_0 anaconda
imageio 2.8.0 py_0 anaconda
imageio-ffmpeg 0.4.2 py_0 conda-forge
imagesize 1.2.0 pypi_0 pypi
imgaug 0.4.0 pypi_0 pypi
importlib-metadata 1.7.0 py36_0 anaconda
importlib_metadata 1.7.0 0 anaconda
intel-openmp 2019.4 245 anaconda
ipykernel 5.3.0 py36h5ca1d4c_0 anaconda
ipyparallel 6.3.0 pypi_0 pypi
ipython 7.16.1 py36h5ca1d4c_0 anaconda
ipython_genutils 0.2.0 py36_0 anaconda
ipywidgets 7.5.1 py_0 anaconda
jedi 0.17.1 py36_0 anaconda
jinja2 2.11.2 py_0 anaconda
joblib 0.15.1 py_0 anaconda
jpeg 9d he774522_0 conda-forge
json-tricks 3.15.5 pypi_0 pypi
jsonschema 3.2.0 py36_0 anaconda
jupyter 1.0.0 py36_7 anaconda
jupyter_client 6.1.3 py_0 anaconda
jupyter_console 6.1.0 py_0 anaconda
jupyter_core 4.6.3 py36_0 anaconda
keras-applications 1.0.8 py_1 anaconda
keras-preprocessing 1.1.2 pypi_0 pypi
kiwisolver 1.2.0 py36h74a9793_0 anaconda
krb5 1.18.2 hc04afaa_0 anaconda
leptonica 1.78.0 h919f142_2 conda-forge
libarchive 3.3.3 h0643e63_5 anaconda
libcurl 7.71.0 h2a8f88b_0 anaconda
libffi 3.2.1 h6538335_1007 conda-forge
libgpuarray 0.7.6 hfa6e2cd_1003 conda-forge
libiconv 1.15 vc14h29686d3_5 [vc14] anaconda
libmklml 2019.0.5 0 anaconda
libpng 1.6.37 h2a8f88b_0 anaconda
libprotobuf 3.12.3 h7bd577a_0 anaconda
libsodium 1.0.18 h62dcd97_0 anaconda
libssh2 1.9.0 h7a1dbc1_1 anaconda
libtiff 4.1.0 h56a325e_0 anaconda
libwebp 1.0.2 hfa6e2cd_5 conda-forge
libxml2 2.9.10 h464c3ec_1 anaconda
libxslt 1.1.34 he774522_0 anaconda
lxml 4.5.0 py36h1350720_0 anaconda
lz4-c 1.8.1.2 h2fa13f4_0 anaconda
lzo 2.10 he774522_2 anaconda
m2w64-gcc-libgfortran 5.3.0 6 conda-forge
m2w64-gcc-libs 5.3.0 7 conda-forge
m2w64-gcc-libs-core 5.3.0 7 conda-forge
m2w64-gmp 6.1.0 2 conda-forge
m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge
mako 1.1.0 py_0 anaconda
markdown 3.3.4 pypi_0 pypi
markupsafe 1.1.1 py36he774522_0 anaconda
matplotlib 3.1.3 py36_0 anaconda
matplotlib-base 3.1.3 py36h64f37c6_0 anaconda
mccabe 0.6.1 py36_1 anaconda
mistune 0.8.4 py36he774522_0 anaconda
mkl 2018.0.3 1 anaconda
mkl_fft 1.0.6 py36hdbbee80_0 anaconda
mkl_random 1.0.1 py36h77b88f5_1 anaconda
mock 4.0.3 pypi_0 pypi
more-itertools 8.4.0 py_0 anaconda
moviepy 1.0.1 py_0 conda-forge
msys2-conda-epoch 20160418 1 conda-forge
nbconvert 5.6.1 py36_0 anaconda
nbformat 5.0.7 py_0 anaconda
networkx 2.4 py_0 anaconda
ninja 1.9.0 py36h74a9793_0 anaconda
nose 1.3.7 pypi_0 pypi
notebook 6.0.3 py36_0 anaconda
numpy 1.19.5 pypi_0 pypi
oauthlib 3.1.0 py_0 anaconda
olefile 0.46 py36_0 anaconda
opencv-python 3.4.1.15 pypi_0 pypi
openjpeg 2.3.1 h57dd2e7_3 conda-forge
openssl 1.1.1h he774522_0 anaconda
opt-einsum 3.3.0 pypi_0 pypi
packaging 20.4 py_0 anaconda
pandas 1.0.3 py36h47e9c7a_0 anaconda
pandoc 2.9.2.1 0 anaconda
pandocfilters 1.4.2 py36_1 anaconda
parso 0.7.0 py_0 anaconda
pcre 8.44 ha925a31_0 anaconda
pep8-naming 0.8.2 py36_0 anaconda
pickleshare 0.7.5 py36_0 anaconda
pillow 7.1.2 py36hcc1f983_0 anaconda
pip 20.2.4 py36_0 anaconda
pluggy 0.13.1 py36_0 anaconda
poppler 0.87.0 hdbe765f_0 conda-forge
poppler-data 0.4.9 1 conda-forge
proglog 0.1.9 py_0 conda-forge
prometheus_client 0.8.0 py_0 anaconda
prompt-toolkit 3.0.5 py_0 anaconda
prompt_toolkit 3.0.5 0 anaconda
protobuf 3.12.3 py36h33f27b4_0 anaconda
psutil 5.8.0 pypi_0 pypi
py 1.9.0 py_0 anaconda
pyasn1 0.4.8 py_0 anaconda
pyasn1-modules 0.2.8 pypi_0 pypi
pycocotools 2.0 pypi_0 pypi
pycodestyle 2.6.0 py_0 anaconda
pycparser 2.20 py_0 anaconda
pyflakes 2.2.0 py_0 anaconda
pygments 2.6.1 py_0 anaconda
pygpu 0.7.6 py36h7725771_1001 conda-forge
pyjwt 1.7.1 py_0 conda-forge
pyopenssl 19.1.0 py36_0 anaconda
pyparsing 2.4.7 py_0 anaconda
pyqt 5.9.2 py36h6538335_2 anaconda
pyreadline 2.1 py36_1001 conda-forge
pyrsistent 0.16.0 py36he774522_0 anaconda
pysocks 1.7.1 py36_0 anaconda
pytesseract 0.3.3 pyh8c360ce_0 conda-forge
pytest 5.4.3 py36_0 anaconda
python 3.6.10 h9f7ef89_1 anaconda
python-dateutil 2.8.1 py_0 anaconda
python_abi 3.6 1_cp36m conda-forge
pytorch 1.5.1 py3.6_cpu_0 [cpuonly] pytorch
pytz 2020.1 py_0 anaconda
pywavelets 1.1.1 py36he774522_0 anaconda
pywin32 223 py36hfa6e2cd_1 anaconda
pywinpty 0.5.7 py36_0 anaconda
pyyaml 5.3.1 py36he774522_0 anaconda
pyzmq 19.0.1 py36ha925a31_1 anaconda
qt 5.9.7 vc14h73c81de_0 [vc14] anaconda
qtconsole 4.7.5 py_0 anaconda
qtpy 1.9.0 py_0 anaconda
requests 2.24.0 py_0 anaconda
requests-oauthlib 1.3.0 pyh9f0ad1d_0 conda-forge
rsa 4.6 pyh9f0ad1d_0 conda-forge
scikit-image 0.16.2 py36h47e9c7a_0 anaconda
scikit-learn 0.20.1 py36hb854c30_0 anaconda
scipy 1.4.1 pypi_0 pypi
send2trash 1.5.0 py36_0 anaconda
setuptools 50.3.0 py36h9490d1a_1 anaconda
shapely 1.6.4 pypi_0 pypi
simplejson 3.17.0 py36he774522_0 anaconda
sip 4.19.8 py36h6538335_0 anaconda
six 1.15.0 py_0 anaconda
sklearn 0.0 pypi_0 pypi
slidingwindow 0.0.14 pypi_0 pypi
snowballstemmer 2.1.0 pypi_0 pypi
sphinx 3.5.2 pypi_0 pypi
sphinxcontrib-applehelp 1.0.2 pypi_0 pypi
sphinxcontrib-devhelp 1.0.2 pypi_0 pypi
sphinxcontrib-htmlhelp 1.0.3 pypi_0 pypi
sphinxcontrib-jsmath 1.0.1 pypi_0 pypi
sphinxcontrib-qthelp 1.0.3 pypi_0 pypi
sphinxcontrib-serializinghtml 1.1.4 pypi_0 pypi
sqlite 3.32.3 h2a8f88b_0 anaconda
swig 3.0.12 h047fa9f_3 anaconda
tbb 2020.0 h74a9793_0 anaconda
tbb4py 2020.0 py36h74a9793_0 anaconda
tensorboard 1.13.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.0 pypi_0 pypi
tensorboardx 1.6 py_0 conda-forge
tensorflow 2.4.1 pypi_0 pypi
tensorflow-estimator 1.13.0 pypi_0 pypi
tensorflow-gpu 1.13.1 pypi_0 pypi
tensorflow-gpu-estimator 2.1.0 pypi_0 pypi
termcolor 1.1.0 pypi_0 pypi
terminado 0.8.3 py36_0 anaconda
testpath 0.4.4 py_0 anaconda
theano 1.0.4 py36h003fed8_1002 conda-forge
threadpoolctl 2.1.0 pyh5ca1d4c_0 anaconda
tk 8.6.10 he774522_0 anaconda
toolz 0.10.0 py_0 anaconda
torchfile 0.1.0 py_0 conda-forge
torchvision 0.6.1 py36_cpu [cpuonly] pytorch
tornado 6.0.4 py36he774522_1 anaconda
tqdm 4.47.0 py_0 anaconda
traitlets 4.3.3 py36_0 anaconda
typing-extensions 3.7.4.3 pypi_0 pypi
urllib3 1.25.11 py_0 anaconda
vc 14.1 h0510ff6_4 anaconda
visdom 0.1.8.9 0 conda-forge
vs2015_runtime 14.16.27012 hf0eaf9b_3 anaconda
vs2017_win-64 19.16.27038 h2e3bad8_2 conda-forge
vswhere 2.7.1 h21ff451_0 anaconda
wcwidth 0.2.5 py_0 anaconda
webencodings 0.5.1 py36_1 anaconda
websocket-client 0.57.0 py36_1 anaconda
werkzeug 1.0.1 pyh9f0ad1d_0 conda-forge
wget 1.16.3 0 menpo
wheel 0.35.1 py_0 anaconda
widgetsnbextension 3.5.1 py36_0 anaconda
win_inet_pton 1.1.0 py36_0 anaconda
wincertstore 0.2 py36h7fe50ca_0 anaconda
winpty 0.4.3 4 anaconda
wrapt 1.12.1 py36h68a101e_1 conda-forge
xz 5.2.5 h62dcd97_0 anaconda
yacs 0.1.8 pypi_0 pypi
yaml 0.1.7 hc54c509_2 anaconda
zeromq 4.3.2 ha925a31_2 anaconda
zipp 3.3.1 py_0 anaconda
zlib 1.2.11 h62dcd97_4 anaconda
zstd 1.3.7 h508b16e_0 anaconda
No errors reported.
Solution
Solved.
Pytorch was installing CPU only version for some reason, reinstalling pytorch didn't help.
Uninstalling pytorch: conda uninstall pytorch
Followed by uninstalling cpu only: conda uninstall cpuonly
Then installing pytorch again solved it.
Answered By - bboyenergetic
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.