The gpu and cpu problem of tensorflow2.1

Problems encountered in installing nezha environment

The project team developed a grpc project involving the machine learning framework tensorflow. I pulled down the project code and needed to put it on the local running client to connect to the server deployed in the remote k8s cluster.

First of all, I need to batch install the software packages that need to be used in requirements.txt

Foreplay

pip install -r requirements.txt

The following series of problems occurred during the installation process, but fortunately they were all solved through the pip command

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way t
hat it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

paddlepaddle 1.8.5 requires gast==0.3.3, but you'll have gast 0.2.2 which is incompatible.
paddlepaddle 1.8.5 requires opencv-python<=4.2.0.32, but you'll have opencv-python 4.4.0.46 which is incompatible.

pip uninstall gast
pip install gast==0.3.3
pip uninstall opencv-python
pip install==opencv-python==4.2.0.32


Uninstalling pybind11-2.6.1:
      Successfully uninstalled pybind11-2.6.1
    Running setup.py install for fasttext ... error
    ERROR: Command errored out with exit status 1:
     command: 'd:\pythonproject1\venv\scripts\python.exe' -u -c 'import sys, setuptools
, tokenize; sys.argv[0] = '"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pip-install-rmpf
lwfu\\fasttext\\setup.py'"'"'; __file__='"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pi
p-install-rmpflwfu\\fasttext\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(_
_file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(cod
e, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\shupe\AppData\Local\Temp\pip-
record-gljkd8cp\install-record.txt' --single-version-externally-managed --compile --ins
tall-headers 'd:\pythonproject1\venv\include\site\python3.7\fasttext'
         cwd: C:\Users\shupe\AppData\Local\Temp\pip-install-rmpflwfu\fasttext\
    Complete output (18 lines):
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.7
    creating build\lib.win-amd64-3.7\fasttext
    copying python\fasttext_module\fasttext\FastText.py -> build\lib.win-amd64-3.7\fast
text
    copying python\fasttext_module\fasttext\__init__.py -> build\lib.win-amd64-3.7\fast
text
    creating build\lib.win-amd64-3.7\fasttext\util
    copying python\fasttext_module\fasttext\util\util.py -> build\lib.win-amd64-3.7\fas
ttext\util
    copying python\fasttext_module\fasttext\util\__init__.py -> build\lib.win-amd64-3.7
\fasttext\util
    creating build\lib.win-amd64-3.7\fasttext\tests
    copying python\fasttext_module\fasttext\tests\test_configurations.py -> build\lib.w
in-amd64-3.7\fasttext\tests
    copying python\fasttext_module\fasttext\tests\test_script.py -> build\lib.win-amd64
-3.7\fasttext\tests
    copying python\fasttext_module\fasttext\tests\__init__.py -> build\lib.win-amd64-3.
7\fasttext\tests
    running build_ext
    building 'fasttext_pybind' extension
    error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++
xe' -u -c 'import sys, setuptools, toxe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\     xe' -u -xe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pip-inst
all-rmpflwfu\\fasttext\\setup.py'"'"'; __file__='"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pip-install-rmpflwf
u\\fasttext\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"
'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\shupe\App
Data\Local\Temp\pip-record-gljkd8cp\install-record.txt' --single-version-externally-managed --compile --install-
headers 'd:\pythonproject1\venv\include\site\python3.7\fasttext' Check the logs for full command output.
xe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pip-install-rmpflwfu\\fastt
ext\\setup.py'"'"'; __file__='"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pip-install-rmpflwfu\\fasttext\\setup.py'"'"';f=getattr(t
okenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__,
 '"'"'exec'"'"'))' install --record 'C:\Users\shupe\AppData\Local\Temp\pip-record-gljkd8cp\install-record.txt' --single-version-ext
ernally-managed --compile --install-headers 'd:\pythonproject1\venv\include\site\python3.7\fasttext' Check the logs for full comman
d output.



ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way t
hat it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

tensorflow 2.1.0 requires gast==0.2.2, but you'll have gast 0.3.3 which is incompatible.
Successfully installed gast-0.3.3

>这里很奇怪 paddlepaddle 1.8.5 需要gast==0.3.3,tensorflow 2.1.0 需要 gast==0.2.2


ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way t
hat it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

paddlepaddle 1.8.5 requires gast==0.3.3, but you'll have gast 0.2.2 which is incompatible.

>上面是不是很坑呢？不是的，因为人家项目requirements.txt 文件中没有要求安装 paddlepaddle 这款软件，可能是之前我安装了，而这款 paddlepaddle 1.8.5 又依赖 gast==0.3.3,所以就装了gast的0.3.3版本。人家项目 requiresment.txt 中tensorflow 2.1.0安装依赖gast 0.2.2,所以有冲突，我就把这个paddlepaddle 卸载了

pip uninstall paddlepaddle
pip install -r requirements.txt  # 这条命令执行后，来到我的黑暗时刻


2020-11-25 16:45:40.694415: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2020-11-25 16:45:40.695091: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
  File "D:/pythonproject1/nezha/nezha/client.py", line 3, in <module>
    from nlp import nlp_pb2, nlp_pb2_grpc
ImportError: cannot import name 'nlp_pb2' from 'nlp' (D:\pythonproject1\venv\lib\site-packages\nlp\__init__.py)

Process finished with exit code 1

# 注：V:显示全部信息  D:显示调试信息 I:显示一般信息 W:显示警告信息 E:显示错误信息
# 下面我们一个一个看错误
w 是警告信息  Could not load dynamic library '[cudart64_101.dll]'; dlerror: cudart64_101.dll not found
大意就是不能动态加载这个库- cudart64_101.dll, 这个库没有找到
I 是正常提示信息 Ignore above cudart dlerror if you do not have a GPU set up on your machine
大意就是如果你计算机上没有设置GPU，请忽略上面那个 cudart dlerror 错误

# ImportError: cannot import name 'nlp_pb2' from 'nlp'
由于我已经按照了第三方nlp包，所以本地自己创建的nlp包里的方法就无法导入运行的程序中,把已经按照nlp包卸载就可以了
解决办法：pip uninstall nlp

Reference connection 1:

https://blog.csdn.net/qq_41999081/article/details/104515513

The content is as follows:

Reference connection 2:

https://zhuanlan.zhihu.com/p/125690923

The situation of my local computer is similar to that of link 2 , but I can't find that folder. It seems that the computer has not installed CUDA this stuff! Check the display adapter of my own notebook:

https://zhuanlan.zhihu.com/p/125690923

Check if my graphics card supports CUDA, but what version of 9.0, 10.0, 10.1, or 10.2 does it support?
I was checking the version of the NVIDIA graphics card driver for my machine:
Control Panel -> Search NVIDIA -> Double-click to enter the NVIDIA Control Panel:
Click Help -> System Information -> Components:

Reference link: https://blog.csdn.net/qq_27825451/article/details/89082978

I mistakenly thought that CUDA was installed according to the NVIDIA CUDA 10.2.178 driver version, but when I downloaded the 10.2 version of the driver, I found:

After the download is complete, when I click Install, the following picture appears:

However NVIDIA, the graphics card driver is a CUDAcompletely different concept. It CUDAis a parallel computing framework NVIDIAintroduced for its own GPU, which means that it CUDAcan only run on NVIDIA GPUs, and only when the computing problem to be solved can be massively parallel. It can only play CUDAa role when calculating . CUDAThe essence is a toolkit (ToolKit) ; although the two are not the same.

Graphics driver installation:

When we use a computer, the default NVIDIAgraphics card driver is already installed , because the graphics card CUDAcan not be used without the graphics card driver, but at this time we are not available. We can update our driver. The update link is:
https: //www.nvidia.com/Download/index.aspx?lang=en-us
Here you can choose the latest driver according to your graphics card type. The current size of the graphics card driver is about 500M.

Installation of CUDA ToolKit:

The download address of CUDA is: https://developer.nvidia.com/cuda-downloads
Then I tried the installation 10.1, 10.1but it didn't work, just follow it 10.0, 10.0it's ok .
And download the dynamic library cudart64_101.dll http://docs.tangwww.top:18080/CUDA/cudart64_101.zip from the Internet, after decompression, the paper put the local directory C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\binunder

The most useful documents:

https://blog.csdn.net/qq_27825451/article/details/89082978

Key content inside:



refer to the following link:
https://developer.download.nvidia.cn/compute/cuda/9.0/Prod/docs/sidebar/CUDA_Installation_Guide_Windows.pdf
https://blog.csdn.net/qq_27825451/article/details /89082978
https://blog.csdn.net/qq_41999081/article/details/104515513
https://www.cnblogs.com/jfdwd/p/11178704.html
https://blog.csdn.net/SpadgerZ/article/ details/89454247
https://www.mmbyte.com/article/54222.html
https://www.cnblogs.com/jfdwd/p/11178704.html