tensorflow2.1之gpu和cpu问题

安装nezha环境遇到的问题

项目组开发了一个grpc的项目涉及到机器学习框架tensorflow,我把这个项目代码拉取下来需要放到本地跑客户端来连接部署在远程k8s集群中的server端。

首先我就需要对requirements.txt中需要用到的软件包进行批量安装

前戏

pip install -r requirements.txt

在安装过程中出现下面一系列问题，好在都通过pip命令进行解决了

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way t
hat it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

paddlepaddle 1.8.5 requires gast==0.3.3, but you'll have gast 0.2.2 which is incompatible.
paddlepaddle 1.8.5 requires opencv-python<=4.2.0.32, but you'll have opencv-python 4.4.0.46 which is incompatible.

pip uninstall gast
pip install gast==0.3.3
pip uninstall opencv-python
pip install==opencv-python==4.2.0.32


Uninstalling pybind11-2.6.1:
      Successfully uninstalled pybind11-2.6.1
    Running setup.py install for fasttext ... error
    ERROR: Command errored out with exit status 1:
     command: 'd:\pythonproject1\venv\scripts\python.exe' -u -c 'import sys, setuptools
, tokenize; sys.argv[0] = '"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pip-install-rmpf
lwfu\\fasttext\\setup.py'"'"'; __file__='"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pi
p-install-rmpflwfu\\fasttext\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(_
_file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(cod
e, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\shupe\AppData\Local\Temp\pip-
record-gljkd8cp\install-record.txt' --single-version-externally-managed --compile --ins
tall-headers 'd:\pythonproject1\venv\include\site\python3.7\fasttext'
         cwd: C:\Users\shupe\AppData\Local\Temp\pip-install-rmpflwfu\fasttext\
    Complete output (18 lines):
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.7
    creating build\lib.win-amd64-3.7\fasttext
    copying python\fasttext_module\fasttext\FastText.py -> build\lib.win-amd64-3.7\fast
text
    copying python\fasttext_module\fasttext\__init__.py -> build\lib.win-amd64-3.7\fast
text
    creating build\lib.win-amd64-3.7\fasttext\util
    copying python\fasttext_module\fasttext\util\util.py -> build\lib.win-amd64-3.7\fas
ttext\util
    copying python\fasttext_module\fasttext\util\__init__.py -> build\lib.win-amd64-3.7
\fasttext\util
    creating build\lib.win-amd64-3.7\fasttext\tests
    copying python\fasttext_module\fasttext\tests\test_configurations.py -> build\lib.w
in-amd64-3.7\fasttext\tests
    copying python\fasttext_module\fasttext\tests\test_script.py -> build\lib.win-amd64
-3.7\fasttext\tests
    copying python\fasttext_module\fasttext\tests\__init__.py -> build\lib.win-amd64-3.
7\fasttext\tests
    running build_ext
    building 'fasttext_pybind' extension
    error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++
xe' -u -c 'import sys, setuptools, toxe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\     xe' -u -xe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pip-inst
all-rmpflwfu\\fasttext\\setup.py'"'"'; __file__='"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pip-install-rmpflwf
u\\fasttext\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"
'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\shupe\App
Data\Local\Temp\pip-record-gljkd8cp\install-record.txt' --single-version-externally-managed --compile --install-
headers 'd:\pythonproject1\venv\include\site\python3.7\fasttext' Check the logs for full command output.
xe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pip-install-rmpflwfu\\fastt
ext\\setup.py'"'"'; __file__='"'"'C:\\Users\\shupe\\AppData\\Local\\Temp\\pip-install-rmpflwfu\\fasttext\\setup.py'"'"';f=getattr(t
okenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__,
 '"'"'exec'"'"'))' install --record 'C:\Users\shupe\AppData\Local\Temp\pip-record-gljkd8cp\install-record.txt' --single-version-ext
ernally-managed --compile --install-headers 'd:\pythonproject1\venv\include\site\python3.7\fasttext' Check the logs for full comman
d output.



ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way t
hat it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

tensorflow 2.1.0 requires gast==0.2.2, but you'll have gast 0.3.3 which is incompatible.
Successfully installed gast-0.3.3

>这里很奇怪 paddlepaddle 1.8.5 需要gast==0.3.3,tensorflow 2.1.0 需要 gast==0.2.2


ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way t
hat it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

paddlepaddle 1.8.5 requires gast==0.3.3, but you'll have gast 0.2.2 which is incompatible.

>上面是不是很坑呢？不是的，因为人家项目requirements.txt 文件中没有要求安装 paddlepaddle 这款软件，可能是之前我安装了，而这款 paddlepaddle 1.8.5 又依赖 gast==0.3.3,所以就装了gast的0.3.3版本。人家项目 requiresment.txt 中tensorflow 2.1.0安装依赖gast 0.2.2,所以有冲突，我就把这个paddlepaddle 卸载了

pip uninstall paddlepaddle
pip install -r requirements.txt  # 这条命令执行后，来到我的黑暗时刻


2020-11-25 16:45:40.694415: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2020-11-25 16:45:40.695091: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
  File "D:/pythonproject1/nezha/nezha/client.py", line 3, in <module>
    from nlp import nlp_pb2, nlp_pb2_grpc
ImportError: cannot import name 'nlp_pb2' from 'nlp' (D:\pythonproject1\venv\lib\site-packages\nlp\__init__.py)

Process finished with exit code 1

# 注：V:显示全部信息  D:显示调试信息 I:显示一般信息 W:显示警告信息 E:显示错误信息
# 下面我们一个一个看错误
w 是警告信息  Could not load dynamic library '[cudart64_101.dll]'; dlerror: cudart64_101.dll not found
大意就是不能动态加载这个库- cudart64_101.dll, 这个库没有找到
I 是正常提示信息 Ignore above cudart dlerror if you do not have a GPU set up on your machine
大意就是如果你计算机上没有设置GPU，请忽略上面那个 cudart dlerror 错误

# ImportError: cannot import name 'nlp_pb2' from 'nlp'
由于我已经按照了第三方nlp包，所以本地自己创建的nlp包里的方法就无法导入运行的程序中,把已经按照nlp包卸载就可以了
解决办法：pip uninstall nlp

参考连接1：

https://blog.csdn.net/qq_41999081/article/details/104515513

内容如下：

参考连接2：

https://zhuanlan.zhihu.com/p/125690923

我本地计算机的情况和链接2类似，也是找不到那个文件夹，看来是计算机没有安装过CUDA 这个东东！查看我自己的笔记本的显示适配器：

https://zhuanlan.zhihu.com/p/125690923

查看我的显卡是支持CUDA的，但是支持什么版本的9.0，10.0，10.1，还是10.2呢？
我在查看了自己本机关于NVIDIA的显卡驱动程序版本:
控制面板->搜索NVIDIA->双击进入NVIDIA控制面板：
点击帮助->系统信息->组件：

参考链接：https://blog.csdn.net/qq_27825451/article/details/89082978

我错误以为就是按照NVIDIA CUDA 10.2.178 driver 版本来安装 CUDA，但是当我下载这个10.2版本的驱动时发现：

下载完成后，我点击安装时，出现了下图：

然而NVIDIA的显卡驱动程序和CUDA完全是两个不同的概念哦，CUDA是NVIDIA推出的用于自家GPU的并行计算框架，也就是说CUDA只能在NVIDIA的GPU上运行，而且只有当要解决的计算问题是可以大量并行计算的时候才能发挥CUDA的作用。CUDA的本质是一个工具包（ToolKit）；二者虽然不一样的。

显卡驱动的安装：

当我们使用一台电脑的时候默认的已经安装了NVIDIA的显卡驱动，因为没有显卡驱动根本用不了显卡嘛，但是这个时候我们是没有CUDA可以用的，我们可以更新我们的驱动，更新链接为：
https://www.nvidia.com/Download/index.aspx?lang=en-us
在这个里面可以根据自己的显卡类型选择最新的驱动程序。显卡驱动程序当前大小大概500多M。

CUDA ToolKit的安装：

CUDA的下载地址为：https://developer.nvidia.com/cuda-downloads
接着我就尝试了安装 10.1 ，10.1也不行，就按照10.0，10.0是可以的。
并且从网上下载了动态库cudart64_101.dll http://docs.tangwww.top:18080/CUDA/cudart64_101.zip, 解压后，把该文件放到了本地目录C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.0\bin下

最有用文档：

https://blog.csdn.net/qq_27825451/article/details/89082978

里面重点内容：



参考下面链接：
https://developer.download.nvidia.cn/compute/cuda/9.0/Prod/docs/sidebar/CUDA_Installation_Guide_Windows.pdf
https://blog.csdn.net/qq_27825451/article/details/89082978
https://blog.csdn.net/qq_41999081/article/details/104515513
https://www.cnblogs.com/jfdwd/p/11178704.html
https://blog.csdn.net/SpadgerZ/article/details/89454247
https://www.mmbyte.com/article/54222.html
https://www.cnblogs.com/jfdwd/p/11178704.html