https://www.jianshu.com/p/2cf00bb9db34?utm_source=oschina-app
Poor student learning neural network must feel it too , "wait" pain . . .
Cycle time epoch worried to death, but there are dozens of hundreds of cycles waiting for you!
You stare at the screen watching the progress bar a crawl forward, really turn iron into steel, he developed a certain thought so in the future to buy a good computer ...
only the first dry as the other thing, but my heart is not at ease, always run, the result was not finished training, well, can not do the positive thing is, the phone brush it ~... ... (half an hour in the past)
Training finally over! Look at the results:
WOC ... @ # $ & & * * &Hard to force a re-training .... o (╥﹏╥) o
A few days ago Google launched a watch online, said poor children free of GPU dedicated to machine learning with! Suddenly excited me up immediately to try a bit. (Oh, bring their own ladders ...)
Tutorial:
Google Colab as Free GPU the Tutorial
and gather wool capitalist, free GPU with Google
But because their own projects with the tutorial are not the same, there are some details were not clear, check a lot of information has finally solved, here to decide my own course record:
Google Colaboratory
Google it directly:
Click to check it:
Introduction of Colaboratory found, which with some simple code to tell you how to use this stuff. You will find it with Jupyternotebook like, because it is developed on the basis of jupyter notebook:
所以很好理解，Google给你准备了一个云端的jupyter notebook，你把你的代码搬过来，在云端运行你的代码，耗费的不是你自己电脑的计算力，而是Google Colaboratory的计算力，而Google Colaboratory配备了Tesla K80 GPU！这可是贵族用的玩意儿呀，哎呀捡着大便宜了！
但是这个GPU不是默认开启的，需要你设置一下notebook，很简单：
点击笔记本上的“修改（edit）-->笔记本设置（notebook setting）”：
就出现了下面的窗口，在“硬件加速器”中选择“GPU”即可：
吼吼，就好了！
上面介绍的是Colaboratory的简单使用和怎么开启GPU，下面介绍怎么把本地的项目放到云端运行：
Google drive+Colaboratory 运行你的项目
Google Colaboratory是基于Google云端硬盘（drive）的，你的所有代码文件都会保存在Google Drive中，所有我们首先需要Google drive。
先说说我自己的是什么项目吧：
我在本地的notebook中有这样的一个做“神经网络风格迁移（neural style transfer）”的项目，目录如下：
.ipynb就是我的jupyter notebook文件，
imagenet-vgg-verydeep-19是我下载的VGG神经网络模型,
nst_utils.py里面有一些功能函数，也需要在.ipynb中调用。
什么是neural style transfer呢？就是做这个：
这个神经网络跑起来可是相当费时的，我自己的电脑，生成一个400×300大小的图片，跑200个epoch需要30分钟！所以虽然这个项目很激动人心，但是等一次结果真的很急人！
好了，现在我们开始把这个本地项目转移到Google drive上，并通过Colaboratory来运行！：
1. upload folders to Google drive
Figure NST file in the folder that I uploaded. With my local directory structure like:
File can not be opened directly, but Google drive there Colaboratory plug-in, you can open .ipybn file:
Right Select Open:
I have this document which needs to import nst_utils.py my NST file folder , because it wrote a number of important functions, but run the program, and will definitely being given that nst_utils.py can not find the file .
why? Because Colaboratory and is not associated with your Google drive! So, then set:
2. Install the necessary libraries, authorization, connect your Google drive
This Colaboratory thing is very fast hardware, it is not just a notebook, you can also use the command line when!
In all our previous code, adding a cell paste following command:
!apt-get install -y -qq software-properties-common python-software-properties module-init-tools
!add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null !apt-get update -qq 2>&1 > /dev/null !apt-get -y install -qq google-drive-ocamlfuse fuse from google.colab import auth auth.authenticate_user() from oauth2client.client import GoogleCredentials creds = GoogleCredentials.get_application_default() import getpass !google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL vcode = getpass.getpass() !echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} run! You should see something like this result:
Click on the verification link the results of which will jump to a page that displays a long string of code ,
the code to copy and paste the text box above, enter!
Well, and then enter the following command in a new cell:
!mkdir -p drive
!google-drive-ocamlfuse drive
Run! It is set up!
In this way, Google drive on and connected to Google Colaboratory Well, the basic can operate as a local file to manipulate clouds code.
3. How to import files in Google drive
Now if you write directly to:
from nst_utils import *
Still it is given that can not be found ! The place I was tortured for a long time, do not know how children.
Later found, Colaboratory seems to be to create a single virtual machine, I only ipybn file to run in a virtual machine, and no other files NST folder.
Our first step is setting, just let Colaboratory and the drive is connected, that is, Colaboratory can call the file in the drive, but should be imported by an absolute path similar manner, the correct approach would be to write :
from drive.NST.nst_utils import *
drive is our root through. "" directory to the next level , so that you can import our drive files!
Similarly, we need to import behind NST folder imagenet-vgg-verydeep-19.mat file, which is our neural network model, so writing is wrong:
model = load_vgg_model("imagenet-vgg-verydeep-19.mat")
The correct wording is:
model = load_vgg_model("drive/NST/imagenet-vgg-verydeep-19.mat")
Similarly, as long as the file to be imported in the drive, all in accordance with a method similar to the absolute path to import it.
4. see how this GPU effects
Specific code does not show, just for everyone to look at the results of the model runs:
Run the 200 epoch, it took a total of 46 seconds!
And I'm running locally, it took 30 minutes or so ~
This shows that the faster the pieces! Google Dafa Well ~
attached below the results of each of the 20 iterations:
The beginning of the picture are these two:
The last picture is generated:
Well, on the first use on Colaboratory introduced here, mainly I only know so much. . . In the future there will continue to update new experiences!
August 1 Update:
These days found in the use, the mount command that two of google drive, every time a new notebook needs to run once , so we can put the two commands in a convenient place to view, copy and paste any time .
Furthermore, a project may be to the next day you can not connect the drive, this time need to re-run the command . Anyway, my projects are up on the need to re-connect drive the next morning.
So it is somewhat cumbersome. . .
In addition, the first time the command is run, jumped out to a link, then click and enter the verification code, it might incite an identical link, while the entire layout like to be emptied, like this time, simply click the link again, enter the code again and press enter to see the "Access token retrieved correctly." to explain it.
Then do not forget to start a new line, run:
!mkdir -p drive
!google-drive-ocamlfuse drive
Then you can connect the drive.
Welcome attention to my "deep learning" column:
DeepLearning.ai study notes
with me a deep learning step by step to learn it!
Author: Stack_empty
link: https: //www.jianshu.com/p/2cf00bb9db34
Source: Jane book
Jane book copyright reserved by the authors, are reproduced in any form, please contact the author to obtain authorization and indicate the source.