最近刚学完Boyd的Convex Optimization,真是对Boyd神佩服得五体投地。在他的lecture slides末尾发现原来还有进阶课程Stanford的ee364b,那本convex optimization只包括了ee364a,然而ee364b没有现成的完整slides一次性下载,只好写个爬虫挨个下载保存slides,在ee364b里的内容更加专业深入,估计实际很少用到。然后我把爬虫的代码贴上来,还好他们的网页结构比较简单,代码量不大。下载好的文件里有些是空白的,回网站一查发现确实是他们没有在里面留东西,就这样吧。
import requests import re import os from bs4 import BeautifulSoup def GetPage(url): page = requests.get(url) html = page.text return html def GetList(html): soup = BeautifulSoup(html, "html5lib") list = soup.find_all(href=re.compile("lectures/")) pdfs = [] for li in list: if (li.get('href'))[-4:] == ".pdf": pdfs.append(li.get('href')) return pdfs def DownloadPdf(pdf,root_url): path = "C:/Users/Downloads/cvx/" + pdf[9:] urls = root_url + pdf r = requests.get(urls) f = open(path, "wb") f.write(r.content) f.close() return urls url = "https://web.stanford.edu/class/ee364b/lectures.html" root_url = "https://web.stanford.edu/class/ee364b/" #print(GetList(GetPage(url))) pdfs = GetList(GetPage(url)) for pdf in pdfs: print("Download finished: "+DownloadPdf(pdf, root_url))
还有计划把Standford的cs224n的lecture slides下载下来慢慢看,就在这个代码的基础上改吧