1 Andrew NG Deep Learning.ai
http://deeplearning.ai/
NetEase Cloud Classroom (Chinese subtitles): http://mooc.study.163.com/smartSpec/detail/1001319001.htm
Recommended reason: Mr. Andrew Ng is an expert in teaching. Many people know him from Stanford's classic "Machine Learning" course. Mr. Andrew's teaching ideas are clear and concise. I believe this first course he launched after he announced his entrepreneurship will not disappoint.
Recently, Andrew Ng tweeted about the course. I've been following him since early 2014, and I'm taking one of his courses on Coursera to learn mathematics related to machine learning. Coming from an engineering background, his first course was very interesting and a bit difficult at the same time. At the end of 2016, Mr. Wu resigned from Baidu, no longer served as Baidu's chief scientist, and launched a new deep learning specialization course on August 8 this year. I only remembered it later in the year because my time had been occupied by some other projects. Then I read a blog from Arvind N's student on Fast.ai about how he completed all 3 lessons in 4 days and his thoughts on fast.ai and deeplearning.ai.
Blog for all 3 lessons in 4 days
https://medium.com/towards-data-science/thoughts-after-taking-the-deeplearning-ai-courses-8568f132153
I wanted to challenge myself to see if I could do the same in less than 4 days. And I succeeded and completed 3 courses in 3 days.
About the instructor:
Andrew Ng is a professor at Stanford University, co-founded Coursera, founded and led the deep learning project at Google Brain, and was a former chief scientist at Baidu. Courses reflect what he has learned on a variety of large-scale problems.
This course helps you understand the mathematics required for deep learning, and by the end of course 2, learn how to build several key components of deep learning algorithms from scratch.
About the course:
The course is divided into 5 sections. As of this blog post (August 19th), the first 3 lessons have been published. Andrew Ng adopts a bottom-up teaching arrangement in this course. In his earlier courses he chose octave for programming assignments, but this course he chose python.
1. In Course 1, he provides a detailed introduction to the mathematics and intuition required for many concepts in deep learning. He carefully balances course content and teaches math, which is necessary to understand the fundamentals of deep learning. Designated sessions are very helpful for practicing mathematics programmatically. All the formulas are already provided, so you can focus on implementing them without much math knowledge.
2. Course 2 covers many techniques such as regularization, momentum, batch normalization and dropout to improve the performance of DL models. The best part of this course is using python and numpy to implement all the techniques.
3. In the 3rd class, he introduces many tips and tricks he has learned from years of experience. At the end of the third part of the course, he introduced the DL framework. The assignment for course 3 is to learn how to use TensorFlow. The job design is very clear.
4. Lesson 4 is about CNNs. I will update this section when the course is published.
5. Lesson 5 is about RNNs or sequence data. I will update this section once the course is released.
Highlights:
1. When you finish the first 3 lessons, your basic knowledge will be very strong.
2. The first 3 courses take a framework-independent approach. This allows you to easily use any framework.
3. The course contains practical tips on how to design evaluation metrics, how to split the dataset for training, and avoid variance and bias issues.
4. Of course, the best part is the case study, where you can find an opportunity to verify that you have understood how to successfully execute a deep learning project.
5. All jobs are provided as jupyter notebooks and run on Coursera servers, so there is no need to worry about the underlying framework.
6. After 3 wonderful lessons, there are still 2 lessons to look forward to.
7. There is a weekly talk called "Deep Learning Heroes". This is a great source for the history and inspiration behind deep learning.
Limitations:
1. For some brand new deep learning or machine learning project, it can be very challenging to build each part from scratch and understand the math behind it.
2. Since the course follows a bottom-up approach, even after the third lesson, you will still have a hard time building solutions using DL in your domain.
3. Since the basic framework is provided, you will no longer learn how to manage your own workstations. And if you're going to be using a lot of technology, it's important to have the ability to configure machines in the cloud or in your home/office.
4. The course provides many sample codes to make it easier to complete the homework. But to really master a skill, it is very important to reproduce the entire assignment without using example code. This may not be a limitation of the course, but it is very important if you want to get the most out of it.
cost:
All course content is provided free of charge. But I don't know if these assignments are also available for free. If you're taking the paid version of the course, then you end up spending around $367 or $55 per month.
【2】 fast.ai 的 Making neural nets uncool again
http://www.fast.ai/
Chinese subtitles https://edu.csdn.net/course/detail/5192
Recommended reason: This course is different from many traditional teaching methods. It starts from the application and gradually goes deeper, so that you can first understand the fun of "driving", and then you can learn the principle of the car and even repair it with great interest. It's been suggested to get started with this, have fun, and move on to a more basic course.
Fast.ai
I found this course in an article by kdddnudgets. It was the first time I heard about Jeremy Howard, so I searched for him on Wikipedia and was very impressed. So I bought this MOOC taught by Jeremy and Rachel Thomas. After watching the first class, I was deeply impressed by their teaching style.
You'll learn in a few hours how to build a state-of-the-art image classifier for less than a few dollars.
Jeremy Howard：
https://medium.com/@jeremyphoward
https://en.wikipedia.org/wiki/Jeremy_Howard_%28entrepreneur%29
About the instructor:
Jeremy's background is very different from the usual professors, he is not a doctoral supervisor of any top university, and he has never worked for top companies like Google, Baidu, Microsoft, etc. Self-taught, kaggle competition master, entrepreneur, and CEO of Fast.ai, his only goal at the moment is to demystify deep learning. His unique background makes this course unique, as he teaches people of different backgrounds how to use deep learning without the need for large amounts of data or computing power.
About the course:
The course is divided into 2 parts, with one part completed every 7 weeks.
The first part of the course teaches how to use deep learning in computer vision and natural language processing (NLP).
The second part teaches cutting-edge research work such as generative networks, GANs, sequence models, how to read research papers, and practical tips on how to stay ahead in the field of deep learning. These techniques are developing at an astonishing rate.
The teaching style of this course is unique. Course authors are used to this unique approach.
We don't make you learn to play cricket (or any game) after learning about speed, momentum, analysis, etc. and finally come in at 18. Instead, we take you to the field first, holding a ball or baseball bat, and then learn the rest in action.
With this unique approach, you'll first learn how to deploy workstations on the cloud, install software, and quickly build solutions using deep learning. Every week Jeremy picks a new problem and he teaches you how to use techniques to improve the performance of your model. He teaches practical things like using pre-convolutional features, pseudo-labels and many very useful tricks. By the end of the first part, you will be able to use deep learning to build practical applications in your field of work.
The second part of the course introduces cutting-edge research, helping you read, understand, and implement a variety of research papers on generative models, image segmentation, and sequence-to-sequence models. You will learn to build some interesting projects like style transfer, low resolution image to high resolution image, GAN, image segmentation, translation and how to apply deep learning to structured data. The second most important part is building your own workstation. This is useful if you are passionate about deep learning and want to build various deep learning applications.
Highlights:
The ability to build best-in-class computer vision or NLP systems.
Understand and use the modern mainstream architectures that power deep learning applications.
Practical tips on how to quickly apply DL when you have limited data and computing power.
A large community to support you in learning and implementing your solution at various stages.
Easily use 3 popular DL frameworks: Keras, TensorFlow, PyTorch.
After the course, you will no longer have the headache of reading research papers and building new projects. Fast.ai's blog, as well as the entire community, will support you.
Limitations:
Since this course follows a top-down approach, you will rely heavily on a framework to gain some understanding of basic mathematics. If you are working on a job or planning to do more research in the field, it will be helpful to improve your ability to understand mathematics in DL.
Some institutions take a certificate very seriously, they see it as proof that you have completed the course. But I guess Jeremy thinks we're all mature kids, so no credential of any kind is offered. Jeremy and Rachel encourage blogging, building projects, discussing at conferences, and more to replace the traditional credential with prowess. I personally find this to be very useful.
cost:
MOOCs themselves have no associated fees. But to get started with these projects, you'll end up spending on AWS, and maybe you'll choose to provision the machines yourself, but that's expensive. Of course having a powerful workstation at home is very helpful[3] Stanford University's course CS231n --- Convolutional Neural Networks for Visual Recognition
http://cs231n.stanford.edu/
Reasons for recommendation: This course is taught by Fei-Fei Li, director of the Stanford Artificial Intelligence Laboratory. As the main initiator of IMAGE-NET, Li's laboratory has been active in the frontier research field of computer vision and has cultivated many young talents. It is recommended that students engaged in computer vision pay attention to this course.
[4] 2016 Montreal Deep Learning Summer School
https://www.youtube.com/watch?list=PL5bqIc6XopCbb-FvnHmD1neVlQKwGzQyR&v=xK-bzjIQkmM
Recommended reason: Take a look at the guest lineup. Professor Yoshua Bengio lectures on recurrent neural networks, Professor Surya Ganguli lectures on theoretical neuroscience and deep learning theory, Professor Sumit Chopra lectures on reasoning summit and attention, Jeff Dean lectures on large-scale machine learning with TensorFlow, and Ruslan Salakhutdinov lectures on learning deep generation model, Ryan Olson on GPU programming for deep learning, and more.
[5] Stanford University's course CS 20SI: Tensorflow for Deep Learning Research
http://web.stanford.edu/class/cs20si/index.html
Recommended reason: Tensorflow is dominating the world. This course will take you into the deep learning world with Tensorflow. Whether it is scientific research or application, it is a good introductory material.
【6】Geoffrey Hinton Neural Networks for Machine Learning | Coursera
https://www.coursera.org/learn/neural-networks
Recommended reason: Yes, the instructor is to bring the deep neural network to the brilliant Mr. Hinton by himself. The course is not small, but I believe that the rewards will be not small if you stick to it.
[7] Stanford University's course CS224d: Deep Learning for Natural Language Processing
http://cs224d.stanford.edu/
Recommended reason: The deep learning course for NLP, starting from the basic neural network, to the probability model, to the word2vec, and finally to the application of the RNN model in the NLP field, is a required course in the NLP direction.
[8] Yann Lecun Deep Learning Open Course
https://www.college-de-france.fr/site/en-yann-lecun/course-2016-04-15-11h00.htm
Recommended reason: As the head of Facebook AI Labs (FAIR) and the inventor of CNN, Yann Lecun is at the forefront of machine learning research in the industry. Through this course, you can learn about the latest progress in deep learning research in recent years. This series serves as an advanced course for exploring deep learning.
[9] Machine Learning for Undergraduates, a machine learning course for UBC undergraduates
https://www.youtube.com/playlist?list=PLE6Wd9FR--Ecf_5nCbnSQMHqORpiChfJf
Recommended reason: From the name of the course, it can be seen that this is a basic-oriented course, and it is an introductory required course for understanding the basics of machine learning. Mathematics is the focus of this course. Teacher Nando de Freitas gave a good explanation of basic mathematical principles and introduced more advanced mathematical concepts.
------------------------------------------------------------------------------------
We have analyzed the courses of Andrew Ng, Udacity and Fast.ai for you
introduction
For the past 2 years, I have been actively focusing on the field of deep learning. My interest in deep learning started in early 2015, when Google just open sourced Tensorflow. I quickly tried a few routines based on Tensorflow's documentation, and the feeling at the time was that deep learning wasn't easy. Part of the reason is because the deep learning framework is very new and requires better hardware support and patience to explore.
Fast forward to 2017 and I've spent hundreds of hours on deep learning projects. And because of software (ease of use, eg Keras, PyTorch), hardware (for people like me working in India, GPUs have grown into commercial products, although still not cheap), data availability, good books, and the availability of MOOCs. As development progresses, this technology is becoming more and more accessible.
After completing the three most popular MOOCs in Deep Learning, Fast.ai, deeplearning.ai/Coursera (not fully released) and Udacity, I believe in writing this blog to tell you what you can expect yourself to learn from these 3 courses What, it must be useful for future deep learning enthusiasts.
Fast.ai：http://www.fast.ai/
deeplearning.ai /Coursera：https://www.deeplearning.ai/
Udacity：https://in.udacity.com/
In this article, I will introduce each course from 5 aspects that will help you decide.
About the instructors : Each course is taught by people with different backgrounds. I believe these experiences have a big impact on teaching style, so we'll take a look at the backgrounds of the course instructors.
About the course : A high-level overview of the course.
Highlights : The best part of the course.
Limitations : I'm very harsh on this term. Because I know that all of these courses put a huge and sincere effort into making the content easier to learn. I would like to interpret this part as, what we missed in the course. Some limitations may be due to the design of the course.
Fees : Fees incurred to attend the course.
Siraj Raval Deep Learning Nanodegree on Udacity
I am a fan of Udacity. They have very good courses on many topics. So when I read the announcement of the deep learning course early this year, I was very happy and signed up for the first batch.
About the author: Vishnu Subramanian, lifelong learner, keen on deep learning, distributed computing. Currently actively looking for AI/deep learning opportunities.
If you wanted to learn deep learning, which course would you choose?