Deep learning involves a lot of content, and the vast amount of various materials is even more dazzling. It is often the case that you don't know the whole thing when you glance at one. Often after reading the knowledge points, I understand them, but what is the purpose of learning them? Where is it used? What scene? Why this idea? What is the purpose? Wait, you will have a series of questions. Every knowledge point is just scattered beads, and you can't connect them logically with cause and effect. So before writing a series of articles on deep learning, I think it is very necessary to show the entire framework of deep learning. I just stumbled and took a lot of detours, and now I know a little bit about it, and now I’m marking it down.
Your best learning route is: start with three routes: DNN, CNN, and RNN. Note that the order cannot be reversed, because DNN is the basis for you to learn CNN, and CNN is the basis for you to get started with RNN. If the foundation is not strong, you will become more and more crazy later. Of course, before you learn DNN, it is assumed that you have a certain computer foundation, underlying memory, operating system, language, data structure, database, etc. It is assumed that you already know everything you should know.
Then you can now start building your deep learning knowledge framework in the order of DNN, CNN, and RNN. But whether you are learning DNN, CNN or RNN, in general, you must master these three aspects: first, the principles, including network structure, data propagation process, loss function, optimization algorithm, training process, optimization techniques, etc.; second, In terms of mathematics, you need to know what kind of mathematical calculation process the data propagation process is, you need to know the specific expression of the loss function or the selection, you need to know the mathematical logic behind the optimization algorithm, and you need to know the mathematics of the training process. The process, etc., means that you need to be clear about the mathematical process of each step, so that you can know where the problem lies and you can take targeted measures; third, in terms of code implementation, all the above-mentioned links must be implemented with code, because any of your Ideas and behaviors must be realized with the help of code, so you must have basic coding skills.
Only if you have mastered the above three aspects can you show that you have learned this knowledge point. My personal experience is:
Mathematics is the easiest. If you are a graduate student, your mathematical reserves are basically sufficient, because after all, deep learning does not have a strict mathematical process like machine learning. Deep learning involves performing simple operations on huge amounts of data, at most matrix multiplication.
Secondly, in terms of principles, the principles of some knowledge points will involve complex and rigorous mathematical processes, but you don’t need to pay attention to the mathematical processes in detail. You only need to use its conclusions, which is to use its conclusions to confirm your understanding of the principles. understand. You think we don’t have to invent Newton’s laws, but we can use Newton’s laws to solve real-life problems. That is, even if you don't know how Newton's laws are derived, as long as you know what this formula does and what purpose it can achieve, you can use this formula freely. At the same time, you need to be flexible enough to understand it, that is, understand it from multiple perspectives, because there are too many new terms in the deep learning principle part, but as long as you understand the mathematical process behind it, you can understand it flexibly. For example, it is the multiplication of a matrix and a matrix. In deep learning, some people will say that dimensionality reduction, spatial transformation, feature transformation, high dimensionality, low dimensionality, mapping, etc. have invented different professional vocabulary in different scenarios, but its essence all the same. For another example, multiplying a number by a coefficient is a simple calculation. In different scenarios, you will be told whether it is scaling, whether it is adjusting the data flow, etc., or whether it is memorizing something. , refer to something, or even say it is this door or that door, in fact, it is just multiplied by a coefficient, so you have to bypass the pretense and see the essence.
Finally, in terms of code, this is where you learn. No matter what ideas or actions you have, you have to use a computer to realize or show your ideas to everyone, so you must have basic coding capabilities. And this ability is accumulated bit by bit, and learning it alone is not very effective, because if you don’t have the purpose and background of learning it, you can only forget it after learning it. So my recommendation is to learn the code while learning the principles, so that you know where this function is used, what functions it helps me achieve, and what tasks it completes. You can use it when you encounter the same task in the future. It is a tool, and it is a very complex tool. You need to understand it and become proficient in it over time.
Okay, let’s start from the order of DNN, CNN, and RNN, and refine what we need to learn for each network:
The first stage: DNN stage.
There are only so many lines of knowledge that I can think of for the time being. I originally planned to continue writing CNN and RNN, but there are too many and time is limited. CNN and RNN are similar to the four-dimensional map above. I will add them when I start writing.
Later, I will start updating the article along the lines of DNN, CNN, and RNN.