How to systematically and scientifically teach yourself programming knowledge?

Preface

Do you feel like you graduated from school with little more than toy programs? After entering the workplace, even if you have little experience, you can go through the following extracurricular exercises (complaints from friends: school courses always start from theory, and homework projects do not have any practical effect, it is better to start from the needs at work)

suggestion:

• Don't buy books randomly, don't chase new technologies and new terms. The basic things have been accumulated over a long time and will be common for at least the next 10 years.
• Only by looking back at history and looking at the development of technology on the historical timeline can you understand what tomorrow will be like.
• You must do it. No matter how simple the example is, it is recommended to at least type it by yourself to see if you understand the details inside.
• You must learn to think and think about why you should do this and not that. We also need to think by drawing inferences from one example to another.

Note : You may be wondering why the following things are very Unix/Linux. This is because I think programming under Windows may have no future in the future. The reasons are as follows:

• Today's user interfaces are almost dominated by two things, 1) the web, and 2) mobile devices iOS or Android. The graphical interface of Windows is no longer popular.
• More and more enterprises are using low-cost, high-performance Linux and various open source technologies to build their systems. The cost of Windows is too high.
• Microsoft's stuff becomes too fast, very short-lasting, and they are completely toying with programmers. For details, see "History of Windows Programming Revolution"

Therefore, I personally think that the future trend is that the front end will be Web + mobile, and the back end will be Linux + open source. There is basically nothing to do with Windows on the development side.

Introduction to enlightenment

1. Learn a scripting language, such as Python/Ruby

It can free you from the fear of low-level languages, and scripting languages ​​can allow you to quickly develop small programs that can be used. Practice projects:

• Process text files, or csv (keywords python csv, python open, python sys) Read a local file and process it line by line (such as word count, or process log)
• Traverse the local file system (sys, os, path), for example, write a program to count the sizes of all files in a directory, sort them by various conditions, and save the results
• Deal with the database (python sqlite) and write a small script to count the number of entries in the database
• Learn to debug using simple and crude methods such as various prints
• Academic Google (phrase, domain, use reader to follow tech blogs)

Why should we learn scripting languages? Because they are so convenient. Many times we need to write some gadgets or scripts to help us solve problems. You will find that formal programming languages ​​are too difficult to use.

2. Be familiar with a programmer’s editor (not IDE) and some basic tools

• Vim/Emacs/Notepad++, learn how to configure code completion, appearance, external commands, etc.
• Source Insight (or ctag)

Using these things is not for cool, but these editors will be faster and more efficient in viewing and modifying code/configuration articles/logs.

3. Familiar with Unix/Linux Shell and common command lines

• If you use Windows, at least learn to use Linux in a virtual machine. vmware player is free. Install Ubuntu.
• Be sure to use less graphical interfaces.
• Learn to use man to view help
• File system structure and basic operations ls/chmod/chown/rm/find/ln/cat/mount/mkdir/tar/gzip…
• Learn to use some text manipulation commands sed/awk/grep/tail/less/more…
• Learn to use some management commands ps/top/lsof/netstat/kill/tcpdump/iptables/dd…
• Understand the various configuration articles in the /etc directory, learn to view the system logs under /var/log, and the system operating information under /proc
• Learn about regular expressions and use them to find files.

For programmers, Unix/Linux is much simpler than Windows. (See my CSDN blog post "In fact, Unix is ​​very simple" four years ago) Once you learn to use Unix/Linux, you will find that the graphical interface is too difficult to use at certain times, which greatly reduces work efficiency.

4. Learn Web basics (HTML/CSS/JS) + server-side technology (LAMP)

The future is bound to be a Web world, and the best website to learn the basics of WEB is W3School.

• Learn basic HTML syntax
• Learn how CSS selects HTML elements and applies some basic styles (keyword: box model)
• Learn to use Firefox + Firebug or chrome to view the structure of web pages that you think are cool, and modify them dynamically.
• Learn to use Javascript to manipulate HTML elements. Understanding DOM and Dynamic Web Pages (Dynamic HTML: The Definitive Reference, 3rd Edition - O'Reilly Media) There are free chapters online, which are enough. Or see DOM.
• Learn to debug Javascript code with Firefox + Firebug or chrome (set breakpoints, view variables, performance, console, etc.)
• Configure Apache or Nginx on one machine
• Learn PHP, let back-end PHP and front-end HTML interact with data, and form a preliminary understanding of the server's corresponding browser requests. Implement a form submission and reverse display function.
• Connect PHP to the local or remote database MySQL (it is enough to learn and use MySQL and SQL now)
• Don’t think it takes more than one semester to complete a network programming course from a prestigious school (for example: http://www.stanford.edu/~ouster/cgi-bin/cs142-fall10/index.php). College students take one semester full-time. Choose 3-5 courses and you will definitely be able to keep up in your spare time
• Learn a javascript library (such as jQuery or ExtJS) + Ajax (asynchronously reading a server-side image or database content) + JSON data format.
• HTTP: The Definitive Guide After reading the first 4 chapters, you will understand what happens when you use the browser every day (proxy, gateway, browsers)
• Make a small website (for example: a small message board, support user login, Cookie/Session, add, delete, modify, check, upload image attachments, display in pages)
• Buy a domain name, rent a space, and build your own website.

Advanced and deepening

1. C language and operating system calls

• Learn C language again, understand pointers and memory models, and use C language to implement various classic algorithms and data structures. Recommend "The Art of Computer Programming", "Introduction to Algorithms" and "Programming Pearls".

• Learn (Free MIT Course) Introduction to Computer Science and Programming

• Learn (MIT free course) C language memory management

• Learn Unix/Linux system calls (Unix advanced environment programming) and understand things at the system level.

• • Use this system knowledge to operate the file system, user (implement a small program that can copy a directory tree)
  • Use fork/wait/waitpid to write a multi-process program, and use pthread to write a multi-thread program with synchronization or mutual exclusion. Multi-process multi-process ticket purchasing program.
  • Use signal/kill/raise/alarm/pause/sigprocmask to implement a program for semaphore communication between multiple processes.
  • Learn to use gcc and gdb to program and debug programs (see my "Debugging Programs with gdb")
  • Learn to use makefiles to compile programs. (See my "Write Makefile with Me")
  • IPC and Socket things can be put into advanced practice.

• Learn Windows SDK programming (Windows programming, MFC programming)

• • Write a window and understand WinMain/WinProcedure and the message mechanism of Windows.
  • Write some programs to operate resource files or various graphic controls in Windows SDK, as well as drawing programming.
  • Learn how to use MSDN to view related SDK functions, various WM_ messages, and some routines.
  • There are many routines in this book. Please do not copy them in practice. Try to write your own routine.
  • You don’t need to be proficient in these things too much, because GUI is being replaced by the Web. The main thing is to understand the programming of Windows graphical interface. @virushuo said: "I think GUI is indeed not that popular, but it is important to fully understand how GUI works. Including mobile device development, it is still difficult without basic knowledge. In other words, mobile device development must understand GUI work, or in Learn on win, or learn on mac/iOS."

2. Learn Java

• To learn Java, I mainly read the classic Core Java "Java Core Technology Programming" and "Java Programming Thoughts" (there are two volumes, I only linked to the first volume, which is enough, because you only need to understand the graphical interface of Java)
• Learn JDK and learn to read Java API Doc Java Platform SE 6
• Understand the differences in compilation and execution between Java, a virtual machine language, and C and Python languages. Think about "cross-platform" technology from C, Java, and Python.
• Learn to use the IDE Eclipse and use Eclipse to compile, debug and develop Java programs.
• Build a Tomcat website and try JSP/Servlet/JDBC/MySQL Web development. Try to implement the small PHP project mentioned earlier using JSP and Servlet.

3. Web security and architecture

• There are many tutorials on the Internet for learning HTML5. KuKe has introduced many of them before, so I won’t list them here.

• Learn the security issues of web development (refer to the attack on Sina Weibo and this article on Ruby)

• Learn the rewrite mechanism of HTTP Server, the reverse proxy mechanism of Nginx, fast-cgi (such as: PHP-FPM)

• Learn the static page caching technology of the Web.

• Learn the asynchronous workflow processing, data cache, data partitioning, load balancing, and horizontal expansion architecture of the Web.

• Practical tasks:

• • Use HTML5 canvas to create some web animations.
  • Try to conduct SQL injection, JS injection, and XSS attacks in the web application developed earlier.
  • Change the web application developed earlier into a website constructed on Nginx + PHP-FPM + static page caching

4. Learn relational databases

• You can install MSSQLServer or MySQL to learn databases.
• Learn the paradigms of database design in textbooks, 1NF, 2NF, 3NF,...
• Learn database storage, triggers, views, index building, cursors, etc.
• Learn SQL statements and understand various concepts of table joins (see "Illustration of SQL Join")
• Learn how to optimize database queries (see "MySQL Optimization")
• Practical task : Design a forum database that meets at least 3NF, and use SQL statements to query the latest articles of this week and this month, the articles with the most comments, and the most active users.

5. Some development tools

• Learn to use SVN or Git to manage program versions.
• Learn to use JUnit to unit test Java.
• Learn the coding standard or coding guideline of C language and Java language. (I wrote a very simple article about C language - "Programming Cultivation" N years ago. You can check a lot of such things online).
• Recommended reading: "Code Encyclopedia", "Refactoring" and "How to Clean Code"

Advanced in-depth

1. C++/Java and object-oriented

I personally think that learning C++ and Java well is just a piece of cake. But the learning curve of C++ is quite steep. However, I think C++ is the language that needs to be learned most. Please refer to the two interesting articles "C++ Learning Confidence Chart" and "Learn C++ Well in 21 Days"

• Learn (MIT Free Course) C++ Object-Oriented Programming

• Read the books recommended in my "How to Learn C++ Well" at least twice (if your understanding of C++ can go as deep as my "C++ Virtual Function Table Analysis" or "C++ Object Memory Storage" (Part 1) ) (Part 2)", or "The Trap of C/C++ Returning Internal Static Members" would be very good)

• Then reflect on why C++ does this, but Java doesn’t? You must learn to compare the differences between C++ and Java. For example, initialization, garbage collection, interfaces, exceptions, virtual functions, etc. in Java.

• Practical tasks:

• • Use C++ to implement a BigInt that supports 128-bit integer addition, subtraction, multiplication and division operations.
  • Use C++ to encapsulate the capacity of a data structure, such as a hash table.
  • Use C++ to encapsulate and implement a smart pointer (be sure to use templates).

• "Design Patterns" must be read once, at least twice, and think about the application scenarios of these 23 patterns. Mainly two points: 1) Favor composition instead of inheritance, 2) Favor interfaces instead of implementation. (Also recommend "Introduction to Design Patterns in a Simple Way")

• Practical tasks:

• • Implement a memory pool using factory pattern.
  • Use Strategy Pattern to create a class that can align text files to the left, right and center.
  • Use command mode to implement a command line calculator and support undo and redo.
  • Use modification mode to implement a hotel room price pricing strategy - peak season, service, VIP, tour group, and other factors that affect the price.

• Learn the usage of STL and its design concepts - containers, algorithms, iterators, functors. If possible, please read its source code.

• Practical tasks: Try out various skills in object-oriented, STL, design patterns, and WindowsSDK graphics programming

• • Make a game of Snake or Tetris. Supports different levels and difficulties.
  • As a file browser, you can browse files in a directory and perform different operations on different files. Text files can be opened and edited, executable files can be executed, mp3 or avi files can be played, and picture files can display pictures.

• Learn the design of some C++ class libraries, such as: MFC (check out Mr. Hou Jie’s "In-depth Introduction to MFC"), Boost, ACE, CPPUnit, STL (STL may be too difficult, but if you can understand the design patterns and The design would be great. It would be great if you could go deeper into the "Copy-on-Write Technology of STL String Classes" written by me. ACE requires strong system knowledge. See "Enhancing Understanding of the System" below. )

• Java is a true object-oriented language. There are too many design patterns in Java. It is also the best language to learn object-oriented design patterns (see Design Patterns in Java).

• Recommended reading "Effective Java" and "Java Answers"

• Learn Java frameworks. There are many Java frameworks, such as Spring, Hibernate, Struts, etc. The main purpose is to learn Java design, such as IoC, etc.

• There are also many Java technologies, focusing on learning J2EE architecture and JMS, RMI, and other messaging and remote calling technologies.

• Learn to use Java to do Web Service (official tutorial here)

• Practical task: Try to build a remote calling program for a networked Web Service under the Spring or Hibernate framework, and can pass messages through JMS in the two services.

Neither C++ nor Java can be learned in a short period of time. C++ is very deep and Java is very broad. I suggest you choose one of the two. My personal learning experience is:

• Study C++ in depth (I have studied C/C++ for more than ten years)
• Learn various design patterns in Java.

2. Strengthen system understanding

It is important to read the following books:

• "The Art of Unix Programming" understands the design and development philosophy, ideological and cultural systems, principles and experiences in the field of Unix systems. You will definitely have a feeling of enlightenment.

• "Unix Network Programming Volume 1, Sockets" is a book that will help you understand network programming after reading it. It is important to note the differences between TCP, UDP, and multiplexed system calls select/poll/epoll.

• "TCP/IP Detailed Explanation Volume 1: Protocol" - This is a book that you can become a network hacker after reading it. Understand the operating principles of Ethernet, understand the TCP/IP protocol, operating principles and how to tune TCP.

• Practical tasks:

• • Understand what blocking (synchronous IO), non-blocking (asynchronous IO), and multiplexing (select, poll, epoll) IO technology are.
  • Write an online chat program with a chat server and multiple chat clients (the server uses UDP to Multicast or Broadcast some or all of the chat clients).
  • Write a simple HTTP server.

• "Unix Network Programming Volume 2, Inter-Process Communication" Semaphores, pipes, shared memory, messages and other IPC... These technologies seem a bit old-fashioned, but they are still worth knowing.

• Practical tasks:

• • Mainly practice various IPC program communication methods.
  • Try to write a pipeline program, and the parent and child processes exchange data through the pipeline.
  • Try to write a shared memory program. Two processes exchange a C structure array through shared memory.

• Study the book "Windows Core Programming". Master CreateProcess, Windows threads, thread scheduling, thread synchronization (Event, semaphore, mutex), asynchronous I/O, memory management, and DLL.

• **Practical task:** Use CreateProcess to start a Notepad or IE and monitor the running of the program. Implement the simple HTTP service I wrote earlier using a thread pool. Write a DLL hook program to monitor the closing event of a specified window, or record the keystrokes of a certain window.

• With the basics of multi-threading, multi-process communication, TCP/IP, sockets, C++ and design patterns, you can study ACE. Use ACE to rewrite the above chat program and HTTP server (with thread pool)

• Practical tasks: pass all the above knowledge and try

• • Writing a server to transfer large files to the client requires using more than 80% of the 100M bandwidth. (Note that disk I/O and network I/O may be problematic. Think about how to solve them. In addition, please pay attention to the maximum unit MTU of network transmission)
  • Understand the working principle of BT downloading and simulate the principle of BT downloading in a multi-process manner.

3. System architecture

• Load balancing. HASH style, purely dynamic. (You can go to Google Scholar to search for some articles about load balancing and read them)
• Multi-layer distributed system – client service node layer, computing node layer, data cache layer, and data layer. J2EE is a classic multi-layer structure.
• CDN system – nearby access, content marginalized.
• P2P system, study the algorithms of BT and eDonkey. For example: DHT algorithm.
• Server backup, dual-machine backup system (Live-Standby and Live-Live system), how do two machines monitor each other through heartbeats? Cluster master node backup.
• Virtualization technology, using this technology, the operating system can be switched or reconfigured and deployed as an application.
• Learn Thrift, a binary high-performance communication middleware that supports data (object) serialization and multiple types of RPC services.
• Learn Hadoop. The core designs in the Hadoop framework are: MapReduce and HDFS. The idea of ​​MapReduce was mentioned in a paper by Google and became widely circulated. In a simple sentence, MapReduce is "the decomposition of tasks and the summary of results." HDFS is the abbreviation of Hadoop Distributed File System, which provides underlying support for distributed computing and storage.
• Understand the NoSQL database (some people say it may be an over-hyped technology), but because ultra-large-scale and highly concurrency purely dynamic websites are becoming mainstream, and SNS websites have rigid requirements such as real-time in the data access process, this makes At present, NoSQL databases are gradually becoming the focus of people's attention, and are likely to replace relational databases and become the mainstream data storage model in the future. There are currently many NoSQL databases, most of which are open source. Among the more well-known ones are: MemcacheDB, Redis, Tokyo Cabinet (upgraded version to Kyoto Cabinet), Flare, MongoDB, CouchDB, Cassandra, Voldemort, etc.

Having written so much, looking back, I feel quite accomplished. I hope everyone will not be scared. I have been learning continuously for the past ten years. I am also learning today. Life is a process of continuous learning and leveling. However, there must be some omissions and errors, and I hope you will add and correct them .

1. Introduction to Python

The following content is the basic knowledge necessary for all application directions of Python. If you want to do crawlers, data analysis or artificial intelligence, you must first learn them. Anything high-end is built on a primitive foundation. By laying a good foundation, the road ahead will be more stable.All information is available for free at the end of the article！！！

Include:

Computer Basics

python basics

Python introductory video episode 600:

Watch zero-based learning videos. Watching videos is the fastest and most effective way to learn. It is easy to get started by following the teacher's ideas in the video, from basic to in-depth.

2. Python crawler

As a popular direction, crawlers are a good choice whether you use them part-time or as an auxiliary skill to improve work efficiency.

Through crawler technology, relevant content can be collected, analyzed and selected to get the information we really need.

This information collection, analysis and integration work can be applied to a very wide range of areas. Whether it is life services, travel, financial investment, product market demand of various manufacturing industries, etc., crawler technology can be used to obtain more accurate and effective information. use.

Python crawler video information

3. Data analysis

The "Digital Transformation of China's Economy: Talent and Employment" report released by Tsinghua University School of Economics and Management shows that the data analysis talent gap is expected to reach 2.3 million in 2025.

With such a huge talent gap, data analysis is like a vast blue ocean! Starting salary of 10K is really commonplace.

4. Database and ETL data warehouse

Enterprises need to regularly transfer cold data from the business database and store it in a warehouse dedicated to storing historical data. Each department can provide unified data services based on its own business characteristics. This warehouse is a data warehouse.

The traditional data warehouse integrated processing architecture is ETL. Using the capabilities of the ETL platform, E = extract data from the source database, L = clean the data (data that does not comply with the rules) and transform the table (perform different dimensions and granularity on the table according to business needs) degree, different business rules calculation and statistics), T=load the processed table to the data warehouse in increments, full quantities, and different times.

5. Machine Learning

Machine learning is to learn from a part of the computer data, and then predict and judge other data.

The core of machine learning is "using algorithms to parse data, learn from it, and then make decisions or predictions about new data." That is to say, the computer uses the data obtained to derive a certain model, and then uses this model to make predictions. This process is somewhat similar to the human learning process. For example, after a person acquires certain experience, he or she can predict new problems.

Machine learning materials:

6. Advanced Python

From basic syntax content to many in-depth advanced knowledge points and understanding of programming language design, after studying here, you will basically understand all the knowledge points from entry to advanced python.

At this point, you can basically meet the company's employment requirements. If you still don't know where to find interview materials and resume templates, I have compiled one here for you. It can really be said to be a systematic learning route for nannies and caregivers. .

But learning programming does not happen overnight, but requires long-term persistence and training. In organizing this learning route, I hope to make progress together with everyone, and I can also review some technical points myself. Whether you are a newbie in programming or an experienced programmer who needs to advance, I believe everyone can gain something from it.

It doesn’t happen overnight, but requires long-term persistence and training. In organizing this learning route, I hope to make progress together with everyone, and I can also review some technical points myself. Whether you are a newbie in programming or an experienced programmer who needs to advance, I believe everyone can gain something from it.

Data collection

This complete version of Python learning materials has been uploaded to CSDN official. If you need it, you can click on the CSDN official certification WeChat card below to get it for free ↓↓↓ [Guaranteed 100% free]