Let’s talk about the gossip, September has arrived, and the next few months will not only be the traditional peak season for salary increases and job changes, but also the climax period for Python technology to take on private jobs! The demand for [Python crawlers] in various industries may surge in a short period of time.
It is estimated that friends in the circle will not be idle and will take advantage of the peak season to make a fortune.
Therefore, I have received a lot of messages about [Technology Advancement & Technology Realization] recently. After answering everyone’s confusion one by one, it can be roughly summarized into the following situations:
- I want to start a second career, but my skills are average and I can’t handle difficult projects.
- I don’t have much practical experience and I don’t know how to start with many difficult tasks.
- I want to learn technology to increase my income, but I don’t know how to learn it systematically
At this point in time, there are many orders, great demand, easy remuneration, and stable cooperation. As long as you have enough skills, it is relatively easy to make some extra money doing private work.
Usually, for a project that can be completed within 4 hours, the reward will not be less than 1.5K, and those with sufficient energy can do it frequently.
After asking many friends in the circle, everyone takes on private work in their spare time, and many people’s income from private work has exceeded that of their main business.
Of course, although the demand is high and the rewards are high, profitable private projects require overcoming various extremely complex [anti-climbing measures] to be completed, and one must master sufficient technology and have sufficient practical experience. If you are not technically proficient, you will definitely not be able to reverse crawl, and you will not be able to obtain valuable resources, let alone receive high-value orders.
Efficient method for rapid advancement
Going back to the three major questions mentioned above, in order to help friends who want to use python technology to start a side business to make money, but are not familiar with anti-crawling technology, have no practical experience, and do not know how to learn
For friends who have changed careers or are new to IT technology, the most important things are two things:
- How to stay focused and be able to persist
- Find ways to learn quickly
The first thing is actually not difficult. There are usually two reasons for staying focused and persisting: financial income or great hobbies . Looking at my growth experience, this is actually the reason why I persist. At the beginning of my studies, it was entirely out of interest and the pressure to change careers. At the 3-month stage, having some small income was the main reason for me to continue moving forward.
The second thing is to find ways to learn quickly. I can say that my progress has been very rapid and solid. I deeply understand the principle of "it is difficult to build a high platform on floating sand", so in the process of learning, I wrote articles to consolidate what I learned, and I only focused on one field (crawlers).
Python can develop in many directions, such as WEB, crawlers, data analysis and deep learning. I know that if I want to take root, I must focus on one area. From the initial network request, web page parsing to data storage, crawler deployment and scheduling, to asynchronous, multi-threaded and multi-process, deduplication and increment, and even anti-crawler research, they all belong to the field of crawlers.
In order to stay focused, I changed my work and rest schedule from the original "wake up at 8 or 9 a.m. and go to bed at 1 or 2 a.m." to "wake up at 6 or 7 a.m. and go to bed at 11 p.m." The same is true on weekends. Going to bed early and getting up early gives me enough energy to keep working and studying.
After choosing a certain field and maintaining it for enough time, factors that can make people progress quickly include: reading source code and building wheels. Reading the source code allows us to understand the design of the program from a higher perspective and learn many cool operations that are usually difficult to see. Making wheels tests our ability to convert knowledge into code.
The projects that I have read/studied source code include:
- Scrappy
- Scrapy
- Celery
- Gerapy
- SpiderKeeper
- SpiderWeb
- WebSockets
- WebsocketClient
- Parcel
- apscheduler
- Job_Spider, a scheduled task library written in Rust language
The wheels I have built include:
- aiowebsocket, an asynchronous WebSocekt client written in Python
- Gerapy 2.0 backend (not released), a crawler project deployment and scheduling platform written using the Tornado framework
- django_cron_scheduler, a crawler project deployment and scheduling platform written during work
Here is a suggestion. When reading source code and checking information, we must use search engines. Articles listed by search engines are not necessarily accurate. Some articles are even missing the beginning and the end, so it is best to directly read the documents or find some articles written by well-known authors to learn.
About Python technical reserves
Learning Python well is good whether you are getting a job or doing a side job to make money, but you still need to have a learning plan to learn Python. Finally, we share a complete set of Python learning materials to give some help to those who want to learn Python!
1. Python learning outline
The technical points in all directions of Python are organized to form a summary of knowledge points in various fields. Its usefulness is that you can find corresponding learning resources according to the above knowledge points to ensure that you learn more comprehensively.
2. Essential development tools for Python
3. Introductory learning video
4. Practical cases
Optical theory is useless. You must learn to follow along and practice it in order to apply what you have learned to practice. At this time, you can learn from some practical cases.
5. Python side job part-time and full-time routes
6. Internet company interview questions
We must learn Python to find a high-paying job. The following interview questions are the latest interview materials from first-tier Internet companies such as Alibaba, Tencent, Byte, etc., and Alibaba bosses have given authoritative answers. After finishing this set I believe everyone can find a satisfactory job based on the interview information.
This complete version of Python learning materials has been uploaded to CSDN. If you need it, you can scan the official QR code of csdn below or click on the WeChat card at the bottom of the homepage and article to get the method. [Guaranteed 100% free]
