foreword
Every year around the New Year's Day, there will be a small climax of Python's part-time orders. During this time, the demand for reptiles in various industries will skyrocket. Many friends in the circle are not idle during the New Year's Day holiday, and those who earn tens of thousands of dollars in two days are not there. few.
So recently, many friends have asked me about technical realization + part-time orders. I summed up the questions and found that most of them have the same confusion.
- Technology 0 foundation, want to use Python to make money, but don't know what technology to learn?
- Not familiar with mainstream technology, less practical experience, want to take private work, worry about not being able to handle the project
- I have never taken over private work, I have no experience and worry about stepping on the pit
How to take up Python private life?
When I first learned Python, a friend introduced me to take orders for private work. I still remember that I was crawling data for a company, and I earned 5.5K for that order. Since then, I have gradually become proficient, and I have successively received a lot of private work for data collection and processing in my spare time . On average, I can earn about 20,000 yuan per month by doing private work part-time .
Python technology receives more orders and earns more money, and it is generally reptiles. It mainly crawls the data of websites, small programs or APPs, analyzes and processes the data, or directly provides crawler programs and technical support to customers. key technology
Reptiles, as the most used technology for private use, are a must-have magic skill for part-time jobs. But many people say that reptiles are a bit complicated, and they haven't mastered it for a long time. In fact, as long as you master the correct implementation ideas , reptiles can learn quickly.
First of all, let's understand how the crawler works. The crawler usually consists of four steps: target information website , page crawling , page analysis , and data storage . The detailed process of crawling website resources is as follows:
- Import the corresponding library for request and web page parsing
- Then request the web page to get the source code
- Initialize the Soup object
- Open the landing page with a browser
- Locate the location of the required resources
- Then analyze the source code at that location
- Find tags and attributes for targeting
- Finally, write the parsing code to get the desired resource
Common technical issues
When we are familiar with the principles and processes, it is easy to implement a crawler, and the data of general websites can be easily crawled.
Of course, this does not mean that this is enough. Mastering the basic crawlers, you can indeed obtain resources at will in small undefended websites, but the really valuable resources are often in large sites with perfect anti-crawler measures.
At this time, it is the key link of Python crawler learning - website anti-crawling strategy and its countermeasures . Here are some common mainstream anti-climbing measures:
- The target detects that the crawler has blocked the IP
- The target returned encrypted data
- Target returned dirty data, unrecognizable
- Target website must be logged in to access
- Javascript dynamic rendering, crawler can not read
- The target website has a verification code and cannot be accessed
- Ajax asynchronous transmission, crawler grabs empty information
- Image camouflage and obfuscation + CSS offset + SVG mapping
If these problems cannot be solved, it is impossible to fully grasp the Python crawler technology, especially various anti-crawler measures, which have become the biggest obstacle to our data crawling.
Part-time order platform
Yuanjisong: https://www.yuanjisong.com/Outsourcing
Master: http://www.waibaodashi.com/Renren
Development: http://www.rrkf.com/Code
Easy: https://www.mayigeek .com
Open Source Crowdsourcing: https://zb.oschina.net/ Zhichou
: http://zhichou.com/
Development State: https://www.kaifabang.com/
Code Market: https://codemart.com /
Zhu Bajie: https://luoyang.zbj.com/Program
Ape Inn: http://www.proginn.com
…
No foundation, no experience, how to learn technology to make money?
Learning Python well is good, whether it is employment or a side business to make money, but you still need a learning plan to get started with Python. Here I will share with you a complete set of Python learning materials, a little help for friends who want to use Python to open up a side business to make money, but lack part-time experience and are not familiar with the mainstream technologies of major manufacturers!
Python learning roadmap
Just started learning python, if you don't even plan the complete learning steps, it is basically impossible to learn python.
Python development tools
As the saying goes: If a worker wants to do a good job, he must first sharpen his tool, and the learning of Python is the same.
Python learning books
Python Getting Started Video
If you have a certain self-learning ability, it is relatively easy to get started with Python through tutorials. Here are some free introductory video tutorials.
Python practical case
To do more with less study, proper study method is the key to success. The best way to learn programming is to practice, practice, practice, there are no shortcuts.
This full version of Python's full set of learning materials has been uploaded to the CSDN official. If you need it, friends can scan the CSDN official certification QR code below on WeChat [free access] .
