A simple requirement is to start a python script regularly. This requirement is very common. For example, start a program regularly to collect server status and write it to a file, which is convenient for post-operation and maintenance audits, and checks the peak hours of server usage to determine the company. The product is used by many people during this time period, or the logs of other programs are regularly cleared to free up space on the online server. This common architecture is a dump program that suspends the logs through the nginx file service, and then the program requests This kind of file is stored in the data server, and the log of the online server is not needed (the game log is usually relatively large, so the dump program also needs to be designed).
This chapter is mainly to realize the need to start python regularly. Of course, it is the same to start any other program regularly.
Python threading module
At the beginning, in order to save trouble, I directly used the threading module of python. There is a Timer module under the threading module, which can meet the needs of starting python programs at regular intervals. The usage is as follows:
It is worth mentioning that the timer needs to use the global timer . It is said that when it tries to run, it will release unused occupied resources.
The implementation method is very simple, that is, create a Timer() instance, pass in two parameters, which are the time interval (in seconds) and the timing task itself, to form a dead recursion (because there is no escape condition), and then call the Timer instance start() method.
Not recommended, although online blogs say that using the global timer will release useless resources, but there is no actual research. The program that runs on the server with this method usually breaks within a day. I start the program on Sunday and come to the company on Monday to see the corresponding python The program hangs.
APScheduler
APScheduler is a third-party framework used by Python to perform timing operations. As a framework, it has various concepts corresponding to it. There is no need to make it so complicated, and the learning cost is a bit high. Give up
Linux crontab
In the end, I switched to the crontab service of Linux. This service is mainly used to implement timing tasks. Its syntax is as follows:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR
#sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * command to be executed
- minute: Represents the minute within an hour, ranging from 0-59.
- hour: Represents the hour of the day, ranging from 0-23.
- mday: represents the day of the month, ranging from 1-31.
- month: Represents the month of the year, ranging from 1-12.
- wday: Represents the day of the week, ranging from 0-7 (0 and 7 are both Sundays).
- who: what identity to use to execute the command, when you use crontab -e, you don't need to add this field.
- command: The command to be executed.
crontab service status
sudo service crond start
#启动服务
sudo service crond stop
#关闭服务
sudo service crond restart
#重启服务
sudo service crond reload
#重新载入配置
sudo service crond status
#查看服务状态
View scheduled tasks
crontab
l
So far, the common crontab files tell you to use crontab-e to write the corresponding crontab configuration file. The syntax of the configuration content is as above, and the example is as follows:
But this side won’t do it like this. This way of writing is not suitable for real work. It’s just a toy. What I hope is full automation. Here, the automatic addition of crontab tasks is realized through shell scripts.
The shell script code is as follows:
This is the complete shell script I use. The command to automatically add crontab tasks is only one line, which is echo "* * * * * ${work_path}start.sh start >> ${work_path}logs/cron.log 2>&1" >>/var/spool/cron/root, this command will call the start.sh script every minute, and python is started in the start.sh script, a few pitfalls need to be noted, please use the absolute path in crontab, because crontab starts the program , the coordinate system corresponding to the relative path is actually different from that when you manually start the script. Using an absolute path saves trouble. Here, the output content of the star.sh script is redirected to the corresponding log file.
Why not start the python program directly through crontab? Instead, you have to go around one more layer and start it through a shell script. This is actually a pit. Unless you have a single python file, you usually start python in the form of a shell script instead of using crontab directly. This is also because The coordinate system of the relative path of the task started by crontab has changed. When multi-file python projects import files from each other, the coordinate system used is different from that used when crontab starts, which will cause crontab to fail to start the python project directly, so the trick is **, through the shell Script to start the python program. Before starting, enter the directory corresponding to the python project through the cd command, so that the coordinate system of the relative path at startup is changed to be consistent with that of python**
For details, you can take a look at my start.sh script, the code is as follows:
The key command to start the task through python is
First, you will enter the directory where you want to start the python project, and then start the corresponding py file through python. Here, you must also use the full path of the python interpreter, because there are multiple pythons in the online system, because the python program is a time-consuming program. , so I want it to run in the background, so I use the nohup and & keywords to put it in the background to run.
Digression:
The yum in the centos system depends on python. Specifically, in centos6, its yum depends on the python2.6 that exists in the system itself, but the development environment usually uses python2.7. At this time, it is best not to delete the python2.6 that comes with the system , if you delete it directly, it will cause yum to be unusable. At this time, you need to modify the python pointer in the yum corresponding file. The best way is to install python2.7 directly, and then create the corresponding soft link under /usr/bin to use.
summary
In fact, python programmers can't only know python at work, because although python is powerful, it also has its flaws, so what is easy to use and what is right to use, and python is a language, don't be limited by the language.
About Python Technical Reserve
It is good to learn Python whether it is employment or sideline business to make money, but to learn Python, you still need a study plan. Finally, everyone will share a full set of Python learning materials to help those who want to learn Python!
1. Learning routes in all directions of Python
The route of all directions in Python is to organize the commonly used technical points of Python to form a summary of knowledge points in various fields. Its usefulness lies in that you can find corresponding learning resources according to the above knowledge points to ensure that you learn more comprehensively.
2. Learning software
If a worker wants to do a good job, he must first sharpen his tools. The commonly used development software for learning Python is here, which saves you a lot of time.
3. Introductory learning video
When we watch videos and learn, we can’t just move our eyes and brain without using our hands. A more scientific learning method is to use them after understanding. At this time, the hands-on project is very suitable.
4. Practical cases
Optical theory is useless, you have to learn to follow along, and you have to do it yourself, so that you can apply what you have learned to practice. At this time, you can learn from some actual combat cases.
5. Interview materials
We must learn Python to find high-paying jobs. The following interview questions are the latest interview materials from first-line Internet companies such as Ali, Tencent, and Byte, and Ali bosses have given authoritative answers. After finishing this set The interview materials believe that everyone can find a satisfactory job.
Career Development
As a code wage earner, for the vast majority of programmers, there is still a long way to go to become an awesome real money-making programmer, and they can't slack off for a moment.
We cannot guess which state of the interview is more favored by the interviewer from the perspective of HR or the technical leader. But through the large amount of interview experience we have accumulated, you can more or less infer some of the necessary conditions to become a competitive programmer.
Gathering of bigwigs and rich information
At the beginning, I met a very, very senior senior in Byte. He came to Byte three years earlier than me, but his level was not very high for various reasons. I asked him at the time, since you are so dissatisfied with the status quo, why don't you think about leaving to find better opportunities?
He pondered for a moment and told me that although my stay here is not going well, the people I have come into contact with are all excellent. I have a problem, and I can discuss it with you. If I go out, if I encounter problems again, there may not even be anyone to discuss.
I just thought it made sense when I heard it at the time, but looking back now, I feel very profound. Three outlooks, structure, ability, those who can enter a large company, these three aspects are generally not too bad. Among other things, in terms of personal ability, I have traveled abroad for several months, and I have been fortunate enough to meet many colleagues from various famous overseas schools, and learn and communicate with them on artificial intelligence. This really gave me a deeper understanding of AI. know.
In addition to excellent colleagues, large companies often have rich internal documents and materials. At that time, I saw many excellent articles inside Byte, and there were also many excellent technical salons and sharing. Now that I think about it, I haven’t been there a few times in the past two years, and I haven’t read too many articles and materials. It’s a pity now that I think about it. Among other things, as far as the field of recommendation is concerned, the papers with good quality in recent years often come from large companies, especially large domestic companies, mainly Tencent, Huawei and Toutiao. In addition to the public papers, there are a lot of technology-related materials and documents within the company. These are really valuable and have no market, and are very precious.
All the information displayed in this article is free to share, just scan the CSDN official certification WeChat QR code below【保证100%免费】
Python Knowledge Manual
Linux Knowledge Manual
The crawler query manual
Moreover, these materials are not scanned versions, and the text inside can be copied directly, which is very convenient for us to learn:
Data Analysis Knowledge Manual:
Machine Learning Knowledge Handbook:
Handbook of Financial Quantitative Knowledge:
Job referral, learning exchange
We need a large number of front-end positions, python positions, Java positions, Android and iOS development positions, working location: Beijing Byte, welcome school recruiters to scan the QR code below and find me to recommend
If you feel that your learning efficiency is low and you lack correct guidance, you can join the CSDN official technology circle with rich resources and a strong learning atmosphere to learn and communicate together!
