1. Project document well in advance of filing
Every time you start a new job, before I was always the sake of convenience, Code, Data, documents are concentrated in one folder, it looks messed up, so that once back process is very painful, or changed computers, files are all running to die, we need to modify the path on their own, very painful.
After some exploration of their own, we can roughly divide the project into several sub-folders, code in your main folder:
2. Never manually modify the source data backup and do
we need a good backup of the source data to help us next time backtracking, may be the next operation or modification of the intermediate steps, and, on the other codes and other documents also need to be backed up to avoid accidental loss.
Xu Liang Linux here from an article, recommended four tools:
-
Git version control system
-
Rsync backup file
-
Dropbox cloud storage
-
Time Machine Time Machine
Introduction and use more tools I will not start here, you can go to inform themselves about the pictures.
3. do properly configured path
Many students at the time of writing are like direct path absolute path, although not a problem under normal circumstances, but if the code is shared with other people learning or running when the problem comes, are in many cases can not be directly run through,
It is recommended that:
Using a relative path: the script is located in the main directory, other resources (such as data, third-party packages, etc.) in which the same or lower directories, such as ./data/processed/test1.csv
Global Path configuration variables:
# 设置主目录
HOME_PATH = r'E:ML90615- PROJECT1'
# 读取数据
data = open(HOME_PATH+'/data/processed/test1.csv')
data = pd.read_csv(data)
data.head()
4. Code where necessary make Notes and Remarks
I believe that most people identify with, do not believe? Back a month ago to write the code to see it, to see what can understand how much (if not done Remarks words)
5. Accelerate your Python loop code
It is recommended that article: 24 accelerometers your python
Covered up, see more and more times, to form good habits chant, this way you will write code faster and faster -
6. Visualize your loop code progress
Here introduce a Python library, tqdm, install it: pip install tqdm
This is a cycle can show the progress of the library, it can have a more strategizing.
You can look at the following example:
7. Use an efficient tool to capture abnormal
Abnormal positioning bug, I often also a former print () function to go in the end, although that is also no problem, but the efficiency is still relatively slow, but later found a man named PySnooper decorator, if discovered the New World.
We generally debug, we may feel are in place to have a problem, go to the printout, look at what the actual output, and thinking the problem, which requires us to change the code, very carefully change compared directly add decoration device, it is very troublesome.
You can look at Example:
'''
遇到问题没人解答？小编创建了一个Python学习交流QQ群：857662006 寻找有志同道合的小伙伴，
互帮互助,群里还有不错的视频学习教程和PDF电子书！
'''
import pysnooper
@pysnooper.snoop('./file.log')
def number_to_bits(number):
if number:
bits = []
while number:
number, remainder = divmod(number, 2)
bits.insert(0, remainder)
return bits
else:
return [0]
number_to_bits(6)
We put every step of the output function are saved as file.log, we can see directly what went wrong in the end.
Project address: https: //github.com/cool-RR/pysnooper
8. Consider the code to be more robust
What is the robustness of the code, by definition, can stand up to test various scenarios exception, the exception processing "capture" and "throw" of two parts. "Capture" refers to the use of a particular parcel try ... except statement, proper completion of error handling process. And appropriate use raise the initiative to "throw" an exception, more elegant code is an integral part summed up the following points for your reference:
1) know what parameters passed, the type, the number (exception processing logic determines)
def add(a, b):
if isinstance(a, int) and isinstance(b, int):
return a+b
else:
return '参数类型错误'
print(add(1, 2))
print(add(1, 'a'))
2) only the most accurate exception caught
Sometimes we think of the script work is king, so willy-nilly to engage in a try ... except the big block of code is wrapped up, but it is easy to be thrown out of the original AttibuteError swallowed. Which gives us the debug process adds unnecessary trouble.
So, we will never capture only those statements block may throw an exception, but only try to capture the exact type of exception, rather than the vague Exception.
'''
遇到问题没人解答？小编创建了一个Python学习交流QQ群：857662006 寻找有志同道合的小伙伴，
互帮互助,群里还有不错的视频学习教程和PDF电子书！
'''
from requests.exceptions import RequestException
def save_website_title(url, filename):
try:
resp = requests.get(url)
except RequestException as e:
print(f'save failed: unable to get page content: {e}')
return False
# 这段正则操作本身就是不应该抛出异常的，所以我们没必要使用 try 语句块
# 假如 group 被误打成了 grop 也没关系，程序马上就会通过 AttributeError 来
# 告诉我们。
obj = re.search(r'<title>(.*)</title>', resp.text)
if not obj:
print('save failed: title tag not found in page content')
return False
title = obj.group(1)
try:
with open(filename, 'w') as fp:
fp.write(title)
except IOError as e:
print(f'save failed: unable to write to file {filename}: {e}')
return False
else:
return True
3) Exception handling should not be distracting
Speaking on a like exception catching to be accurate, but if each is very accurate, in fact we have a lot of code will try ... except statement block that disturb the core code, the code overall readability.
Here, we can use the context manager to improve our exception handling processes, simplifying the exception processing logic is repeated.
class raise_api_error:
"""captures specified exception and raise ApiErrorCode instead
:raises: AttributeError if code_name is not valid
"""
def __init__(self, captures, code_name):
self.captures = captures
self.code = getattr(error_codes, code_name)
def __enter__(self):
# 该方法将在进入上下文时调用
return self
def __exit__(self, exc_type, exc_val, exc_tb):
# 该方法将在退出上下文时调用
# exc_type, exc_val, exc_tb 分别表示该上下文内抛出的
# 异常类型、异常值、错误栈
if exc_type is None:
return False
if exc_type == self.captures:
raise self.code from exc_val
return False
In the above code, we define a context manager named raise_api_error, which is entering the context and do nothing. But when you exit context, it will determine whether the current context type self.captures thrown exception, if so, replace it with APIErrorCode exception class.
After use context manager, concise code is as follows:
def upload_avatar(request):
"""用户上传新头像"""
with raise_api_error(KeyError, 'AVATAR_FILE_NOT_PROVIDED'):
avatar_file = request.FILES['avatar']
with raise_api_error(ResizeAvatarError, 'AVATAR_FILE_INVALID'),
raise_api_error(FileTooLargeError, 'AVATAR_FILE_TOO_LARGE'):
resized_avatar_file = resize_avatar(avatar_file)
with raise_api_error(Exception, 'INTERNAL_SERVER_ERROR'):
request.user.avatar = resized_avatar_file
request.user.save()
return HttpResponse({})