8 tips commonly used by Python programmers, very helpful

Hello brothers, in this article, let’s share some common tips for Python programmers, hoping to help you.

Collating string input

The problem of collating user input is extremely common in programming. Usually, converting characters to lowercase or uppercase is sufficient, and sometimes you can use the regular expression module "Regex" for this job. But if the problem is complex, there might be a better way to solve it:

user_input = "This\nstring has\tsome whitespaces...\r\n"

character_map = {
    
    
    ord('\n') : ' ',
    ord('\t') : ' ',
    ord('\r') : None
}
user_input.translate(character_map)  # This string has some whitespaces... 

In this example, you can see that the space characters "\n" and "\t" have been replaced with a single space, and "\r" has been deleted. This is just a very simple example, we can go one step further, use the "unicodedata" package to generate a large remapping table, and use the "combining()" in it to generate and map, we can

Iterator slice (Slice)

If the iterator is sliced, a "TypeError" will be returned, indicating that the generator object has no subscript, but we can use a simple solution to solve this problem:

import itertools

s = itertools.islice(range(50), 10, 20)  # <itertools.islice object at 0x7f70fab88138>
for val in s:
    ...

We can use "itertools.islice" to create an "islice" object, which is an iterator that yields the items we want. Note, however, that this operation consumes all generator items up to the slice, and all items in the "islice" object.

skip the beginning of an iterable

Sometimes you have to deal with files that start with unwanted lines such as comments. "itertools" again provides a simple solution:

string_from_file = """
// Author: ...
// License: ...
//
// Date: ...
Actual content...
"""

import itertools

for line in itertools.dropwhile(lambda line: line.startswith("//"), string_from_file.split("\n")):
    print(line)

This code only prints what follows the initial comment section. This is useful if we only want to discard the beginning of the iterable (in this case, the opening comment line), but we don't know how long we want it to be.

Functions that contain only keyword arguments (kwargs)

When using functions like the following, it can be helpful to create functions that only require keyword arguments as input to provide a clearer function definition:

def test(*, a, b):
    pass

test("value for a", "value for b")  # TypeError: test() takes 0 positional arguments...
test(a="value", b="value 2")  # Works...

As you can see, prefixing the keyword arguments with a " " solves the problem. If we put some parameters before the " " parameter, they are obviously positional parameters.

Create an object that supports the "with" statement

For example, we all know how to use the "with" statement to open a file or acquire a lock, but can we implement our own context expressions? Yes, we can use " enter " and " exit " to implement the context management protocol:

class Connection:
    def __init__(self):
        ...

    def __enter__(self):
        # Initialize connection...

    def __exit__(self, type, value, traceback):
        # Close connection...

with Connection() as c:
    # __enter__() executes
    ...
    # conn.__exit__() executes

This is the most common way of implementing context management in Python, but there are simpler ways:

from contextlib import contextmanager

@contextmanager
def tag(name):
    print(f"<{
      
      name}>")
    yield
    print(f"</{
      
      name}>")

with tag("h1"):
    print("This is Title.")

The code above implements the content management protocol using the contextmanager's manager decorator. The first part of the tag function (the part before yield) is executed when entering the with block, then the with block is executed, and finally the rest of the tag function is executed.

Save memory with " slots "

If you've ever written a program that created a large number of instances of a certain class, you've probably noticed that your program suddenly requires a lot of memory. That's because Python uses dictionaries to represent attributes of class instances, which makes it fast, but not very memory efficient. Usually, this is not a serious problem. However, if your program is seriously affected by this, try " slots ":

class Person:
    __slots__ = ["first_name", "last_name", "phone"]
    def __init__(self, first_name, last_name, phone):
        self.first_name = first_name
        self.last_name = last_name
        self.phone = phone

When we defined the " slots " attribute, Python did not use a dictionary to represent the attribute, but a small fixed-size array, which greatly reduces the memory required for each instance. Using " slots " also has some disadvantages: we cannot declare any new properties, we can only use existing properties on the " slots ". Also, classes with " slots " cannot use multiple inheritance.

Limit "CPU" and memory usage

If you don't want to optimize the program's memory or CPU usage, but want to directly limit it to a certain number, Python also has a corresponding library that can do it:

import signal
import resource
import os

# To Limit CPU time
def time_exceeded(signo, frame):
    print("CPU exceeded...")
    raise SystemExit(1)

def set_max_runtime(seconds):
    # Install the signal handler and set a resource limit
    soft, hard = resource.getrlimit(resource.RLIMIT_CPU)
    resource.setrlimit(resource.RLIMIT_CPU, (seconds, hard))
    signal.signal(signal.SIGXCPU, time_exceeded)

# To limit memory usage
def set_max_memory(size):
    soft, hard = resource.getrlimit(resource.RLIMIT_AS)
    resource.setrlimit(resource.RLIMIT_AS, (size, hard))

We can see that in the code snippet above, there are options to set the maximum CPU runtime and the maximum memory usage limit at the same time. When limiting the runtime of the CPU, we first get the soft and hard limits for that particular resource (RLIMIT_CPU), and then set them with the number of seconds specified by the parameter and the previously retrieved hard limit. Finally, we will signal the system to exit if the CPU runtime exceeds the limit. In terms of memory usage, we again retrieve the soft and hard limits and set it using 'setrlimit' with the 'size' parameter and the previously retrieved hard limit.

Control what can/cannot be imported

Some languages ​​have very obvious mechanisms for exporting members (variables, methods, interfaces), for example in Golang only members starting with a capital letter are exported. However, in Python, all members are exported (unless we use " all "):

def foo():
    pass

def bar():
    pass

__all__ = ["bar"]

In the code above, we know that only the "bar" function is exported. Likewise, we can make " all " empty, so that nothing is exported, which will cause an "AttributeError" when imported from this module.

Simple way to implement comparison operators

It is tedious to implement all comparison operators (eg lt , le , gt , ge ) for a class. Is there an easier way to do this? At this time, "functools.total_ordering" is a good helper:

from functools import total_ordering

@total_ordering
class Number:
    def __init__(self, value):
        self.value = value

    def __lt__(self, other):
        return self.value < other.value

    def __eq__(self, other):
        return self.value == other.value

print(Number(20) > Number(3))
print(Number(1) < Number(5))
print(Number(15) >= Number(15))
print(Number(10) <= Number(2))

How does it work here? We simplify the implementation of ordering class instances with the "total_ordering" decorator. We only need to define " lt " and " eq ", which are the minimum set of operations required to implement the rest of the operations (here also reflects the role of the decorator - filling in the blanks for us).

In the new year, you must kill your classmates and colleagues to improve your Python strength. Here I have prepared the latest Python learning materials for 2023, hundreds of e-books, basic video tutorials, and practical case tutorials. Get the business card at the end of the article directly!

epilogue

Not all of the features mentioned in this article are necessary or useful in everyday Python programming, but some of them may come in handy from time to time, and they may simplify some otherwise tedious and annoying tasks. It's also worth pointing out that all of these functions are part of the Python standard library. In my opinion, some of these functions do not seem to be standard content contained in the standard library, so when you use Python to implement some of the functions mentioned in this article, please refer to Python's standard library first, if you can't find what you want function, probably just because you haven't looked it up enough (and if there isn't, then it must exist in some third-party library).

That's all for today's sharing, see you next time!