This article recommends 10 practical python libraries, each of which is the best in the same kind of library. After trying it out, you are guaranteed to fall in love with such a library if you want to learn it.
There are an upgraded version of FastAPI Typer, the CLI into a colorful Rich, a GUI framework-based Dear PyGui, and PrettyErrors that simplifies error messages...
Let's take a look at it together~
1 、 Types
Typer has the same principle as FastAPI, and is a high-performance framework for building API services on Python.
It is an upgraded version of FastAPI that not only accurately documents code, but also enables easy CLI verification.
Typer is easy to learn and use, and does not require users to read complex tutorial documents to get started. Support editor (such as VSCode) code auto-completion, improve the development efficiency of developers and reduce the number of bugs.
Secondly, Typer can also be used in conjunction with the command line artifact Click, which can take advantage of Click's advantages and plug-ins to achieve more complex functions.
Open source address: https://github.com/tiangolo/typer
2、Rich
Who said the CLI interface had to be black and white? It can also be colored.
Rich API can not only provide rich color text and beautiful formatting in terminal output, but also provide beautiful tables, progress bars, editors, trackers, syntax highlighting, etc. As shown below.
It can also be installed on the Python REPL, and all data structures can be outputted or annotated beautifully.
All in all, it's colorful, beautiful, and powerful.
Rich compatibility is also good and works on a variety of systems such as Linux, Mac, and Windows. True colors/emojis work with the new Windows Terminal.
Note, however, that Rich must have Python 3.6.1 or above.
Open source address: https://github.com/willmcgugan/rich
3、Dear PyGui
As shown above, although the terminal application can be made beautiful. However, you may also need a real GUI.
Dear PyGui is an easy-to-use, powerful Python GUI framework. But it is fundamentally different from other Python GUIs.
It uses the immediate mode paradigm and the computer's GPU to implement dynamic interfaces. The immediate mode paradigm is very popular in video games, which means its dynamic GUI doesn't need to keep any data, but is drawn independently frame by frame. At the same time, it also uses the GPU to build dynamic interfaces.
Dear PyGui can also draw, create themes, create 2D games, and also has some small tools, such as built-in documentation, logging, source code viewer, etc., these small tools can assist in App development.
The systems that support it are: Windows 10 (DirectX 11), Linux (OpenGL 3) and macOS (Metal), etc.
Open source address: https://github.com/hoffstadt/DearPyGui
4、PrettyErrors
PrettyErrors is a tool for streamlining Python error messages, featuring a very simple and friendly interface.
Its most notable function is to support color output in the terminal, mark file stack traces, find error information, filter out redundant information, extract key parts, and perform color annotation, thereby improving the efficiency of developers.
Moreover, it can be used directly in the imported project without installation, but some parameters need to be configured first. The imported and configured parameters are as follows:
Open source address:
https://github.com/onelivesleft/PrettyErrors
5、Diagrams
When a programmer is programming, sometimes he needs to explain the complex structural relationship between the program codes he designed with his colleagues. However, this cannot be explained clearly in one or two sentences. It needs to draw a table or make a diagram.
Typically, programmers use GUI tools to process diagrams and visualize documents. But there are better ways, like using the Diagrams library.
Diagrams lets you draw cloud system structures directly in Python code without any design tools. Their icons come from multiple cloud service providers, including AWS, Azure, GCP, etc.
Arrow symbols and structure diagrams can be easily created with just a few lines of code.
Since it uses Graphviz to render graphs, Graphviz needs to be installed first.
Open source address:
https://github.com/mingrammer/diagrams
6、Hydra and OmegaConf
When doing machine learning projects, a lot of environment configuration work needs to be done. Therefore, in some complex applications, configuration management work becomes complicated accordingly.
Hydra makes configuration easy. It can override parts from the command line or configuration files, without maintaining similar configuration files, and configure them in a combined way, thus speeding up the running speed of experiments.
Hydra has strong compatibility and has a plug-in structure, which can be well integrated with the developer's operation file. Its plug-ins also allow you to publish code to AWS or other cloud systems directly from the command line.
Hydra is also inseparable from OmegaConf. The relationship between the two is inseparable. OmegaConf provides a collaborative API for Hydra's layered configuration system. The two work together to support YAML, configuration files, objects, and CLI parameters.
Open source address: https://github.com/facebookresearch/hydra,
https://github.com/omry/omegaconf
7、PyTorch Lightning
PyTorch Lightning is also a research result of Facebook. It is a lightweight PyTorch wrapper for high-performance AI research, and its most important feature is the ability to parse PyTorch code, allowing the separation of code research components and engineering components.
Its extended model can run on any hardware (CPU, GPU, TPU) and is easily copied, removes a large number of file samples, maintains its own flexibility, and runs fast.
Lightning is able to automate more than 40 parts of DL/ML research, such as GPU training, distributed GPU (cluster) training, TPU training, and more…
Because Lightning will automatically export files to ONNX or TorchScript, it is suitable for AI researchers doing fast reasoning, BERT or self-supervised learning research teams, etc.
Open source address:
https://github.com/PyTorchLightning/PyTorch-lightning
8、Hummingbird
Hummingbird is a Microsoft research effort that assembles already trained ML models into tensor computations, eliminating the need to design new models.
It also allows users to use neural network frameworks such as PyTorch to accelerate traditional ML models.
Its inference API is very similar to the sklearn paradigm, both can reuse existing code, but it is implemented using Hummingbird-generated code.
Hummingbird also provides a convenient unified inference API after the Sklearn API. This makes it possible to interchange Sklearn models with Hummingbird-generated models without changing the inference code.
It is also highlighted because of its ability to support a wide variety of models and formats.
So far, Hummingbird supports various ML models such as PyTorch, TorchScript, ONNX, and TVM.
Open source address:
https://github.com/microsoft/hummingbird
9、HiPlot
As ML models become more complex and have many hyperparameters, HiPlot is needed. HiPlot is a library released by Facebook in March this year, which is mainly used for processing high-dimensional data.
Facebook AI uses HiPlot to analyze deep neural networks with dozens of hyperparameters and more than 100,000 experiments.
It uses parallel graphs and other image methods to help AI researchers discover correlations and models of high-dimensional data. It is a lightweight interactive visualization tool.
Compared with other visualization tools, HiPlot has its own unique advantages:
First of all, it is highly interactive. Because the parallel graph is interactive, it can satisfy image visualization in many situations.
Second, it's simple to use and can be used directly through IPython Notebook or through a service with the "hiplot" command.
It is also extensible. By default, HiPlot's web service can parse CSV or JSON files, and can also provide it with a custom Python parser to convert experiments into HiPlot experiments.
Open source address:
https://github.com/facebookresearch/hiplot
Reference link:
https://ai.facebook.com/blog/hiplot-high-dimensional-interactive-plots-made-easy
10、Scalene
Scalene is a CPU and memory profiler for Python scripts that correctly handles multithreaded code and also differentiates between the runtime of Python code and native code.
You don't need to modify the code, just run the Scalene script and it will generate a textual report showing the CPU and memory usage for each line of code. With this text report, developers can improve the efficiency of their code.
Scalene is fast, accurate, and able to annotate energy-intensive lines of code.
Open source address
https://github.com/emeryberger/scalene
In addition to the above 10, several high-performance Python libraries have been named, such as Norfair, Quart, Alibi-detect, Einops, etc. For details, see the link at the bottom.
So, have you found a useful Python library this year?
If so, please share it in the comments section~
Reference link
https://tryolabs.com/blog/2020/12/21/top-10-python-libraries-of-2020/
https://www.upgrad.com/blog/reasons-why-python-popular-with-developers/
Reprinted from: qubits