Python code scanning: a new generation of Python Linter tool Ruff - highly recommended - why is it fast because it is written in Rust?

Table of contents

RUFF overview

Features

Installation and use

Application scenarios

Configuration

Editor integration

PyCharm (external tool)

Ruff's rules

Several reasons why Ruff is fast

at last

Note: Subsequent technology sharing, immediate updates, and more timely technical information and learning technical materials will be releasedon the official account CTO Plus . Please follow the official account: CTO Plus

Ruff is a Python-based code inspection tool that can help us find potential problems and provide repair suggestions when writing Python code. In this article, I will introduce the characteristics, usage and application scenarios of Ruff in projects under Python 3.11 version.

In recent years, Rust has become more and more influential in the field of front-end tool chains. It has even been proposed that "Rust is the future of JavaScript infrastructure" . All front-end tools that can be rewritten in Rust should be rewritten in Rust. The result is one word - fast. This trend has also been quietly brought into other interpreted language fields. Ruby uses Rust to implement the YJIT compiler; in the Python language field, the Ruff project has appeared, with an official introduction:

A very fast Python linter written in Rust: 10-100 times faster than existing linters ⚡️

This article on the public account CTO Plus gives a basic introduction to Rust " Rust: A safe, concurrent and high-performance system programming language, what benefits can it bring to us ", you can refer to this article for details, I will We will conduct an in-depth analysis of Ruff's source code. Let's take a brief look at its source code structure.

As can be seen from the source code, the project project uses Rust's Cargo construction tool. This tool will be described in the article "Rust Advanced 1. Install the Rust programming environment and write your first Rust program, as well as two rustc and cargo Introduction to tool usage and summary of commonly used instructions.

Under his core code directory crates, you can see all standard Rust projects built based on Cargo

And there are a lot of .rs files. .rs files are source code files written in Rust, while .py is Python source code files.

This article is the third in the [Code Standards and Scanning] series. You can read the previous articles according to your own situation:

RUFF overview

The first version of Ruff was released on August 30, 2022. It can be said to be a very new project. Although the author is iterating very actively (the current version is v0.0.198), the rule implementation is not as many as the commonly used Pylint rules , and It cannot completely replace Pylint. Currently, Pylint implements a total of 409 rules, while Ruff implements 224, of which 60 overlap with the Pylint rule set. Subjectively, Pylint tends to implement more type inference-based rules (e.g., verify the number of arguments in a function call). There is also official tracking on the alternative implementation of Pylint rules, see #970 for details . Ruff currently does not support third-party plug-ins. It only has a plug-in system within the project scope. For details, see https://github.com/astral-sh/ruff/issues/283.

Ruff uses Rust Python 's AST parser to implement its own AST traversal, visitor abstraction, and lint rule logic. It aims to be orders of magnitude faster than other tools, while providing a one-stop solution for code inspection, autofix, etc.

Ruff can be used to replace flake8 (plus various plugins), isort , pydocstyle , yesqa , eradicate , pyupgrade , and autoflake , all of which perform dozens or hundreds of times faster than any individual tool. Ruff goes beyond the responsibilities of a traditional linter and instead serves as an advanced code conversion tool, capable of updating type annotations, rewriting class definitions, sorting imports, and more.

Features

Ruff has the following characteristics:

1. First of all, it is fast and powerful.

2. Customized rules: Ruff provides a series of default rules and also supports custom rules. We can define our own rules based on the needs of the project to meet specific code quality requirements.

3. Code style check: Ruff can check whether the code style complies with PEP 8 specifications, such as indentation, naming conventions, code layout, etc.

4. Code quality inspection: Ruff can check some common problems in the code, such as unused variables, unused imports, duplicate code, etc., to improve the volume and maintainability of the code.

5. Static analysis: Ruff checks for potential problems by statically analyzing Python code, and can find potential errors and irregular code without running the code.

6. Report generation: Ruff can generate detailed reports including a description, location and recommended fixes for each problem. These reports can help us quickly locate and solve problems in the code.

Here is a brief introduction to usage:

Installation and use

The installation is very simple. With the help of maturin , Ruff can be installed through pip like other third-party Python packages. Just follow the steps below:

1. Install Ruff: You can install Ruff through the pip command, as shown below:

pip install ruff

After the installation is complete, you can see an executable file ruff.exe in the D:\env311\Scripts directory.

2. Execute inspection: Execute the following command on the command line to inspect the specified Python code:

Scan the specified directory: ruff /path/to/code

This will check all Python files under the specified path and generate the corresponding report.

Scan for specified files

ruff path/to/code/to/check.py

Scan using wildcards

ruff path/to/code/*.py

You can also run Ruff in watch mode to automatically execute it when the file changes:

ruff path/to/code/ --watch

Perform checks on different directories and files

ruff check . # Lint all files in the current directory (and any subdirectories)

ruff check path/to/code/ # Lint all files in `/path/to/code` (and any subdirectories)

ruff check path/to/code/*.py # Lint all `.py` files in `/path/to/code`

ruff check path/to/code/to/file.py # Lint `file.py`

Also works with pre-commit :

- repo: https://github.com/charliermarsh/ruff-pre-commit

# Ruff version.

rev: 'v0.0.198'

hooks:

- id: ruff

# Respect `exclude` and `extend-exclude` settings.

args: ["--force-exclude"]

3. Interpret the report: Ruff will generate a report file containing details of the inspection results and problems. We can use the recommendations in the report to fix issues in the code.

Application scenarios

Ruff can be applied in many projects, especially in large projects and team collaboration. The following are some application scenarios of Ruff:

1. Code specifications: For projects that follow the PEP 8 specification, Ruff can help us check whether the code style complies with the specification and provide corresponding repair suggestions.

2. Code quality: Ruff can help us check some common problems in the code, such as unused variables, unused imports and duplicate code, etc., to improve the quality and maintainability of the code.

3. Team collaboration: In team collaboration, Ruff can help us unify code style and quality standards to improve code consistency and readability.

4. Continuous integration: In a continuous integration environment, Ruff can be used as a checking step to help us find problems before the code is submitted to avoid merging wrong code into the main branch.

Configuration

Ruff can be configured via pyproject.toml and the command line. For a complete list of configurable options, see the API documentation . The default configuration is as follows:

[tool.ruff]
line-length = 88

# Enable Pyflakes `E` and `F` codes by default.
select = ["E", "F"]
ignore = []

# Exclude a variety of commonly ignored directories.
exclude = [
    ".bzr",
    ".direnv",
    ".eggs",
    ".git",
    ".hg",
    ".mypy_cache",
    ".nox",
    ".pants.d",
    ".ruff_cache",
    ".svn",
    ".tox",
    ".venv",
    "__pypackages__",
    "_build",
    "buck-out",
    "build",
    "dist",
    "node_modules",
    "venv",
]
per-file-ignores = {}

# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"

# Assume Python 3.10.
target-version = "py310"

[tool.ruff.mccabe]
# Unlike Flake8, default to a complexity level of 10.
max-complexity = 10

Of course, you can also customize the configuration. For example, the following will configure Ruff as:

Avoid checking line-length issues (E501)

Do not remove unused imports (F401)

Ignore import-at-top-of-file errors (E402) in file __init__.py

[tool.ruff]
# Enable Pyflakes and pycodestyle rules.
select = ["E", "F"]

# Never enforce `E501` (line length violations).
ignore = ["E501"]

# Never try to fix `F401` (unused imports).
unfixable = ["F401"]

# Ignore `E402` (import violations) in all `__init__.py` files, and in `path/to/file.py`.
[tool.ruff.per-file-ignores]
"__init__.py" = ["E402"]
"path/to/file.py" = ["E402"]

Ruff mimics Flake8's error code system, where each error code consists of a 1-3 letter prefix followed by 3 numbers (e.g. F401). The prefix indicates the "source" of the error code (e.g., F for Pyflakes, E for pycodestyle, ANN for flake8-annotations). The set of enabled errors is determined by the select and ignore options, which support both full error codes (e.g. F401) and prefixes (e.g. F).

Ruff also supports its own configuration file ruff.toml (similar to ESLint's .eslintrc), which lacks [tool.ruff] at the level, for example:

# Enable Pyflakes and pycodestyle rules.
select = ["E", "F"]

# Never enforce `E501` (line length violations).
ignore = ["E501"]

# Always autofix, but never try to fix `F401` (unused imports).
fix = true
unfixable = ["F401"]

# Ignore `E402` (import violations) in all `__init__.py` files, and in `path/to/file.py`.
[per-file-ignores]
"__init__.py" = ["E402"]
"path/to/file.py" = ["E402"]

Some common configuration settings can also be provided via the command line:

ruff path/to/code/ --select F401 --select F403

Similar to ESLint , Ruff supports hierarchical configuration, and the Python file uses the "closest" pyproject.toml as its configuration. Unlike ESLint , Ruff does not merge settings across configuration files; instead, the "most recent" configuration file is used, and any parent configuration files are ignored. Instead of this implicit cascading, Ruff supports an extension field that allows us to inherit settings from another pyproject.toml. like this:

# Extend the `pyproject.toml` file in the parent directory.

extend = "../pyproject.toml"

# But use a different line length.

line-length = 100

The above rules also apply to the ruff.toml configuration file. You can use the # noqa: {code} annotation in your code to ignore errors in lines/blocks of code:

# Ignore F841.

x = 1 # I: F8

# Ignore E741 and F841.

i = 1 # noga: E741, F841

# Ignore _all_ errors.

x = 1 # no

For the currently supported rules, see: https://github.com/charliermarsh/ruff#supported-rules

Editor integration

PyCharm ( external tool )

Ruff can be installed as an external tool in PyCharm. Open Preferences and navigate to Tools > External Tools. There, add a new tool with the following configuration:

Ruff can be used as a runnable action.

Ruff 's rules

Ruff supports over 500 lint rules, many of which are inspired by popular tools such as Flake8, isort, pyupgrade, etc. Regardless of the rule's origin, Ruff reimplements every rule in Rust as a first-party feature. Specific official reference rule library: https://beta.ruff.rs/docs/rules/

By default, Ruff enables Flake8's E and F rules. Ruff supports all rules of class F, as well as a subset of class E, omitting those style rules that are obsolete by using automatic formatters such as Black.
Flake8 and Black will be introduced in detail in the article later on the official account CTO Plus, so stay tuned.

If you're just starting out with Ruff, the default ruleset is a great place to start: it catches a variety of common errors (like unused imports) with zero configuration.

In addition to the default settings, Ruff reimplements some of the most popular Flake8 plugins and related code quality tools, including:

autoflake
eradicate
flake8-2020
flake8-annotations
flake8-async
flake8-bandit (#1646)
flake8-blind-except
flake8-boolean-trap
flake8-bugbear
flake8-builtins
flake8-commas
flake8-comprehensions
flake8-copyright
flake8-datetimez
flake8-debugger
flake8-django
flake8-docstrings
flake8-eradicate
flake8-errmsg
flake8-executable
flake8-future-annotations
flake8-gettext
flake8-implicit-str-concat
flake8-import-conventions
flake8-logging-format
flake8-no-pep420
flake8-pie
flake8-print
flake8-pyi
flake8-pytest-style
flake8-quotes
flake8-raise
flake8-return
flake8-self
flake8-simplify
flake8 slots
flake8-super
flake8-tidy-imports
flake8-all
flake8-type-checking
flake8-use-pathlib
flynt (#2102)
isort
mccabe
pandas know
pep8-naming
pydocstyle
pygrep-hooks
pylint-airflow
pyupgrade
tryceratops
yes, it is

Several reasons why Ruff is fast

The reason why Ruff can run very fast can be attributed to the following aspects:

1. Using AST (Abstract Syntax Tree) analysis: Ruff uses Python's built-in library ast to parse the source code and generate an abstract syntax tree. Compared to directly parsing the source code, AST analysis can traverse and analyze the code structure more efficiently, thereby improving performance.

2. Rule-based static analysis: Ruff checks code by defining a series of rules that can detect potential problems and style violations in the code. Compared with complex analysis techniques such as dynamic analysis or symbolic execution, rule-based static analysis methods are simpler and more efficient.

3. Parallel processing: Ruff can process multiple files in parallel to increase the speed of analysis. It can start multiple analysis tasks at the same time, and each task runs independently without interfering with each other. This can make full use of the computing power of multi-core processors and speed up analysis.

4. Optimization algorithms: Ruff uses some optimization algorithms to improve the efficiency of analysis. For example, you can cache modules that have been analyzed to avoid repeated parsing and analysis; you can also use pruning and other technologies to reduce unnecessary analysis paths and improve the efficiency of analysis.

at last

Ruff can also be used with black, another Python formatting tool, as long as line-length has the same configuration. A detailed introduction to the black tool will be given in the following article "Python Code Scanning: The Black Magic of One-Click Formatting Python Code - Black Usage Tutorial". Please stay tuned for the following article on the public account CTO Plus .

Ruff is a powerful Python code inspection tool that can help us find potential problems and provide repair suggestions. It checks code style and quality by statically analyzing Python code and generates detailed reports. In scenarios such as code specifications, code quality, team collaboration, and continuous integration, Ruff can play an important role in helping us improve the quality and maintainability of our code.

References

GitHub - astral-sh/ruff: An extremely fast Python linter, written in Rust.

https://github.com/charliermarsh/ruff#supported-rules

Meta issue: plugin system · Issue #283 · astral-sh/ruff · GitHub

https://beta.ruff.rs/docs/tutorial/

https://beta.ruff.rs/docs/rules/

https://beta.ruff.rs/docs/configuration/

https://beta.ruff.rs/docs/settings/

Python code specifications: Enterprise-level code static scanning - code specifications, logic, syntax, security checks, and automatic arrangement of code specifications (1)_pycharm Check code specifications_SteveRocket's blog-CSDN blog

https://blog.csdn.net/zhouruifu2015/article/details/129877179

Python column
https://blog.csdn.net/zhouruifu2015/category_5742543

More information · Search the WeChat public account [ CTO Plus ] and follow it to get more information. Let’s learn and communicate together.

For a description of the public account, visit the following link

For more exciting news, follow my official account and learn and grow together.

About Articulate "Be a porter of knowledge and technology. Be a lifelong learning enthusiast. Be a technical circle with depth and breadth." I have always wanted to develop skills in the professional field https://mp.weixin.qq. com/s?__biz=MzIyMzQ5MTY4OQ==&mid=2247484278&idx=1&sn=2b774f789b4c7a2ccf10e465a1b9def6&chksm=e81c2070df6ba966026fd7851efa824b5e2704e3fd34e 76228ca4ce64d93f7964cd4abe60f2b#rd

Standard Library Series-Recommended Reading: