PromQL selectors and operators

1. Introduction and research on platform unified monitoring
2. Intuitive experience of PromQL and its data types
3. PromQL selectors and operators

PromQL matchers

1. equality matcher (=)

Select the data exactly as the string provided

Example: filter out data with id="G1 Eden Space"

jvm_memory_used_bytes{id="G1 Eden Space"}

2. Not equal matcher (!=)

Contrary to the equality matcher, used to select data that is not the same as the provided string

Example: select data whose id is not G1 Eden Space

jvm_memory_used_bytes{id!="G1 Eden Space"}

3. Regular expression matcher (=~)

Select data that matches the provided regular expression

Example: filter out the data starting with G1 from the id tag

jvm_memory_used_bytes{id =~ "G1.*"}

4. Regular expression matcher for not equal to (!~)

Select data that does not match the provided regular expression

Example: filter out data that does not start with G1 from the id tag

jvm_memory_used_bytes{id !~ "G1.*"}

jvm_memory_used_bytes is the same as { name = "jvm_memory_used_bytes"}, other matchers can also be used

PromQL selectors

Momentary Vector Selector

Returns the latest sample value queried at the specified timestamp

Simplest form: returns an instant vector of all time series containing the indicator name

Example: Filter out all the data whose index is jvm_memory_used_bytes

jvm_memory_used_bytes

range vector selector

Return sample data over a period of time. Define the time through the end [], such as [1m], which means within 1 minute

Example: return data within one minute

jvm_memory_used_bytes[1m]

Each piece of data in the figure has 4 sample values, indicating that 4 data were collected within 1 minute.

Available time units: seconds (s), minutes (m), hours (h), days (d), weeks (w), years (y)

offset modifier

The time of instantaneous vector and interval vector can be shifted

Example: Query the sample value of jvm_memory_used_bytes in the previous 1 minute

jvm_memory_used_bytes offset 1m

Note the difference with jvm_memory_used_bytes[1m]

@ modifier

The @ modifier can modify the evaluation time of instantaneous vectors and interval vectors, represented by @timestamp

Example: Query the jvm_memory_used_bytes indicator at 2023-01-18 19:08:59

jvm_memory_used_bytes @1674040139

Example: When querying 2023-01-18 19:08:59, the jvm_memory_used_bytes indicator for the first 5 minutes

jvm_memory_used_bytes @1674040139 offset 5m

PromQL Operators

arithmetic operator

Support 6 arithmetic operators, addition (+), subtraction (-), multiplication (*), division (/), modulus (%), power (^)

Example 1: Calculate the heap memory usage
sum(jvm_memory_used_bytes{area="heap"}) indicates the used heap memory
sum(jvm_memory_max_bytes{area="heap"}) indicates the total heap memory

sum(jvm_memory_used_bytes{area="heap"})*100 / sum(jvm_memory_max_bytes{area="heap"})

Logical Operators

and (and), or (or), unless (exclusion)

• vector1 and vector2: generate a new vector, the elements in the vector are composed of the elements of vector1 that completely match the elements of vector2
• vector1 or vector2: Generate a new vector consisting of elements in vector1 and elements in vector2 that do not match vector1
• vector1 unless vector2: Generate a new vector consisting of elements in vector1 that do not match vector2

Still use the jvm_memory_used_bytes indicator as an example.

and example:

vector1: heap memory data of all instances

jvm_memory_used_bytes{area="heap"}

vector2: filter out instance="192.168.0.113:9000", area="heap" data

jvm_memory_max_bytes{instance="192.168.0.113:9000",area="heap"}

After the and operator, only the data with the same label as instance="192.168.0.113:9000", area="heap" is retained

jvm_memory_used_bytes{area="heap"} and jvm_memory_max_bytes{instance="192.168.0.113:9000",area="heap"}

It is found that the and operator does not care whether the index name is the same

or example:

To be more intuitive, use another indicator http_server_requests_seconds_count

unless example:

comparison operator

== equal, != not equal, > greater than, < less than, >= greater than or equal to, <= less than or equal to

Example: Adding the bool keyword after the operator allows the result to return 0 or 1

99 >= bool 88

vector matching

Prometheus's vector-to-vector operations will be based on the default matching rules: find the right vector elements that match the left vector elements (with exactly the same labels) for operation, and if no matching elements are found, discard them directly. Vector matching mainly includes: one-to-one, one-to-many, and many-to-one.

One-to-one matching:

That is, the two passes have exactly the same tags, and the only entry is found to be matched in turn.

process_open_fds / process_max_fds

If the tags on both sides are inconsistent, you can use the on or ignoring keyword to modify the matching behavior between tags.

on: Specify the tag to match, and only match the specified tag.

ignoring: Ignore certain tags, that is, the specified tags do not match, and the others match.

In the following example, only the two tags instance and job are matched.

sum by(instance, job) (rate(node_cpu_seconds_total{mode="idle"}[5m])) 
/ on (instance, job) 
sum by(instance, job) (rate(node_cpu_seconds_total [5m] ))

Comparing two instantaneous vectors returns the expression on the left:

The following examples will return the expression on the left: you can write the expression according to your needs.

process_open_fds < process_max_fds
process_max_fds >  process_open_fds

One -to-many or many-to-one matching:
group_left: more subsets on the left group_right: more subsets
on the right

usage:

<vector expr> <bin-op> ignoring(<label list>) group_left(<label list>) <vector expr>
<vector expr> <bin-op> ignoring(<label list>) group_right(<label list>) <vector expr>
<vector expr> <bin-op> on(<label list>) group_left(<label list>) <vector expr>
<vector expr> <bin-op> on(<label list>) group_right(<label list>) <vector expr>

Grouping can only be used in comparison and arithmetic operators

Other articles by the author:
Grafana series articles, version: OOS v9.3.1

1. Introduction and installation of Grafana
2. Introduction to configuration parameters of Grafana monitoring large screen (1)
3. Introduction to configuration parameters of Grafana monitoring large screen (2)
4. Grafana monitors large-screen visualization charts
5. Grafana query data and transform data
6. Introduction to Grafana Alarm Module
7. Grafana alarm access Feishu notification

---

Spring Boot Admin series

1. Spring Boot Admin Reference Guide
2. The problem that the SpringBoot Admin service is offline and does not display health information
3. Loading of Spring Boot Admin2 @EnableAdminServer
4. Detailed Explanation of Spring Boot Admin2 AdminServerAutoConfiguration
5. Detailed Explanation of Spring Boot Admin2 Instance Status Monitoring
6. Spring Boot Admin2 custom JVM monitoring notification
7. Spring Boot Admin2 custom exception monitoring
8. Spring Boot Admin monitoring indicators connected to Grafana visualization