028. Function of SQL statement

Functions in TiDB expressions

• The function performs calculations on the input data
• The return value of a function can be part of an expression
• Description: function_name([arg1 [, arg2 ,... , argn ]])
• example
  • now(): returns the current date and time
  • version(): returns the current TiDB server version

use function

• Functions can be used anywhere an expression is accepted
• Most functions require parameters to execute
• Columns can be used as arguments
• The output of one function can be used as the input of another function
• Math function returns Null value on error
  • For example, in error_for_division_by_zero SQL mode, an error will be reported

mysql> insert into test01 values(8/0);
ERROR 1365 (22012): Division by 0
mysql> select 8/0;
+------+
| 8/0  |
+------+
| NULL |
+------+
1 row in set, 1 warning (0.00 sec)

mysql> select @@SQL_MODE;
+-------------------------------------------------------------------------------------------------------------------------------------------+
| @@SQL_MODE                                                                                                                                |
+-------------------------------------------------------------------------------------------------------------------------------------------+
| ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+-------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

function type

• Strings: operations on strings
• Date and time: operations on date and time
• Numeric: math operations
• Aggregation: returns a single record based on multiple records input
• Window: returns the result set of the partition calculation

string functions

• Calculate string length
• extract substring
• search or replace substring
• case conversion
• convert bit string or number

case conversion

lower()/lcase(),upper()/ucase: return a string with the result changed to lowercase or uppercase.

mysql> select lower('TiDB'),lcase('TiDB'),upper('TiDB'),ucase('TiDB');
+---------------+---------------+---------------+---------------+
| lower('TiDB') | lcase('TiDB') | upper('TiDB') | ucase('TiDB') |
+---------------+---------------+---------------+---------------+
| tidb          | tidb          | TIDB          | TIDB          |
+---------------+---------------+---------------+---------------+
1 row in set (0.01 sec)

compare strings

• strcmp(): Compare str1 and str2
• return value:
  • 0 means str1 and str2 are the same
  • -1 means that str1 is less than str2
  • 1 means str1 is greater than str2
• =: Test strings for equality

mysql> select strcmp('aBC','Abc'),strcmp('ABC','ABC'),strcmp('Abc','aBC'),'ABC'='ABC';
+---------------------+---------------------+---------------------+-------------+
| strcmp('aBC','Abc') | strcmp('ABC','ABC') | strcmp('Abc','aBC') | 'ABC'='ABC' |
+---------------------+---------------------+---------------------+-------------+
|                   1 |                   0 |                  -1 |           1 |
+---------------------+---------------------+---------------------+-------------+
1 row in set (0.01 sec)

find a string in a string

• insert(), locate(), position(): returns the position of the specified string in another string
• locate(searchStr, str, pos): where pos specifies the starting position of the search

mysql> select instr('TiDB','DB'),instr('TiDB','D'),locate('DB','TiDB'),position('DB' in 'TiDB');
+--------------------+-------------------+---------------------+--------------------------+
| instr('TiDB','DB') | instr('TiDB','D') | locate('DB','TiDB') | position('DB' in 'TiDB') |
+--------------------+-------------------+---------------------+--------------------------+
|                  3 |                 3 |                   3 |                        3 |
+--------------------+-------------------+---------------------+--------------------------+
1 row in set (0.00 sec)

mysql> select locate('DB','TiDB is a NewSQL style DB',7);
+--------------------+-------------------+---------------------+--------------------------+
|              24 |
+--------------------+-------------------+---------------------+--------------------------+
1 row in set (0.00 sec)

reverse, connect

• concat(), concat_ws(): concatenate the given parameters into a string
• reverse() : returns a string with characters in reverse order

mysql> select reverse('abcdefg') , concat('hello','world'),concat_ws(':','pingcap','cmg','pe');
+--------------------+-------------------------+-------------------------------------+
| reverse('abcdefg') | concat('hello','world') | concat_ws(':','pingcap','cmg','pe') |
+--------------------+-------------------------+-------------------------------------+
| gfedcba            | helloworld              | pingcap:cmg:pe                      |
+--------------------+-------------------------+-------------------------------------+
1 row in set (0.00 sec)

Handle Null

• Passing Null to the concat function, causing it to return null
• concat_ws() ignores null parameters

mysql> select concat('welcome ','to ',null),concat_ws(',','welcome','to',null);
+-------------------------------+------------------------------------+
| concat('welcome ','to ',null) | concat_ws(',','welcome','to',null) |
+-------------------------------+------------------------------------+
| NULL                          | welcome,to                         |
+-------------------------------+------------------------------------+
1 row in set (0.00 sec)

Concatenate strings using the pipe operator

mysql> set sql_mode=pipes_as_concat;
Query OK, 0 rows affected (0.00 sec)

mysql> select 'welcome '|| ' to '||'TiDB!';
+------------------------------+
| 'welcome '|| ' to '||'TiDB!' |
+------------------------------+
| welcome  to TiDB!            |
+------------------------------+
1 row in set (0.00 sec)

left,right,lpad,rpad

left(str,numchar),right(str,numchar): Returns the leftmost or rightmost numchar characters in a string
lpad(str,len,padstr),rpad(str,len,padstr): Returns a character on the left or right A string padded with padstr on the right, up to len characters in length.
For example:

mysql> select left('TSO',1),right('TSO',2),lpad('TSO',4,'#'),rpad('TSO',4,'#');
+---------------+----------------+-------------------+-------------------+
| left('TSO',1) | right('TSO',2) | lpad('TSO',4,'#') | rpad('TSO',4,'#') |
+---------------+----------------+-------------------+-------------------+
| T             | SO             | #TSO              | TSO#              |
+---------------+----------------+-------------------+-------------------+
1 row in set (0.01 sec)

substring retrieves part of a string

substring(string,startpos[,length]): Returns a substring whose length does not exceed the character starting at startpos.
If length is omitted, the substring will include all characters at the end
. If startpos is negative, the substring return from the end

mysql> select substring('TiDB SQL',2),substring('TiDB SQL',1,2),substring('TiDB SQL',-2,2);
+-------------------------+---------------------------+----------------------------+
| substring('TiDB SQL',2) | substring('TiDB SQL',1,2) | substring('TiDB SQL',-2,2) |
+-------------------------+---------------------------+----------------------------+
| iDB SQL                 | Ti                        | QL                         |
+-------------------------+---------------------------+----------------------------+
1 row in set (0.00 sec)

substring_index

Returns the substring in the string preceding count occurrences of the delimiter, or reverses direction if count is negative

mysql> select substring_index('853:888:777:598',':',1),substring_index('853:888:777:598',':',2),substring_index('853:888:777:598',':',-2) \G;
*************************** 1. row ***************************
 substring_index('853:888:777:598',':',1): 853
 substring_index('853:888:777:598',':',2): 853:888
substring_index('853:888:777:598',':',-2): 777:598
1 row in set (0.00 sec)

ERROR: 
No query specified

trim,ltrim,rtrim: trim string

trim([BOTH|LEADING|TRAILING [substring] FROM] string)
returns the string after substring is removed
If both,leading trailing options are not specified, the default is both
If substring is omitted, spaces will be removed
ltrim(),rtrim() ,trim(): can remove space characters that appear on the left, right or both sides

mysql> select trim(leading 'Ti' from 'Tidb'),ltrim(' TiDB'),'#'||trim('  TiKV  ')||'#';
+--------------------------------+----------------+----------------------------+
| trim(leading 'Ti' from 'Tidb') | ltrim(' TiDB') | '#'||trim('  TiKV  ')||'#' |
+--------------------------------+----------------+----------------------------+
| db                             | TiDB           |                          0 |
+--------------------------------+----------------+----------------------------+
1 row in set, 3 warnings (0.00 sec)

mysql> set @@sql_mode=ORACLE;
Query OK, 0 rows affected (0.00 sec)
mysql> select trim(leading 'Ti' from 'Tidb'),ltrim(' TiDB'),'#'||trim('  TiKV  ')||'#';
+--------------------------------+----------------+----------------------------+
| trim(leading 'Ti' from 'Tidb') | ltrim(' TiDB') | '#'||trim('  TiKV  ')||'#' |
+--------------------------------+----------------+----------------------------+
| db                             | TiDB           | #TiKV#                     |
+--------------------------------+----------------+----------------------------+
1 row in set (0.00 sec)
mysql> select trim(leading 'Ti' from 'TidbTi');
+----------------------------------+
| trim(leading 'Ti' from 'TidbTi') |
+----------------------------------+
| dbTi                             |
+----------------------------------+
1 row in set (0.00 sec)

mysql> select trim(trailing 'Ti' from 'TidbTi');
+-----------------------------------+
| trim(trailing 'Ti' from 'TidbTi') |
+-----------------------------------+
| Tidb                              |
+-----------------------------------+
1 row in set (0.00 sec)

mysql> select trim(both 'Ti' from 'TidbTi');
+-------------------------------+
| trim(both 'Ti' from 'TidbTi') |
+-------------------------------+
| db                            |
+-------------------------------+
1 row in set (0.00 sec)

Insert and replace parts of strings

replace (str,substr,newstring) : Replace substr in str with a new string
insert(str,pos,len,newstring): Place a new string at pos in str, covering len length characters

mysql> select replace('Use K8s to setup','K8s','tiup'),insert('Use OGG to migrate',5,3,'dumpling and Lightning');
+------------------------------------------+-----------------------------------------------------------+
| replace('Use K8s to setup','K8s','tiup') | insert('Use OGG to migrate',5,3,'dumpling and Lightning') |
+------------------------------------------+-----------------------------------------------------------+
| Use tiup to setup                        | Use dumpling and Lightning to migrate                     |
+------------------------------------------+-----------------------------------------------------------+
1 row in set (0.01 sec)

Determine string length

length(): returns the length of the string in bytes
char_length(): returns the length of the string in characters

mysql> select length('TiDB 部署中'),char_length('TiDB 部署中');
+--------------------------+-------------------------------+
| length('TiDB 部署中')    | char_length('TiDB 部署中')    |
+--------------------------+-------------------------------+
|                       14 |                             8 |
+--------------------------+-------------------------------+
1 row in set (0.00 sec)

Match string pattern: like

Use the like operator to match a string
expression like 'pattern'
Use the wildcard % in the pattern
: to match any sequence of zero or more characters
_: to match any single character
not like: for reverse matching

Match string pattern: rlike

Regular expressions are a powerful method for complex searches of specified patterns
rlike, regxp operators: expression rlike | regxp 'regexp-pattern'
\ as escape character
() to group multiple regex elements
^:beginning of string
$: end of string
.: any single character
x*: zero or more x
x+: one or more x
x?: zero or one x

example

Los|Las, L[oa]s: 表达式等价
^[A-HJ-NP-Z][0-9]{
    
    4}([A-Z]{
    
    3})?$

mysql> select name,mass from universe.planets where name rlike '(Ura|Ve)nus';
+--------+------+
| name   | mass |
+--------+------+
| Venus  | 4.87 |
| Uranus | 86.8 |
+--------+------+
2 rows in set (0.00 sec)

date and time functions

Do something like
Extract part of the date and time Change
the format Convert Date and time data can be generated in a number of ways Copy existing data Use built-in functions Build a string representation that TiDb evaluates as a date and time

date and time format

date type	default format
date	yyyy-mm-dd
time	hh:mm:ss
datetime	yyyy-mm-dd hh:mm:ss
timestamp	yyyy-mm-dd hh:mm:ss
year	yyyy

date and time functions

now(): current date and time on the server, return datetime
curdate(): current date on the server, return date
curtime(): current time on the server, return time
year(): year year format
monthname(): year The month in
dayname(): the total day of the week
dayofyear(): the day of the year
hour(), minute(), second(), microsecond()

mysql> select now(),curdate(),curtime(),year(now()),dayname(now());
+---------------------+------------+-----------+-------------+----------------+
| now()               | curdate()  | curtime() | year(now()) | dayname(now()) |
+---------------------+------------+-----------+-------------+----------------+
| 2023-06-05 18:38:10 | 2023-06-05 | 18:38:10  |        2023 | Monday         |
+---------------------+------------+-----------+-------------+----------------+
1 row in set (0.00 sec)

date and time arithmetic

date_add,date_sub
+/- interval

mysql> select now() + interval '1 12:12:12.123456' DAY_MICROSECOND,now() + interval '12.123456' SECOND_MICROSECOND,now() - interval '1-3' YEAR_MONTH, date_add(now(), interval 1 week),date_sub(now(),interval 1 week),date_sub(now(),interval 1 quarter);
+------------------------------------------------------+-------------------------------------------------+-----------------------------------+----------------------------------+---------------------------------+------------------------------------+
| now() + interval '1 12:12:12.123456' DAY_MICROSECOND | now() + interval '12.123456' SECOND_MICROSECOND | now() - interval '1-3' YEAR_MONTH | date_add(now(), interval 1 week) | date_sub(now(),interval 1 week) | date_sub(now(),interval 1 quarter) |
+------------------------------------------------------+-------------------------------------------------+-----------------------------------+----------------------------------+---------------------------------+------------------------------------+
| 2023-06-07 06:56:21.123456                           | 2023-06-05 18:44:21.123456                      | 2022-03-05 18:44:09               | 2023-06-12 18:44:09              | 2023-05-29 18:44:09             | 2023-03-05 18:44:09                |
+------------------------------------------------------+-------------------------------------------------+-----------------------------------+----------------------------------+---------------------------------+------------------------------------+
1 row in set (0.03 sec)

convert date format to string

date_format()
http://dev.mysql.com/doc/mysql/en/date-and-time-functions.html#function_date-format
get_format()
return get_format({date|time|datetime},{'EUR' ,'USA','YES','ISO','INTERNAL'})

mysql> select date_format(now(),'%a %Y-%m-%d %T'),date_format(now(),'%Y-%m-%d %r'),date_format(now(),'%W,%M %e, %Y'),date_format(now(),'%a, %b %e %l:%i %p'),date_format(now(),'%d-%b-%Y'),date_format(now(),'%a, %b %e ,%Y'),date_format(now(),'%M %e,%Y %H:%i'),date_format(now(),'%W ,the %D of %M') \G;
*************************** 1. row ***************************
    date_format(now(),'%a %Y-%m-%d %T'): Mon 2023-06-05 18:51:58
       date_format(now(),'%Y-%m-%d %r'): 2023-06-05 06:51:58 PM
      date_format(now(),'%W,%M %e, %Y'): Monday,June 5, 2023
date_format(now(),'%a, %b %e %l:%i %p'): Mon, Jun 5 6:51 PM
          date_format(now(),'%d-%b-%Y'): 05-Jun-2023
     date_format(now(),'%a, %b %e ,%Y'): Mon, Jun 5 ,2023
    date_format(now(),'%M %e,%Y %H:%i'): June 5,2023 18:51
  date_format(now(),'%W ,the %D of %M'): Monday ,the 5th of June
1 row in set (0.00 sec)

ERROR: 
No query specified

creation date

makedate(year,dayofyear): create date
maketime(hour,min,sec): create time
str_to_date(str,fmt): create datetime
literal value

mysql> select get_format(DATE,'INTERNAL'),get_format(TIME,'INTERNAL'),get_format(DATETIME,'INTERNAL'),makedate(2023,101),maketime(12,23,54) \G;
*************************** 1. row ***************************
    get_format(DATE,'INTERNAL'): %Y%m%d
    get_format(TIME,'INTERNAL'): %H%i%s
get_format(DATETIME,'INTERNAL'): %Y%m%d%H%i%s
             makedate(2023,101): 2023-04-11
             maketime(12,23,54): 12:23:54
1 row in set (0.01 sec)

ERROR: 
No query specified

compare dates

datediff(now(),'2023-01-01') : returns the number of days between two dates

mysql> select datediff(now(),'2023-01-01'),datediff('2023-01-01',now());
+------------------------------+------------------------------+
| datediff(now(),'2023-01-01') | datediff('2023-01-01',now()) |
+------------------------------+------------------------------+
|                          179 |                         -179 |
+------------------------------+------------------------------+
1 row in set (0.00 sec)

basic arithmetic functions

mysql> select abs(-234.5),sign(-6),floor(343.543),ceiling(2343.34),truncate(343.4543,3),truncate(343.4543,-1),round(343.4543),round(343.4543,2),round(343.4543,-2);
+-------------+----------+----------------+------------------+----------------------+-----------------------+-----------------+-------------------+--------------------+
| abs(-234.5) | sign(-6) | floor(343.543) | ceiling(2343.34) | truncate(343.4543,3) | truncate(343.4543,-1) | round(343.4543) | round(343.4543,2) | round(343.4543,-2) |
+-------------+----------+----------------+------------------+----------------------+-----------------------+-----------------+-------------------+--------------------+
|       234.5 |       -1 |            343 |             2344 |              343.454 |                   340 |             343 |            343.45 |                300 |
+-------------+----------+----------------+------------------+----------------------+-----------------------+-----------------+-------------------+--------------------+
1 row in set (0.01 sec)

Group by group

Group row records
only_full_group_by: If the columns appear in the select list and appear in the group by or aggregation function, the SQL is illegal

window function

• The window function calculates the output for each input record in the result set, rank ranking
• In addition to RANK() in the example, the window functions supported by TiDB include:

cume_dist(),dense_rank(),first_value(),lag(),last_value(),lead(),nth_value(),ntitle(),percent_rank(),row_number()

rank: ranking

mysql> select name,mass,gravity,rank() over (partition by category_id order by mass desc) as mass_rank_within_category from universe.planets;
+--------------------+--------+---------+---------------------------+
| name               | mass   | gravity | mass_rank_within_category |
+--------------------+--------+---------+---------------------------+
| Proxima Centauri b | 7.5819 |    11.3 |                         1 |
| Earth              |   5.97 |     9.8 |                         2 |
| Venus              |   4.87 |     8.9 |                         3 |
| Mars               |  0.642 |     3.7 |                         4 |
| Mercury            |   0.33 |     3.7 |                         5 |
| Pluto              |  0.013 |     0.7 |                         1 |
| Jupiter            |   1898 |    23.1 |                         1 |
| Saturn             |    568 |     9.0 |                         2 |
| Neptune            |    102 |    11.0 |                         3 |
| Uranus             |   86.8 |     8.7 |                         4 |
+--------------------+--------+---------+---------------------------+
10 rows in set (0.00 sec)

lag: display the previous line

mysql> select name,mass,lag(name,1,'---') over (order by mass) from universe.planets order by mass;
+--------------------+--------+----------------------------------------+
| name               | mass   | lag(name,1,'---') over (order by mass) |
+--------------------+--------+----------------------------------------+
| Pluto              |  0.013 | ---                                    |
| Mercury            |   0.33 | Pluto                                  |
| Mars               |  0.642 | Mercury                                |
| Venus              |   4.87 | Mars                                   |
| Earth              |   5.97 | Venus                                  |
| Proxima Centauri b | 7.5819 | Earth                                  |
| Uranus             |   86.8 | Proxima Centauri b                     |
| Neptune            |    102 | Uranus                                 |
| Saturn             |    568 | Neptune                                |
| Jupiter            |   1898 | Saturn                                 |
+--------------------+--------+----------------------------------------+
10 rows in set (0.01 sec)

show the top two lines

mysql> select name,mass,lag(name,2,'---') over (order by mass) from universe.planets order by mass;
+--------------------+--------+----------------------------------------+
| name               | mass   | lag(name,2,'---') over (order by mass) |
+--------------------+--------+----------------------------------------+
| Pluto              |  0.013 | ---                                    |
| Mercury            |   0.33 | ---                                    |
| Mars               |  0.642 | Pluto                                  |
| Venus              |   4.87 | Mercury                                |
| Earth              |   5.97 | Mars                                   |
| Proxima Centauri b | 7.5819 | Venus                                  |
| Uranus             |   86.8 | Earth                                  |
| Neptune            |    102 | Proxima Centauri b                     |
| Saturn             |    568 | Uranus                                 |
| Jupiter            |   1898 | Neptune                                |
+--------------------+--------+----------------------------------------+
10 rows in set (0.02 sec)

lead : display the next line

mysql> select name,mass,lead(name,1,'---') over (order by mass) from universe.planets order by mass;
+--------------------+--------+-----------------------------------------+
| name               | mass   | lead(name,1,'---') over (order by mass) |
+--------------------+--------+-----------------------------------------+
| Pluto              |  0.013 | Mercury                                 |
| Mercury            |   0.33 | Mars                                    |
| Mars               |  0.642 | Venus                                   |
| Venus              |   4.87 | Earth                                   |
| Earth              |   5.97 | Proxima Centauri b                      |
| Proxima Centauri b | 7.5819 | Uranus                                  |
| Uranus             |   86.8 | Neptune                                 |
| Neptune            |    102 | Saturn                                  |
| Saturn             |    568 | Jupiter                                 |
| Jupiter            |   1898 | ---                                     |
+--------------------+--------+-----------------------------------------+
10 rows in set (0.01 sec)

Flow Control function

• Case expression
• nullif() function
• ifnull() function
• if() function

case expression

mysql> select case when null='' then true when null <>'' then false else'=)' end;
+--------------------------------------------------------------------+
| case when null='' then true when null <>'' then false else'=)' end |
+--------------------------------------------------------------------+
| =)                                                                 |
+--------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> select null<>'';
+----------+
| null<>'' |
+----------+
|     NULL |
+----------+
1 row in set (0.00 sec)

mysql> select case null when '' then true when 0 then false else '=)' end;
+-------------------------------------------------------------+
| case null when '' then true when 0 then false else '=)' end |
+-------------------------------------------------------------+
| =)                                                          |
+-------------------------------------------------------------+
1 row in set (0.00 sec)

NULLIF and IFNULL

• nullif(expr1,expr2): Tests whether expr1 and expr2 are equal
• ifnull(expr1,expr2): returns the first non-null expr

mysql> select nullif('A','A'),nullif('A','B');
+-----------------+-----------------+
| nullif('A','A') | nullif('A','B') |
+-----------------+-----------------+
|            NULL | A               |
+-----------------+-----------------+
1 row in set (0.01 sec)

mysql> select ifnull('A','B'),ifnull(17/0,'A');
+-----------------+------------------+
| ifnull('A','B') | ifnull(17/0,'A') |
+-----------------+------------------+
| A               | A                |
+-----------------+------------------+
1 row in set, 1 warning (0.01 sec)
	

IF function

-if(cond,then_return,else_return)

mysql> select if('ABC'>'aBC','+++','---');
+-----------------------------+
| if('ABC'>'aBC','+++','---') |
+-----------------------------+
| ---                         |
+-----------------------------+
1 row in set (0.00 sec)

expression pushdown

• When TiDB reads data from TiKV, TiDB will try to push down some expression operations to TiKV, thereby reducing the amount of data transmission and the computing pressure of a single TiDB node

• Logical operators: and ,&&,or,||,not,!

• Comparison operators: < <= = != <> > >= <=> in , is null,like,is true, is false, coalesce()

• Numerical operations: + - * / abs() ceil() ceiling() floor()

• Control flow operation: case if() ifnull()

• Date operation: date_format()

• …

• Determine whether the operation supports expression pushdown by observing the task.cop[tikv] of the task field in the execution plan

# 出现cop[tikv] 就表示下推
mysql> explain select name,mass from universe.planets where name like 'M%';
+-------------------------------+---------+-----------+---------------------------------+-------------------------------------------------+
| id                            | estRows | task      | access object                   | operator info                                   |
+-------------------------------+---------+-----------+---------------------------------+-------------------------------------------------+
| IndexLookUp_10                | 0.25    | root      |                                 |                                                 |
| ├─IndexRangeScan_8(Build)     | 0.25    | cop[tikv] | table:planets, index:name(name) | range:["M","N"), keep order:false, stats:pseudo |
| └─TableRowIDScan_9(Probe)     | 0.25    | cop[tikv] | table:planets                   | keep order:false, stats:pseudo                  |
+-------------------------------+---------+-----------+---------------------------------+-------------------------------------------------+
3 rows in set (0.00 sec)