The use of fdw in postgresql database causes the number of queries to be too slow

In the past few days, the management database found that there are several tables built using fdw. Due to the large amount of data, the query speed has changed from 20 seconds to more than 1000 seconds. Because it is not in the same database, only fdw can be used to connect, because the original table The amount of data is too large or a normal table

Cause Analysis

It is suspected that the problem of fdw transmission is slow due to the large number of original tables.

solution

Create a view to restrict it.
Discuss with the business side and found that some tables do not need all the data, only the data of the last few days. I decided to create a view in the mode where the original table was located, and only fetch the data of the last few days.

CREATE VIEW sss as
 SELECT id
.........................................
   FROM ‘表名’
  WHERE (commit_date >= ((now())::date + '-10 days'::interval))

Modify the specified table when re-creating the external table of fdw (from the original table to the view)

--创建外部服务器
-- 括号里的三个参数，分别是timescaledb的ip、端口和数据库名称

CREATE SERVER baba FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host '0.0.0.0', port '5432', dbname 'hrmw');
--创建用户映射
-- 括号里的两个参数，分别是timescaledb数据库的用户名、密码
create user mapping for postgres server baba options(user 'postgres', password '数据库密码');

CREATE FOREIGN TABLE "public"."表名" (
  ...........  --字段
)
SERVER "baba" -- 外部链接名
OPTIONS ("schema_name" '模式名', "table_name" 'v_base_promise') --指定模式和视图
;

After testing, the modified query only takes about 10 seconds

before fixing

Insert picture description here

After the modification, it was
Insert picture description here
originally used fdw to transmit all to do the filtering, now it is to use the view to fetch the required data and then use fdw to transmit

Second, change the ordinary table to the partition table

Change the ordinary table to partition. There are many introductions on the Internet. I will not write here. I will modify the link pg database to a partitioned table. Changing
the ordinary table to partition does improve efficiency.

(To be continued)