Logstash incrementally synchronizes postgreSQL, but the data cannot be refreshed

1. Scene

logstash database configuration, refreshed every minute

input {
    
    
 #bms
 jdbc {
    
    
    jdbc_driver_library => "../db/postgresql-9.4-1206-jdbc42.jar"
    jdbc_driver_class => "org.postgresql.Driver"
    jdbc_connection_string => "jdbc:postgresql://xx.xx.xx.xx:5432/portal_test?currentSchema=public"
    jdbc_user => "postgres"
    jdbc_password => "xx"
    jdbc_paging_enabled => true
    jdbc_page_size => "1000"
    #timezone
    jdbc_default_timezone => "Asia/Shanghai"
    # 每分钟同步一次数据
    schedule => "* * * * *"
    clean_run => true
    statement_filepath => "../db/bms.sql"
    #use_column_value => true
    type => "bms"
  }
}


filter {
    
    
    json {
    
    
        source => "message"
        remove_field => ["message"]
    }
}


output {
    
    
    if[type] == "bms" {
    
    
 elasticsearch {
    
    
        hosts => ["xx.xx.xx.xx:9200"]
        index => "%{type}"
        document_id => "%{id}"
        document_type =>"%{type}"}

}
    stdout {
    
    
        codec => json_lines
    }
}

Incremental synchronization script

SELECT
	bms_meeting_detail.ID AS ID,
	bms_meeting_detail.meeting_type AS title,
	bms_meeting.description AS content,
	bms_meeting.meeting_creator AS meetingcreator,
	bms_meeting_detail.meeting_start_time AS createtime,
	bms_meeting_detail.meeting_end_time AS updatetime,
	bms_meeting_detail.modifier AS modifier,
	bms_meeting_detail.participants AS participants,
	bms_meeting.is_delete AS isdelete,
	bms_meeting_detail.meeting_owner AS meetingowner,
	bms_meeting_detail.meeting_conclusion AS meetingconclusion,
	bms_meeting_detail.is_published AS ispublished,
	A.documentcontent AS documentcontent 
FROM
    bms_meeting
	LEFT JOIN bms_meeting_detail ON bms_meeting."id" = bms_meeting_detail.meeting_id
	LEFT JOIN (
	SELECT
		bms_meeting_links.meeting_detail_id AS id,
		GROUP_CONCAT ( bms_meeting_links.link_name ) AS documentcontent 
	FROM
		bms_meeting_links
		LEFT JOIN bms_meeting_detail ON bms_meeting_detail."id" = bms_meeting_links.meeting_detail_id 
	GROUP BY
		bms_meeting_links.meeting_detail_id 
	) AS A ON  A."id" = bms_meeting_detail."id" 
WHERE bms_meeting.is_delete =0 AND bms_meeting_detail.create_time > :sql_last_value 

2. Results and reasons

Results After
starting logstash, through page operations, and then real-time viewing logstash logs, it was found that the refreshed data could not be captured.
Reason The
reason is: bms_meeting_detail.create_time> :sql_last_value The
research found that the time of :sql_last_value is the server time, and the data synchronized each time is the data within 1 minute of that time point; it is
found that the create_time of the database is not within 1 minute.
Then, immediately verify that the operation generates new data once and compares it with the system time. It is found that the create_time recorded in the database is 3 minutes slower.

3. Solution

Synchronize linux system time and database time.

yum install ntp
ntpdate time.nist.gov
date

Reference:
https://zhidao.baidu.com/question/585857949.html?fr=iks&word=linux%D0%A3%D5%FD%B7%FE%CE%F1%C6%F7%CF%B5%CD% B3%CA%B1%BC%E4&ie=gbk