Data warehouse 001- full backup of time zipper

Demo in mysql

Full backup of the zipper in mysql demo time:

1. Demonstrate the first and second full backups.

1. The business system table exists first.

//进入test数据库。
use test
//创建业务表user_data，并赋予最初的业务数据。
create table user_data(
userid int,
ustate varchar(4)
);
insert into user_data values(1,'1');
insert into user_data values(2,'2');
insert into user_data values(3,'3');
insert into user_data values(4,'4');
insert into user_data values(5,'5');

2. The first full backup: use the temporary table bak to extract the data in the business table, add two field information (start time and

End time), and then bak directly inserts all the data into the history table his.

//开始时间的业务数据的当天，这里我们假设时间为2007-01，并将结束时间取为最大。
//在这里，我们的记录的begin_date和end_date都是varchar类型。
create table bak as select u.*,'200701' begin_date,'299901' end_date from user_data u ;
//将bak表的数据插入历史表中。
create table his as select * from bak;

3. View the data after the first full backup

Regardless of whether it is a full backup or an incremental backup, the first backup is to insert all business data into the history table after adding time information, and the next second full backup is the focus.

4. We assume that in 2008-01, the data in the business table will be modified as follows:

update user_data set ustate = 2 where userid = 1;
update user_data set ustate = 4 where userid = 3;
update user_data set ustate = 6 where userid = 5;

The revised business table data is as follows:

Here comes the point. One of our core steps is to compare "yesterday" data with "today" data.

5. When the first full backup is made, the temporary table bak already stores the business data and time information of "yesterday". So we need to insert the data in the bak table into the old_temp table, and insert the "today" business data into the new_temp table with the time field added.

create table old_temp as select * from bak;

create table new_temp as select u.*,'200801' begin_date,'299901' end_date from user_data u ;

At this point, you can check the data of the new table and the old table separately, first look at the data of the old table.

Check the data in the new table.

6. Using the time zipper algorithm, new-old gets the newly added data, and old-new gets the changed data. It should be noted here that the new or modified data we are talking about does not mean the addition or modification of the business table, but the addition or modification we will make to the historical table.

//查看新增的数据
select a.* from new_temp a left join old_temp b on a.userid = b.userid and a.ustate = b.ustate where  b.userid is null;

//查看修改的数据
select b.* from new_temp a right join old_temp b on a.userid = b.userid and a.ustate = b.ustate where  a.userid is null;

7. There is a core step, which is to insert the data of the temporary table bak into the old table, and the business table adds fields into the new table. After this step, the information of the new table at this time needs to be inserted into the temporary table bak. Of course, all data in the target table must be cleared before inserting data into the target table. The bak table at this time is the "yesterday" data of the next full backup.

//清空bak表的数据
truncate table bak;
//将new表的数据全部插入bak中
insert into bak select * from new_temp;

View bak table data.

8. The last step of the second full backup is here, which is to insert the newly added data into the history table and modify the modified data in the history table.

//将新增的数据插入历史表
insert into his select a.* from new_temp a left join old_temp b on a.userid = b.userid and a.ustate = b.ustate 
where b.userid is null;

View the data of his table

//将修改的数据在历史表中修改
update his t1 inner join (select b.* from new_temp a 
right join old_temp b on a.userid = b.userid and a.ustate = b.ustate 
where a.userid is null) t2 on t1.userid = t2.userid and 
t1.ustate = t2.ustate set t1.end_date = '200801';

View the data of his table

Successfully back up the newly added or modified data to the history table.

2. Next, start the third full backup.

1. First, let's look at the business data of 2008-01, which is the business data of "yesterday".

Now we assume that the user information with userid 1 is changed to usdate=3, and the user information with userid 4 is changed to 8.

update user_data set ustate = 3 where userid = 1;
update user_data set ustate = 8 where userid = 4;

View business data for "today".

2. In the second full backup, it shows that the data of the temporary table bak is the "yesterday" data at the third full backup. Here we can check it again.

Then we empty the data of the old table and insert the data of "yesterday" into the old table, then empty the data of the new table and insert the time field of today's business data into the new table.

//清空old表和new表的数据
truncate table new_temp;
truncate table old_temp;
//将bak表的数据插入old表
insert into old_temp select * from bak;
//将今天的业务数据添加时间字段插入new表
insert into new_temp select u.*,'200802' begin_date,'299901' end_date from user_data u ;

3. Don't forget to empty the bak table data and insert the new table data into the bak table, let the bak table save the "yesterday" data of the next full backup. The new data and modified data are calculated by the time zipper algorithm again.

//清空bak表的数据
truncate table bak;
insert into bak select * from new_temp;

View newly added data

//查看新增的数据
select a.* from new_temp a left join old_temp b on a.userid = b.userid and a.ustate = b.ustate where  b.userid is null;

View modified data

//查看修改的数据
select b.* from new_temp a right join old_temp b on a.userid = b.userid and a.ustate = b.ustate where  a.userid is null;

4. Insert the new data into the history table, and modify the modified data to the history table.

//将新增的数据插入历史表
insert into his select a.* from new_temp a left join old_temp b on a.userid = b.userid and a.ustate = b.ustate 
where b.userid is null;

//将修改的数据在历史表中修改
update his t1 inner join (select b.* from new_temp a 
right join old_temp b on a.userid = b.userid and a.ustate = b.ustate 
where a.userid is null) t2 on t1.userid = t2.userid and 
t1.ustate = t2.ustate set t1.end_date = '200802';

The third full backup was successful.