In order to achieve real-time monitoring of the database, change table data is difficult to capture, is the problem we are currently experiencing, but these days I noticed a reference solution, can solve this problem, CDC.
The concept works:
CDC also known as change data capture (Change Data Capture), will open the insert data into the log table when the source table in the cdc insert INSERT, UPDATE, and DELETE DELETE update activities. By CDC
The change data capture capture process to change tables, query functions provided by cdc, we can capture this part of the data.
More 1.SQL server 2008 version of the Enterprise Edition, Developer Edition and Evaluation Edition are available;
2. The need to open agency services (job).
3.CDC need additional disk space outside the business library.
4.CDC table requires a primary key or a unique key.
Formal settings:
First, open proxy service:
Windows environment, can be found directly in SQL server agent service, set to open automatically.
Linux environment:
sudo /opt/mssql/bin/mssql-conf set sqlagent.enabled true
sudo docker restart <container ID>
Second, create a test environment:
1.
/******* Step1:创建示例数据库*******/ USE master GO IF EXISTS(SELECT name FROM sys.databases WHERE name = 'CDC_DB') DROP DATABASE CDC_DB GO CREATE DATABASE CDC_DB GO
2.
/ ******* Step2: Open the database ******* CDC / - CDC database to see whether to enable the SELECT name, the FROM is_cdc_enabled the sys.databases the WHERE name = 'CDC_DB' - enabled database CDC the USE CDC_DB GO EXECUTE sys.sp_cdc_enable_db; GO
- checking is enabled successful SELECT is_cdc_enabled, CASE WHEN is_cdc_enabled = 0 THEN 'CDC function disabling' ELSE 'CDC enabled' END description FROM sys.databases WHERE NAME = 'CDC_DB'
3.
/ ******* Step3: Enable capture on a table changes ******* / - Create test table the USE GO CDC_DB the CREATE TABLE [dbo] [Department] ([the DepartmentID] [smallint] IDENTITY (1. , 1) NOT NULL, [Name ] [nvarchar] (200) NULL, [GroupName] [nvarchar] (50) NOT NULL,
[ModifiedDate] [datetime] NOT NULL, [AddName] [nvarchar](120) NULL, CONSTRAINT [PK_Department_DepartmentID] PRIMARY KEY CLUSTERED ( [DepartmentID] ASC ) ON [PRIMARY] ) ON [PRIMARY] GO
- Enable table captures EXEC sys.sp_cdc_enable_table @ source_schema = 'dbo' , @source_name = 'Department', @role_name = N'cdc_Admin ', @capture_instance = DEFAULT,
@supports_net_changes = 1, @index_name = NULL, @captured_column_list = NULL, @filegroup_name = DEFAULT
- Check if successful SELECT name, is_tracked_by_cdc, CASE WHEN is_tracked_by_cdc = 0 THEN 'CDC function disabling' ELSE 'CDC enabled' END description FROM sys.tables WHERE OBJECT_ID = OBJECT_ID ( ' dbo.Department')
/ ******* Step4: Change capture test DML ******* / - inserting test data INSERT INTO dbo.Department (Name, GroupName, ModifiedDate) VALUES ( 'Marketing', 'Sales and Marketing' , GETDATE ()) - data test update the uPDATE dbo.Department the SET the Name = 'Marketing Group', the ModifiedDate = GETDATE () the WHERE the Name = 'Marketing' - delete test data dELETE FROM dbo.Department WHERE Name = 'Marketing Group' - data capture query SELECT * FROM cdc.dbo_Department_CT
For insert / delete operations, there will be corresponding to one row, and for the update, there will be two rows. __ $ operation column: 1 = delete, insert 2 =, 3 = update (old value), update 4 = (new value);
Figure:
