One: background
1. Tell a story
I recently optimized a friend SQL 慢语句 and spent some time tuning it. Unfortunately, the non-source code of SQLSERVER is not open, and it is not so smooth to play. However, from this experience, I think that a major task for next year is to study it carefully and strive to be in the SQL Server. Do some achievements in SQLSERVER performance optimization, haha! I personally think that if you want to study SQLSERVER in depth, you have to start with its storage engine , and when it comes to the storage engine, you have to 数据页 start with the core. After all, mdf is 数据页 created by spelling. Of course, you can point out if you understand it wrong.
Two: Understand the data page
1. What is a data page
Generally speaking, the efficient management of large resources or data will be divided according to a certain granularity. For example, the memory management of Windows is divided according to the number. The implication is that the management 内存页 (4k) of mdf by SQL Server is also 数据页 （8k) divided according to the number. Draw a picture That's about it.
So how to verify this conclusion? Just now I said that the data is all on the data page. Let's get some data and extract it on the specified data page. Here we use SQLServer 2019 .
CREATE DATABASE MyTestDB
GO
USE MyTestDB;
GO
IF OBJECT_ID('person') IS NOT NULL
DROP TABLE person;
CREATE TABLE person
(
id INT IDENTITY,
name CHAR(5)
);
GO
INSERT INTO dbo.person( name ) VALUES ('john');
INSERT INTO dbo.person( name ) VALUES ('mary');
2. Find the data page where the data is located
Just now the picture also said that mdf is 数据页 made up of countless numbers, so how to find the data page where the person table is located? In fact, Microsoft provides a dbcc ind command to help us find out, and remember to start 3604 marking, so that the output is displayed on the console instead of the default error log. How to use this command, you can check the official document.
DBCC TRACEON(3604)
DBCC IND(MyTestDB,person, -1)
From the output, there are two records. The first one is PagePID=41 the IAM data page, which PagePID=280 is the number of the data page where the record of our person table is located 0n280 * 0n8192 . 0x00230000 .
0:090> ? 0n280 * 0n8192
Evaluate expression: 2293760 = 00000000`00230000
Is that right? You can use WinHex to verify it. In order not to be occupied by the process, go MyTestDB offline first, and finally remember to go online again.
ALTER DATABASE MyTestDB SET OFFLINE
ALTER DATABASE MyTestDB SET ONLINE
From the perspective of WinHex, it is indeed on 0x00230000 the offset data page.
3. How to see the data page from memory
The data page we saw just now is only on the physical hard disk, but the logical relationship between the data page and the data page must be carried in the memory with a certain data structure. Next, let’s see where this is mapped to the SQLSERVER process 数据页 memory Woolen cloth? Microsoft provides DBCC PAGE a command to view detailed information of a specified data page.
DBCC TRACEON(3604)
DBCC PAGE(MyTestDB,1,280,2)
The output is as follows:
DBCC 执行完毕。如果 DBCC 输出了错误信息，请与系统管理员联系。
PAGE: (1:280)
BUFFER:
BUF @0x000002B41104F480
bpage = 0x000002B3F0632000 bPmmpage = 0x0000000000000000 bsort_r_nextbP = 0x000002B41104F3D0
bsort_r_prevbP = 0x0000000000000000 bhash = 0x0000000000000000 bpageno = (1:280)
bpart = 1 ckptGen = 0x0000000000000000 bDirtyRefCount = 0
bstat = 0x9 breferences = 0 berrcode = 0
bUse1 = 12454 bstat2 = 0x0 blog = 0x15ab215a
bsampleCount = 0 bIoCount = 0 resPoolId = 0
bcputicks = 0 bReadMicroSec = 182 bDirtyContext = 0x0000000000000000
bDbPageBroker = 0x0000000000000000 bdbid = 10 bpru = 0x000002B3FA708040
PAGE HEADER:
Page @0x000002B3F0632000
m_pageId = (1:280) m_headerVersion = 1 m_type = 1
m_typeFlagBits = 0x0 m_level = 0 m_flagBits = 0x8200
m_objId (AllocUnitId.idObj) = 179 m_indexId (AllocUnitId.idInd) = 256
Metadata: AllocUnitId = 72057594049658880
Metadata: PartitionId = 72057594043170816 Metadata: IndexId = 0
Metadata: ObjectId = 581577110 m_prevPage = (0:0) m_nextPage = (0:0)
pminlen = 13 m_slotCnt = 2 m_freeCnt = 8060
m_freeData = 128 m_reservedCnt = 0 m_lsn = (37:584:3)
m_xactReserved = 0 m_xdesId = (0:0) m_ghostRecCnt = 0
m_tornBits = -116446693 DB Frag ID = 1
Allocation Status
GAM (1:2) = ALLOCATED SGAM (1:3) = NOT ALLOCATED PFS (1:1) = 0x41 ALLOCATED 50_PCT_FULL
DIFF (1:6) = CHANGED ML (1:7) = NOT MIN_LOGGED
DATA:
Memory Dump @0x000000F840DF8000
000000F840DF8000: 01010000 00820001 00000000 00000d00 00000000 ....................
000000F840DF8014: 00000200 b3000000 7c1f8000 18010000 01000000 ........|...........
000000F840DF8028: 25000000 48020000 03000000 00000000 00000000 %...H...............
000000F840DF803C: 1b2a0ff9 00000000 00000000 00000000 00000000 .*..................
000000F840DF8050: 00000000 00000000 00000000 00000000 10000d00 ....................
000000F840DF8064: 01000000 6a6f686e 20020000 10000d00 02000000 ....john ...........
000000F840DF8078: 6d617279 20020000 00002121 21212121 21212121 mary .....!!!!!!!!!!
000000F840DF808C: 21212121 21212121 21212121 21212121 21212121 !!!!!!!!!!!!!!!!!!!!
000000F840DF80A0: 21212121 21212121 21212121 21212121 21212121 !!!!!!!!!!!!!!!!!!!!
...
000000F840DF9FF4: 21212121 21212121 70006000 !!!!!!!!p.`.
OFFSET TABLE:
Row - Offset
1 (0x1) - 112 (0x70)
0 (0x0) - 96 (0x60)
DBCC 执行完毕。如果 DBCC 输出了错误信息，请与系统管理员联系。
Completion time: 2022-12-30T17:48:20.5466040+08:00
Memory Dump As can be seen from the section section above , the data is 000000F840DF8064 ~ 000000F840DF8078 within the range of the process memory. What I want to complain here is the memory address according to it 大端布局 . It seems very uncomfortable. You can use
windbg 小端布局 is used to display.
0:116> dp 000000F840DF8064
000000f8`40df8064 6e686f6a`00000001 000d0010`00000220
000000f8`40df8074 7972616d`00000002 21210000`00000220
000000f8`40df8084 21212121`21212121 21212121`21212121
000000f8`40df8094 21212121`21212121 21212121`21212121
000000f8`40df80a4 21212121`21212121 21212121`21212121
000000f8`40df80b4 21212121`21212121 21212121`21212121
000000f8`40df80c4 21212121`21212121 21212121`21212121
000000f8`40df80d4 21212121`21212121 21212121`21212121
4. sql request source code research
Friends who like to play windbg must want to have assembly-level insight into sqlserver, for example, what is the execution flow of sql request in sqlserver? In fact, it is very simple, we can handle it like this, use john the memory address of ba to set a hardware breakpoint, that is , then execute a statement ba r4 000000f840df8064+0x4 on SSMS , because it will hit naturally if it is to be extracted.SELECT * FROM personjohn
0:104> ba r4 000000f840df8064+0x4
0:104> g
Breakpoint 0 hit
sqlmin!BTreeMgr::GetHPageIdWithKey+0x4a:
00007ff8`d4ea121a 48894c2478 mov qword ptr [rsp+78h],rcx ss:000000f8`45278028=0000024800000025
0:102> k
# Child-SP RetAddr Call Site
00 000000f8`45277fb0 00007ff8`d4ea0b59 sqlmin!BTreeMgr::GetHPageIdWithKey+0x4a
01 000000f8`45278450 00007ff8`d4ea08b7 sqlmin!IndexPageManager::GetPageWithKey+0x119
02 000000f8`45278d20 00007ff8`d4eb22d1 sqlmin!GetRowForKeyValue+0x203
03 000000f8`45279880 00007ff8`d4eb2a39 sqlmin!IndexDataSetSession::GetRowByKeyValue+0x141
04 000000f8`45279a70 00007ff8`d4eb279b sqlmin!IndexDataSetSession::FetchRowByKeyValueInternal+0x230
05 000000f8`45279ed0 00007ff8`d4eb2883 sqlmin!RowsetNewSS::FetchRowByKeyValueInternal+0x437
06 000000f8`4527a000 00007ff8`d4eaadab sqlmin!RowsetNewSS::FetchRowByKeyValue+0x96
07 000000f8`4527a050 00007ff8`d4f93d60 sqlmin!CMEDScan::StartSearch+0x4f8
08 000000f8`4527a170 00007ff8`d4f93f3a sqlmin!CMEDCatYukonObject::GetTemporalCurrentTableId+0x10e
09 000000f8`4527a380 00007ff8`d801f0d1 sqlmin!CMEDProxyRelation::GetTemporalCurrentTableId+0x7a
0a 000000f8`4527a3c0 00007ff8`d801dfb8 sqllang!CAlgTableMetadata::FPartialBind+0xb58
0b 000000f8`4527a580 00007ff8`d80394b3 sqllang!CAlgTableMetadata::Bind+0x317
0c 000000f8`4527a620 00007ff8`d800415d sqllang!CRelOp_Get::BindTree+0x78f
0d 000000f8`4527a890 00007ff8`d80418a1 sqllang!COptExpr::BindTree+0x85
0e 000000f8`4527a8c0 00007ff8`d800415d sqllang!CRelOp_FromList::BindTree+0x31
0f 000000f8`4527a920 00007ff8`d802c2a3 sqllang!COptExpr::BindTree+0x85
10 000000f8`4527a950 00007ff8`d800415d sqllang!CRelOp_QuerySpec::BindTree+0x2e8
11 000000f8`4527aa60 00007ff8`d80042dd sqllang!COptExpr::BindTree+0x85
12 000000f8`4527aa90 00007ff8`d800415d sqllang!CRelOp_SelectQuery::BindTree+0x489
13 000000f8`4527ab80 00007ff8`d8003f23 sqllang!COptExpr::BindTree+0x85
14 000000f8`4527abb0 00007ff8`d8004e47 sqllang!CRelOp_Query::FAlgebrizeQuery+0x4bd
15 000000f8`4527ae30 00007ff8`d7ff5576 sqllang!CProchdr::FNormQuery+0x8f
16 000000f8`4527ae70 00007ff8`d7ff4a79 sqllang!CProchdr::FNormalizeStep+0x5bd
17 000000f8`4527b4b0 00007ff8`d7ff5124 sqllang!CSQLSource::FCompile+0xea5
18 000000f8`4527e1b0 00007ff8`d7e659c3 sqllang!CSQLSource::FCompWrapper+0xcb
19 000000f8`4527e280 00007ff8`d7e6387a sqllang!CSQLSource::Transform+0x721
1a 000000f8`4527e3e0 00007ff8`d7e6e67b sqllang!CSQLSource::Execute+0x4fa
1b 000000f8`4527e8c0 00007ff8`d7e6d815 sqllang!process_request+0xca6
1c 000000f8`4527efc0 00007ff8`d7e6d5ef sqllang!process_commands_internal+0x4b7
1d 000000f8`4527f0f0 00007ff8`d4096523 sqllang!process_messages+0x1d6
1e 000000f8`4527f2d0 00007ff8`d4096e6d sqldk!SOS_Task::Param::Execute+0x232
1f 000000f8`4527f8d0 00007ff8`d4096c75 sqldk!SOS_Scheduler::RunTask+0xa5
20 000000f8`4527f940 00007ff8`d40bb160 sqldk!SOS_Scheduler::ProcessTasks+0x39d
21 000000f8`4527fa60 00007ff8`d40baa5b sqldk!SchedulerManager::WorkerEntryPoint+0x2a1
22 000000f8`4527fb30 00007ff8`d40bafa4 sqldk!SystemThreadDispatcher::ProcessWorker+0x3ed
23 000000f8`4527fe30 00007ff8`f6d86fd4 sqldk!SchedulerManager::ThreadEntryPoint+0x3b5
24 000000f8`4527ff20 00007ff8`f865cec1 KERNEL32!BaseThreadInitThunk+0x14
25 000000f8`4527ff50 00000000`00000000 ntdll!RtlUserThreadStart+0x21
From the perspective of the thread stack, there are various core elements such as the Scheduler, Task, and command analyzer, query optimizer, and query executor at the core of SQL Server. Let's study it slowly later.
Three: Summary
A deep understanding of the data and the layout of the index on the data page can fundamentally help us understand how to reduce the flow of requests on the data page, reduce logical reads, and thus improve the query performance of SQL.