Search for a substring within the specified main string,
For example, to search for the substring "rqqw" in the main string "asrqqwex",
There are three commonly used methods:
1. Violent search method
The process is simple, compare the first character of the substring with the first string of the main string.
If they are the same, continue to compare the next character. If all the substrings match, it will be searched.
If a certain character is different, then the main string restarts the comparison from the second character, repeats the above operation, and so on.
as follows:
asrqqwex
rqqw
asrqqwex
rqqw
asrqqwex
rqqw
The only advantage of this method is that the process is simple and easy to understand. The disadvantage is that the efficiency is very low. The length of the main string is n, the length of the substring is m,
Then the time complexity is O(m*n).
code show as below:
int ViolentSearch(const char* pat, const char* txt)
{
int n = strlen(txt);
int m = strlen(pat);
for (int i = 0; i <= n - m; i++)//循环主串-子串+1次
{
int j;
for (j = 0; j < m; j++)//循环子串次
{
if (pat[j] != txt[i + j])
{
break;//任意一个不匹配 重新匹配
}
}
if (j == m) return i;//全部匹配返回位置
}
return -1;
}This method is easy to understand, but it is very inefficient like this.
2. KMP algorithm
The violent search method is equivalent to exhaustion, so the efficiency is very low.
For example: characters that are not in the substring appear in the main string, which is equivalent to truncation, and the comparison of a certain segment is meaningless.
Another example: compare "aaab" and "aaaaaaaab",
Start the comparison from the first character, and compare to the fourth character to find the difference.
Then start to compare the 234 bytes that have already been compared from the second character, which is obviously redundant.
Then there are more excellent KMP algorithm and BM algorithm sundy algorithm to improve efficiency. The essence is to exchange space for time, and create an array in advance to record substring information to help us search for strings faster.
Today we will mainly study the ideas, processes and codes of these two algorithms.
The core of the KMP algorithm is an array called next,
The next array is only related to the substring. It is an array with the same length as the substring calculated by the substring. To help us better search for the substring in the main string.
Forget about algorithms.
Let's take a look at the meaning of the next array members.
3. Calculation of next array
For example, the substring char a[] = "abadxyzababmn";
Before calculating the next array, let's first understand what is the prefix and suffix
A prefix is a substring starting from the first character and having a length less than the total string
We count the prefixes in front of "n"
Here is "a" "ab" "aba" "abad" "abadx" "abadxy" "abadxyz" ... "abadxyzabab" without "abadxyzababm" because it is all the strings in front of n
The suffix is a substring starting from the last character and having a length less than the total string
Here is "m" "bm" "abm" "babm" "ababm" "zababm" ... "badxyzababm" does not have "abadxyzababm" because it is all the strings in front of n
Obviously, the prefixes and suffixes here are not the same, we count 0
There is nothing the same here, it is just the result of the calculation of the position of n, it does not mean that there is no previous
For example, take "abad" in front of the substring
We count the prefixes and suffixes in front of the "d"
The prefix is "a" "ab"
The suffix is "a" "ba"
Obviously there is a common prefix and suffix "a" The length is 1 and we count 1
alright,
Knowing the prefixes and suffixes, we can directly see what is in the next array
The correspondence between substrings and next array members is as follows:
a b a d x y z a b a b m n
-1 0 0 1 0 0 0 0 1 2 3 2 0
Find whether the n and d we just mentioned correspond to 0 and 1? Then the problem is very simple
next[0] is fixed at -1. In fact, it can be any special value less than 0 to indicate that the comparison will start again. You will know the code later.
next[i] (such as next[3]) means: the longest equal prefix and suffix of the string (here is "aba") before its corresponding string member a[i] (here is 'd') The length of (here 1).
Do you understand the meaning of the next array
If you don't understand.
Let's say it more generally: next[i] means that there are several consecutive characters in front of a[i] that are the same as the first character of the substring. And it is the longest.
look at the picture
There is a character in front of it that is the same as the first character of the substring, so it is 1
The first three characters are the same as the first character of the substring, so it is 3
Still don't understand? Let's explain the example in more detail
Here next[10] corresponds to a[10] which are 3 and 'b' respectively
There is "aba" in front of b, which is the same as the first character of the substring. And it is the longest. So next[10] is 3
Note again, it cannot be said that "abadxyzaba" in front of b is the same as "abadxyzaba" in front of the substring, and cannot be equal to the maximum length in front
Otherwise, the next array is meaningless
The result of the next array of any substring will be -1 1 2 3 4 5 6 7 8 ...., right?
if you think you understand
Let's come up with a few topics to consolidate
Calculate it yourself. In fact, the students have already discovered that next[0] = -1 next[1] = 0 is inevitable.
Topic 1:
a b a b c
-1 0 0 1 2
Topic 2:
a b a a a b b c a b c
-1 0 0 1 1 1 2 0 0 1 2
Topic 3:
a a a a a a a a b
-1 0 1 2 3 4 5 6 7
Will the next array be calculated manually?
Then we use C++ code to initialize the next array
4. C++ code calculates next array
You may say that traversing all previous prefixes and suffixes from the current character position,
Then calculate the maximum length.
Of course, this is no problem, but it is less efficient. The algorithm itself is the pursuit of efficiency, and of course exhaustion should be avoided.
In order to facilitate everyone's understanding, some Chinese variables are used, and they will be optimized and changed back later. Thank you for your understanding.
void getNext(const char* t, vector<int>& next)
{
next[0] = -777;//设置一个小于0的特殊值
next[1] = 0;//固定的0
int i = 1;//每次为next数组赋值前,i++,所以是从第3个成员开始
int 匹配字符数 = 0;//初始化0 从第一位开始匹配
while (i < strlen(t) - 1)//循环次数 子串长度-2 因为前2个成员已经初始化完毕了
{
if (匹配字符数 == -777)//看完下面的代码 和 图1 图2 再看这里
//等于这个值,说明什么? 这个值只在next[0]中出现
//任鸟飞逆向
//说明刚才 匹配字符数 = next[匹配字符数] 赋值前的 匹配字符数为0. 这就是设置特殊值的意义.
//既然匹配值为0了,那么就要从头匹配了
{
++i;
匹配字符数 += 777;//从头匹配 初始化0 从第一位开始匹配
next[i] = 匹配字符数;//当前没有匹配到 写入0
}
if (t[i] == t[匹配字符数])//下面图1:匹配到 同时+1 继续匹配下一个 同时把当前匹配的数量写入next数组
{
++i;
++匹配字符数;
next[i] = 匹配字符数;
}
if (t[i] != t[匹配字符数])//下面图2:匹配不到是唯一的难点,下面会具体说明为什么是 匹配字符数 = next[匹配字符数];
{
匹配字符数 = next[匹配字符数];
}
}
}Figure 1: (Red represents match) Any position is matched and +1 is continued to match the next one, and the current matching amount is written to next[i+1]
If you continue to match, it is a repeated operation, continue to write 2, 3, 4, 5...
Figure 2: How to deal with a match to a position that does not match?
I believe that everyone here will fully understand the calculation process and method of the next array.
In fact, when the KMP algorithm is looking for a string, the logic used is exactly the same as this.
If you see this, congratulations, you have mastered 80% of this algorithm.
code final optimization
The most concise code that becomes the standard (of course, I will give you the most concise code, I believe many people will be confused)
void getNext(const char* pat,vector<int>& next)
{
next[0] = -1;
int i = 0, j = -1;
while (i < strlen(pat) - 1) {
if (j == -1 || pat[i] == pat[j]) {
++i;
++j;
next[i] = j;
}
else {
j = next[j];
}
}
}Now take the standard KMP to calculate the code of the next array and compare it with the code we wrote ourselves, can we fully understand it?
Because the code is too simplistic, it is really difficult to understand it directly.
5. KMP algorithm process and code
Get the calculation of the next array
Let's follow this logic and manually search for a string.
For example, search for "abab c" in "abaabababc a"
"a b a b c"
The next array is easily calculated as
-1 0 0 1 2
Take a look at the complete matching process:
a b a a b a b a b c a
a b a b c
-1 0 0 1 2
第四位不匹配 next数组里面是1 说明前面有一个字符和前缀相同
也就是说我们第一位不用比较了,从第二位开始和主串此时的第四位继续匹配就完事了
是不是跟计算next数组的时候一模一样啊
还是 j = next[j] 这就是kmp的核心思想.
我们移动子串方便观察
a b a a b a b a b c a
a b a b c
-1 0 0 1 2 0
第二位不匹配 里面是0 j = next[0] = -1
J = -1 要从头匹配 是不是也和next数组算法一样啊?
a b a a b a b a b c a
a b a b c
-1 0 0 1 2 0
第五位不匹配 里面是2 同理
a b a a b a b a b c a
a b a b c
-1 0 0 1 2 0
匹配成功 退出
任鸟飞逆向
我们发现KMP算法 不走回头路 ,不会重复扫描So the algorithm and the next array can be said to be almost exactly the same
We can think of the prefix in the next array algorithm as a substring for comparison, and the substring in the next array algorithm as the main string
Is there no difference?
Then needless to say, the direct code:
int search(const char* pat, const char* text ,vector<int> next)
{
int i = 0;
int j = 0;
while (i < strlen(text))
{
if (j == -1 || text[i] == pat[j])
{
++i;
++j;
if (j == strlen(pat))
{
return i -j;
}
}
else
{
j = next[j];
}
}
}The above two methods are not absolute, who is better and who is not.
When the substring and the main string are not large, the violent search method is very fast. Because KMP needs to do a lot of initialization and comparison processing
With the increase of the main string and substring, KMP becomes more and more obvious, especially when we search for signatures in a memory as large as 4GB
The efficiency of the KMP algorithm is about twice the length of the feature code of the brute force search method.
6. BM algorithm
The BM algorithm is also an efficient string matching algorithm.
I think his design is simpler and more efficient for the task of locating signatures.
First of all, he is also divided into two steps
The first step is to generate a bad word table, similar to KMP's next
The second step starts the matching search
Let's take a look at how his bad word list is generated and what is its function
In the process of our search, the bad word table can help us judge whether there is this character in the substring, and if not, it will directly skip a large section of comparison.
At the same time, the bad word table can help us record the last position of a character.
if you don't understand
Let's look at the code directly, it's very simple:
void getBadCharTable(const char *pat, int len, int bc[256])
{
memset(bc, -1, sizeof(int) * 256);
for (int i = 0; i < len; i++)
{
bc[(unsigned char)pat[i]] = i;
}
}
The initial value of all elements is -1, which means that there is no such character. And 256 covers all the characters of 00-FF, so the number of members is 256
Then bc[(unsigned char)pattern[i]] = i; what does it mean?
It is to write our string position into the bad word table. If it occurs repeatedly, the last occurrence will overwrite the previous one.
Let's look at a simple example
"bcdb"
The bad word list should be
-1 3 1 2 -1 -1 -1 -1 -1 -1 -1...
So the bad word table not only records whether there is a certain character but also records the position where the character appears last.
int bm_search(const char *text, int n, const char *pat, int m)
{
int bc[260];
getBadCharTable(pat, m, bc);
int i = 0;
while (i <= n - m)
{
int j;
for (j = m - 1; j >= 0; j--)
{
if (text[i + j] != pat[j])
{
break;
}
}
if (j < 0)
{
return i;
}
int num = bc[(unsigned char)text[i + m]];// 取下一段的第一个字符坏字表中的值
if (num == -1)//看图3 说明没有 没有就完全截断了 任鸟飞逆向 那么从i+m+1继续开始
i += (m - bc[256]);
else
i += (m - num);//看图4
}
return -1;
}Everyone will find:
The overall code is similar to violent search, it can be said to be exactly the same
just brute force search
i++ changed to
int num = bc[(unsigned char)text[i + m]];
if (num == -1)
i += (m - bc[256]);
else
i += (m - num);This code is two cases
Figure 3: Situation 1
Additional extensions:
Here int bc[260]; instead of int bc[256] i += (m - bc[256]); instead of i+= m +1
Think about why?
When bc[256] is not initialized, it is -1 to meet our conditions
But when we define a special character as a wildcard, he can jump to the position of the wildcard.
For example, if the substring length is 10, we should have i+10+1
But now the wildcard appears at position 5
Figure 4: Case 2
Well, are these two algorithms completely understood?
7. Write signature location code
Well, the algorithm has been learned, we can write a code for feature code positioning to verify the learning results.
head File:
#pragma once
class Scan
{
public:
LPCSTR className = nullptr;
LPCSTR title = nullptr;
HANDLE hHandle = 0;
uintptr_t beginAddr = 0;
uintptr_t endAddr = 0;
const char* sCode = nullptr;
uintptr_t sCodeAddr[50] = { 0 };
int number = 0;
Scan() {};
Scan(LPCSTR c, LPCSTR t, const char* tzm, uintptr_t begin, uintptr_t end) :className(c), title(t)
, sCode(tzm), beginAddr(begin), endAddr(end)
{
if (className == 0 && title == 0)
{
hHandle = GetCurrentProcess();
}
else
{
HWND hGame = ::FindWindowA(className, title);
if (hGame != NULL)
{
DWORD processId;
::GetWindowThreadProcessId(hGame, &processId);
hHandle = ::OpenProcess(PROCESS_ALL_ACCESS, false, processId);
}
}
};
void sTob();
//如果代码哪里出现问题 可以公众号 任鸟飞逆向 探讨交流
DWORD violentSearch();
DWORD kmpSearch();
DWORD bmSearch();
void scanCallAndBase(const char* vName, const char* hName, int offset, DWORD len, DWORD right);
void scanOffset(const char* vName, int hoffset, int offset, int size);
};
C++:
#include "pch.h"
#include "Scan.h"
#include<vector>
using namespace std;
WORD* tzm = 0;
int len = 0;
const DWORD pageSize = 409600;
BYTE page[409600];
void wprintf_scan(const wchar_t* pszFormat, ...)
{
wchar_t szbufFormat[0x10000];
wchar_t szbufFormat_Game[0x11000] = L"";
va_list argList;
va_start(argList, pszFormat);
vswprintf_s(szbufFormat, pszFormat, argList);
wcscat_s(szbufFormat_Game, L"任鸟飞 ");
wcscat_s(szbufFormat_Game, szbufFormat);
OutputDebugStringW(szbufFormat_Game);
va_end(argList);
}
void Scan::sTob()
{
len = (int)(strlen(sCode) / 3 + 1);
tzm = new WORD[len];
for (int i = 0; i < len; i++)
{
if (sCode[i * 3] == '?' && sCode[i * 3 + 1] == '?')
{
tzm[i] = 256;
}
else
{
char c[] = { sCode[i * 3], sCode[i * 3 + 1], '\0' };
tzm[i] = (BYTE)::strtol(c, NULL, 16);
}
}
}
DWORD Scan::violentSearch()
{
sTob();
uintptr_t scanAddr = beginAddr;
number = 0;
while (scanAddr <= endAddr - (DWORD)len)
{
::ReadProcessMemory(hHandle, (LPCVOID)scanAddr, &page, pageSize, 0);
for (int i = 0; i <= pageSize; i++)
{
int j;
for (j = 0; j < len; j++)
{
if (tzm[j] != page[i + j] && tzm[j] != 256)
{
break;
}
}
if (j == len)
{
//如果代码哪里出现问题 可以公众号 任鸟飞逆向 探讨交流
sCodeAddr[number] = scanAddr + i;
number++;
}
}
scanAddr += pageSize;
}
delete[] tzm;
tzm = 0;
return number;
}
void getNext(WORD* tzm,int len, vector<int>& next)
{
next[0] = -1;
int i = 0, j = -1;
while (i < len - 1)
{
if (j == -1 || tzm[i] == tzm[j])
{
++i;
++j;
next[i] = j;
}
else
{
j = next[j];
}
}
}
DWORD Scan::kmpSearch()
{
sTob();
vector<int> next(len);
getNext(tzm,len, next);
uintptr_t scanAddr = beginAddr;
number = 0;
while (scanAddr <= endAddr - (DWORD)len)
{
::ReadProcessMemory(hHandle, (LPCVOID)scanAddr, &page, pageSize, 0);
int i = 0;
int j = 0;
while (i < pageSize)
{
if (j == -1 || page[i] == tzm[j]|| tzm[j] == 256)
{
++i;
++j;
if (j == len)
{
sCodeAddr[number] = scanAddr + i - j;
number++;
j = next[j - (int)1];
}
}
else
{
j = next[j];
}
}
scanAddr += pageSize;
}
delete[] tzm;
tzm = 0;
return number;
}
short bc[260];
void getBadCharTable(WORD* mtzm, int mtzmLength)
{
memset(bc, -1, sizeof(short) * 260);
for (int i = 0; i < mtzmLength; i++)
{
bc[mtzm[i]] = i;
}
}
DWORD Scan::bmSearch()
{
sTob();
getBadCharTable(tzm,len);
uintptr_t scanAddr = beginAddr;
number = 0;
while (scanAddr <= endAddr - (DWORD)len)
{
::ReadProcessMemory(hHandle, (LPCVOID)scanAddr, &page, pageSize, 0);
int i = 0;
while (i <= pageSize)
{
int j;
for (j = len - 1; j >= 0; j--)
{
if (tzm[j] != page[i + j] && tzm[j] != 256)
{
break;
}
}
if (j < 0)
{
sCodeAddr[number] = scanAddr + i;
number++;
}
int num = bc[page[i + len]];
if (num == -1)
i += (len - bc[256]);
else
i += (len - num);
}
scanAddr += pageSize;
}
delete[] tzm;
tzm = 0;
return number;
}
8. Compare three speeds
Was the result something you didn't expect?
The BM algorithm is the fastest and the violent search method is second, while the KMP algorithm is the slowest.
In fact, the reason is very simple, because there are very few repeated characters in the feature code, and there is no difference between KMP and brute force search at this time
However, KMP does a lot of processing, so it leads to the slowest speed
And the probability of scanning the memory feature code to find characters that are not there is extremely high, so BM will be the best!
Of course there is no perfect way
If it is a normal English file, the number of repeated characters may be very large, because there are only 26 letters, and KMP has an advantage at this time.
9. The data in the CQYH course we do positioning exercises
They are in Role
scan.sCode = "E8 ?? ?? ?? ?? 90 48 8B 0D ?? ?? ?? ?? 48 85 C9 74 0D 48 8B 89 48 02 00 00";
scan.scanCallAndBase("BaseRole", "MMOGame-Win64-Shipping.exe", 6, 7, 0);BaseAroundTree
scan.sCode = "0D 00 FF FF FF FF C0 89 05";
scan.scanCallAndBase("BaseSkillAttackCounter", "MMOGame-Win64-Shipping.exe", 7, 6, 0);PacketCall
scan.sCode = "48 89 5C 24 10 48 89 74 24 18 57 48 83 EC 20 80 B9 68 01 00 00 00";
scan.bmSearch();
printf_AsmHook("#define PacketCall (uintptr_t)MMOGamehmodul + 0x%X",scan.sCodeAddr[0] - (uintptr_t)h);PacketCallRCX
scan.sCode = "E8 ?? ?? ?? ?? 4C 8B E8 45 33 E4 48 8D BB 48 09 00 00 8B 77 08 2B 77 34";
scan.bmSearch();
uintptr_t PacketCallRCXAddr = scan.sCodeAddr[0] + 5 + *(int*)(scan.sCodeAddr[0] + 1);
uintptr_t packRCX = PacketCallRCXAddr + 7 + *(int*)(PacketCallRCXAddr + 3) - (uintptr_t)h;
printf_AsmHook("#define PacketCallRCX (uintptr_t)MMOGamehmodul + 0x%X", packRCX);There is no signature around this kind of location
We can only return to the outer positioning, and then fetch it according to the offset of the call
BaseAround
scan.sCode = "7E 41 41 B8 08 00 00 00 8B C8 E8 ?? ?? ?? ?? 48 83 3D";
scan.scanCallAndBase("BaseAround", "MMOGame-Win64-Shipping.exe", 0xF, 8, 1);BaseAroundTree
scan.sCode = "48 3B 1D ?? ?? ?? ?? E9 ?? ?? ?? ?? 48 8B 8F 5C 06 00 00";
scan.scanCallAndBase("BaseAroundTree", "MMOGame-Win64-Shipping.exe", 0, 7, 0);BasePackage_offset
scan.sCode = "73 74 0F B7 C2 45 0F B7 C0";
scan.bmSearch();
printf_AsmHook("#define BasePackage_offset 0x%X", *(DWORD*)(scan.sCodeAddr[0] + 0x11));BasePackage
scan.sCode = "90 8B 53 24 48 8D 0D";
scan.scanCallAndBase("BasePackage", "MMOGame-Win64-Shipping.exe", -0x18, 7, 0);BaseItemNameTree
scan.sCode = "41 8B 54 24 08 4C 8B 05";
scan.scanCallAndBase("BaseItemNameTree", "MMOGame-Win64-Shipping.exe", 5, 7, 0);BaseGroundItemTree
scan.sCode = "4C 8B 05 DF 07 78 01 4D 8B C8 4D 8B 00 48 8D 54 24 48";
scan.scanCallAndBase("BaseGroundItemTree", "MMOGame-Win64-Shipping.exe", 0, 7, 0);