String matching algorithm [learning algorithm]
Preface
2023-8-6 12:06:42
The following content is from "[Learning Algorithm]"
and is for learning and communication purposes only.
copyright
Delete the following words when publishing on other platforms.
This article was first published on the CSDN platform.
The author is CSDN@日星月云.
The homepage of the blog is https://blog.csdn.net/qq_51625007.
Delete the above words when publishing on other platforms.
recommend
28. Find the subscript of the first match in a string
String pattern matching
The knowledge content comes from
Chapter 4 String (Data Structure and Algorithm)
The positioning operation of a substring is to find the position where the substring first appears after the pos character in the main string. It is also called "string pattern matching" or "string matching". This operation is widely used. For example, in text editing programs, it is often necessary to find where a specific word occurs in the text. Obviously, effective algorithms to solve this problem can greatly improve the response performance of text editing programs. In string matching, the main string S is generally called the "target string" and the substring T is called the "pattern string".
There are many algorithms for pattern matching. This chapter only discusses two string matching algorithms, BF pattern matching and KMP pattern matching.
BF pattern matching algorithm
[BE algorithm idea]
The Brute-Force algorithm, also known as the "brute force matching" algorithm (referred to as the BP algorithm), starts from the pos character of the main string S and compares it with the first character of the pattern string. If they are equal, then Continue to compare subsequent characters one by one; otherwise, go back to the pos+1 character of the main string and start comparing it with the pattern string T again. By analogy, until each character in the pattern string is equal to a consecutive character sequence in the main string, it is said that the pattern matching is successful. At this time, the first character of the pattern string in the main string S is returned. position; otherwise there is no character sequence equal to the pattern string in the main string, and the pattern matching is unsuccessful.
[BF algorithm description]
The strategy for comparing the substring starting from the pos-th character of the main string S and the pattern string T is to compare them sequentially from front to back. Therefore, setting the indicator i in the main string indicates the currently compared character in the main string S; setting the indicator j in the pattern string T indicates the currently compared character in the pattern string T.
As shown in Figure 4-5, an example of the matching process is given, in which the characters corresponding to the box shading are mismatched characters that are not equal when comparing the main string S and the pattern string T (assuming pos=1).
Compare the pos-th character in the main string S with the first character of the pattern string T. If they are equal, continue to compare the subsequent characters one by one. At this time, i++;j++; otherwise, start from the next character of the main string (i-j+2 ) is compared with the first character (j=1) of the pattern string, and the analysis details are shown in Figure 4-6(a).
When the match is successful, the position of the first character in the pattern string T relative to the main string (iT.en) is returned; otherwise, 0 is returned. For detailed analysis, see Figure 4-6(b), where m is the length of the pattern string T. .len.
int Index(SString S,int pos,SString T)
int i=pos,j=1;//主串从第pos开始,模式串从头开始
while (i<=S.len&&j<=T.len){
if(S.ch[i]==T.ch[j]){
//当对应字符相等时,比较后续字符
i++;
j++;
}
else{
//当对应字符不等时
i=i-j+2; //主串回溯到j-j+2的位置重新比较
j=1; //模式串从头开始重新比较
}
if(j>T.len) return i-T.len; //匹配成功时,返回匹配起始位置
else return 0; //匹配失败时,返回0
[BF algorithm analysis]
The idea of BF algorithm is relatively simple, but in the worst case, the time complexity of the algorithm is 0 (n×m), where n and m are the lengths of the main string and pattern respectively. The main time consumption of this algorithm is the backtracking of the comparison position after mismatch, which results in too many comparisons. In order to reduce the time complexity, an algorithm without backtracking can be used.
KMP pattern matching algorithm
[KMP algorithm idea]
Knuth-Morris-Pratt algorithm (KMP for short) is an improved algorithm jointly proposed by DEKnuthJ.HMorris and V.RPratt. The KMP algorithm is a classic algorithm in pattern matching. Compared with the BF algorithm, the difference of the KMP algorithm is that it eliminates the backtracking of the main string S pointer i in the BF algorithm. The time complexity of the improved algorithm is 0(n+m).
[KMP algorithm description]
In the KMP algorithm, whenever characters appear to be unequal in a matching process, the pointer in the main string S does not need to backtrack, but uses the "partial matching" result that has been obtained to move the pattern string to the right. After sliding it as far as possible, continue the comparison.
Looking back at the example of the matching process in Figure 4-5, in the third match, when the characters i=7 and j=5 are not equal, the comparison starts again from i=4.j=1. However, after careful observation, it can be found that the three comparisons at i=4 and j=1, i=5 and j=1, and i=6 and j=1 are unnecessary. Because it can be concluded from the third partial matching result that the 4th, 5th, and 6th characters in the main string must be equal to the 2.3.4 characters in the pattern string, that is, they are all 'bca. Because the first character in the pattern string is 'a', it does not need to be compared with these three characters, but only needs to slide the pattern string three characters to the right to continue i=7.j=2 The characters can be compared. In the same way, when the characters are not equal in the first match, you only need to move the pattern to the right by two characters to compare the characters when i=3 and j=1. Therefore, during the entire matching process, the pointer does not backtrack, as shown in Figure 4-7.
Generally speaking, it is assumed that the main string is 'S 1 S 2 ...S n ' and the pattern string is 'T 1 T 2 ...T n '. From the analysis of the above example, we can know that in order to implement the KMP algorithm, the following problems need to be solved. When the matching process When a "mismatch" occurs in the pattern string (i.e. Si ≠ T i ), the sliding distance of the pattern string "slide to the right" is very far. That is to say, when the character Si in the main string and the character T j " in the pattern string are mismatched When matching ", which character in the pattern string should be compared with the character Si in the main string (i pointer does not backtrack) ?
Assuming that the character Si in the main string should continue to be compared with the character T k (k<j) in the pattern at this time, then the main string S and the pattern string T satisfy the following relationship.
S=S 1 S 2 …S i-j+1 S i-j+2 …S i-k+1 …S i-1 S i …S n
T= T 1 T 2 …T j-k+1 … T j-k+2 …
T= T 1 …T k-1 …
It can be seen that if 'T 1 T 2 ...T k-1 '='T j-k+1 T j-k+2 ...T j-1 ' exists in the pattern string , and satisfies 1<k<j, then When Si ≠T j during the matching process , you only need to slide the pattern string to the right until the k-th character is aligned with the i-th character in the main string. The matching only needs to continue from the comparison of Si and T k . No need Traceback of i pointer. In order to "slide" as far as possible during the matching process, a larger k value that meets the conditions should be selected.
If next[j]=k, then next[] indicates that when the j-th character in the pattern "mismatches" with the corresponding character in the main string, the position of the character in the pattern needs to be compared with the character in the main string again. . This leads to the definition of the next function of the pattern string:
It can be seen that the calculation of the next function is only related to the pattern string itself and has nothing to do with the main string. Among them, 'T 1 T 2 ...T k-1 ' is the true prefix substring of 'T 1 T 2 ...T j-1 '', 'T j-k+1 T j-k+2 ...T j-1 ' Is the true suffix substring of 'T 1 T 2 ...T j-1 '. When the set in the next function definition is not empty, the value of next[j] is equal to the maximum substring length + 1 when the true prefix substring and true suffix substring of the string 'T 1 T 2 ...T j-1 ' are equal .
Through the above analysis, the calculation process of the next value of the pattern string 'abaabcac' is deduced as shown in Table 4.1.
(1) When j=1, it is known from the definition that next[1]=0;
(2) When j=2, the k value that satisfies 1<k<j does not exist, and it is known from the definition that next[2] =1
The next function value of the pattern string 'abaabcac' is shown in Figure 4-8.
After obtaining the function of the pattern string next, the matching can be performed as follows: Assume that pointers iand jrespectively indicate the currently compared characters in the main string Sand the pattern string , let the initial value of be , and the initial value of be . If S i =T i during the matching process , then the sums are increased by 1 respectively; otherwise, they remain unchanged, and the returned positions are compared again (that is, Si and T next[j] are compared). If they are equal, the pointers are respectively Increase by 1, otherwise return to the position of the next value, and so on, until the following two possibilities: one is that when the characters are equal when returning to a certain value ( ), the pointers will increase by 1 to continue matching; the other is to return to 0 (that is, "mismatch" with the first character of the pattern), then you need to slide both the main string and the pattern string one position to the right at the same time (j=0 at this time, when sliding one position to the right time, that is, the first character of the pattern string), that is, the comparison starts again from the next character Si +1 of the main string and the pattern Ti . Figure 4-9 is an example of the KMP matching process (assuming pos=1)Tiposj1ijijnext[j]jnextjnextnext[next[···next[j]]]jnext值
[Algorithm 4-13]KMP pattern matching algorithm
int Index_KMP( SString S, in pos, SString T){
int i=pos,i=1; //主串从第pos开始,模式事从头开始
while(i<=S.len && j<=Tlen){
if(j==0||S.ch[j]==T.ch[j]){
//继续比较后续宇符
++i;++j;
}else{
j=next[j]; //模式申向右滑动
}
if(j>T.len) return i-Tlen; //匹配成功时,返回匹配的起始位置
else relurn 0 //匹配失败时，返回0
[Next algorithm description]
The KMP algorithm is executed based on the known next function value of the pattern. So, how to obtain the next function value of the pattern string?
It can be seen from the definition that next[1]=0, assuming next[j]=k, this shows that there is 'T 1 T 2 ···T k-1 '='T j-k+1 T j-k in the pattern string +2 ···T j , such a relationship, in which the value kat this time next[j+1]may have the following two situations to satisfy a certain value of 1<k<j.
(1) If T j =T k , it means that in the pattern string 'T 1 T 2 ···T k-1 '='T j-k+1 T j-k+2 ···T j '
T=T~1~ ··· T~j-k+1~ ··· T~j-1~ T~j~ ··· T~m~
=
T= T~1~ ··· T~k-1~ T~k~···
长度为k
This means next[j+1]=k+1, that is, next[j+1]=next[j]+1 .
(2) If T j ≠T k , it means that in the pattern string 'T 1 T 2 ···T k-1 '≠'T j-k+1 T j-k+2 ···T j '
At this time, the problem of finding the value of the next function can be regarded as a pattern matching problem. The entire pattern string is both the main string and the pattern string, where 1<k'<k<j.
T=T~1~ ··· T~j-k+1~ ··· T~j-k'+1~ ··· T~j-1~ T~j~ ··· T~m~
≠
T= T~1~ ··· T~k-k'+1~ ··· T~k-1~ T~k~ ···
T= T~1~ ··· T~k'-1~ T~k'-next[k]~
①If T j =T k' , and k'=next[k], then next[j+1]=k'+1 , that is, next[j+1]=next[k]+1, which is also equivalent to next [j+1]=next[next[j]]+1.
②If T j ≠T k' , continue to compare T j and T next[k'] , that is, compare T j and T next[next[k]]
...
Then keep repeating, if until the last j=0 If the comparison fails, then next[j+1]=1 .
Through the above analysis, when calculating the next value of the j+1th character, it is necessary to see whether the jth character is equal to the character pointed to by the next value of the jth character.
It can be deduced that the next value calculation process of the pattern string T=abaabcac is as follows.
①When j=1, it is known from the definition that next[1]=0.
j=1
T=a
n=0
②When j=2, the value satisfying 1<h<j does not exist, and next[2]=1 is known from the definition.
j=12
T=ab
n=01
③When j=3, since T 2 ≠ T 1 and next[1]=0, then next[3]=1.
j=12345678
T=abaabcac
n=011
T~2~ ≠ T~next[2]~ (T~1~)
T~2~ ? T~next[1]~ (0)
next[3]=1
④When j=4, since T 3 =T 1 , next[4]=next[3]+1, that is, next[4]=2.
j=12345678
T=abaabcac
n=0112
T~3~ = T~next[3]~ (T~1~)
next[4]=next[3]+1
⑤When j=5, since T 4 ≠ T 2 and the value of next[2] is 1, continue to compare T 4 and T 1. Since T 4 =T 1 , then next[5]=next[2] +1, that is, next[5]=2.
j=12345678
T=abaabcac
n=01122
T~4~ ≠ T~next[4]~ (T~2~)
T~4~ = T~next[2]~ (T~1~)
next[5]=next[2]+1
⑥When j=6, since T 5 =T 2 , then next[6]=next[5]+1, that is, next[6]=3.
j=12345678
T=abaabcac
n=011223
T~5~ = T~next[5]~ (T~2~)
next[6]=next[5]+1
⑦When j=7, since T 6 ≠T 3 and the value of next[3] is 1, continue to compare T 6 and T 1. Since T 6 ≠ T 1 and next[1]=0, then next [7]=1.
j=12345678
T=abaabcac
n=0112231
T~6~ ≠ T~next[6]~ (T~3~)
T~6~ ≠ T~next[3]~ (T~1~)
T~6~ ? T~next[1]~ (0)
next[7]=1
⑧When j=8, since T 7 =T 1 , then next[8]=next[7]+1, that is, next[8]=2.
j=12345678
T=abaabcac
n=01122312
T~7~ = T~next[7]~ (T~1~)
next[8]=next[7]+1,
Therefore, the next value of the pattern string is obtained as shown in Figure 4-8.
j=12345678
T=abaabcac
n=01122312
[Algorithm 4-14] next algorithm
void Get_Next( SString T, int next[]){
int j=1,k=0;
next[1]=0;
while(j<T.len){
if(k==0||T.ch[j]==Tch[k] ){
++j;
++k;
nexi[j]=k；
}else{
k=next[k];
}
}
}
[nextval algorithm description]
The next function defined above has flaws in some cases. Assuming that the main string is 'aaabaaaab' and the pattern string is 'aaaab', the next function value corresponding to the pattern string is as follows.
After obtaining the next value of the pattern string, the matching process is shown in Figure 4-10(a).
It can be seen from the string matching process that when i=4, j=4, S 4 is not equal to T 4 , as shown by next[j], i=4, j=3; i=4, j= 2; Three comparisons of i=4 and j=1. In fact, because the 1st, 2nd, 3rd characters and the 4th character in the pattern are all equal (that is, they are all a), there is no need to compare with the 4th character in the main string, but the pattern can be Swipe 4 characters to the right to directly compare the characters of i=5 and j=1.
That is to say, if next[j]=k is obtained according to the above definition, and T j =T k in the pattern string , then when Si ≠ T j , there is no need to compare Si and T k , and it is directly compared with T next [k] for comparison; in other words, the value of next[j] at this time should be the same as next[k], so next[j] is corrected to nextval[j]. When T j ≠T k in the pattern string , when Si ≠T j , comparison between Si and T k still needs to be performed, so the value of nextval[j] is k, that is, the value of nextval[j] is next [ j ] value.
Through the above analysis, when calculating the nextval value of the jth character, it depends on jwhether the character pointed to by the jth character is equal to the character pointed to by the jth jcharacter next. If equal, then nextval[j]=nextval[next[j]]; otherwise, nextval[j]=next[j]. From this, it can be deduced that the nextval value calculation process of the pattern string T='aaaab' is as follows.
①When j=1, it is known from the definition that nextval[1]=0.
②When j=2, nex[2]=1, and T 2 =T 1 , then nextval[2]=nextval[1], that is nextval[2]=0.
③When j=3, next[3]=2, and T 3 =T 2 , then nextval[3]=nextval[2], that is, nextval[3]=0.
④When j=4, next[4]=3, and T 4 =T 3 , then nextval[4]=nextval[3], that is, nextval[4]=0.
⑤When j=5, next[5]=4, and T 5 ≠ T 4 , then nextval[5]=next[5], that is, nextval[5]=4.
The nextal function value of the pattern string 'aaaab' is as follows.
After obtaining the nextval value of the pattern string, the matching process is shown in Figure 4-10(b).
There are two ways to find the nextval function value. One is to find it directly by observation without relying on the next array value. The other method is to infer based on the next array value as mentioned above. Only the second method is introduced here.
[Algorithm 4-15] nextval algorithm
void Get_NextVal(SString T, int next[ ] ,int nextval[ ]){
int j=2,k=0;
Get_Next(T,next);//通过算法4-14获得T的next值
nextval[1]=0;
while (j<=T.len )
k=next[j];
if(T.ch[j]==T.ch[k]) nextval[j]=nextyal[ k];
else nextval[j]=next[j];
j++;
}
[KMP algorithm analysis]
The KMP algorithm is executed on the basis of next or nextval of a known pattern. If one of them is not known, there is no way to use the KMP algorithm. Although there are next and nextval, their meaning and function are exactly the same, so when matching next or nextval is known, the matching algorithm remains unchanged.
Usually the length m of the pattern string is much smaller than the length n of the main string and the time complexity of calculating the next or nextval function is 0(m). Therefore, the added calculation of next or nextval is worthwhile for the entire matching algorithm.
The time complexity of the BF algorithm is 0(nxm), but in actual execution, m is often much smaller than n, so it is approximately 0(n+m), so it is still used today. The KMP algorithm is faster than the BF algorithm only when there are many "partial matches" between the pattern string and the main string. The biggest feature of the KMP algorithm is that the pointer of the main string does not need to be traced back. During the entire matching process, the main string only needs to be scanned once from beginning to end. It is very effective for processing huge files input from peripherals, and can be matched while reading.
textbook
practise:
abaabcac
01122312
aaaab
01234
abaabcabc
011223123
ababcabaababb
0112312342345
abcabaa
0111232
abcaabbabcabaacbacba
01112231234532211211
The result is exactly the same as the language textbook
import java.util.Arrays;
public class KMP0 {
public static final int MAXLEN=50;
static class SString{
char[] ch=new char[MAXLEN+1];//0号单元不使用
int len;
public SString() {
}
public SString(char[] ch, int len) {
this.ch = ch;
this.len = len;
}
}
public static void main(String[] args) {
// test1();
test2();
}
private static void test2() {
// String t0="0abaabcac";
// String t0="0abaabcabc";
String t0="0aaaab";
SString t=new SString(t0.toCharArray(),t0.length()-1);
int[] next=new int[t.len+1];//0号也不使用
get_next(t,next);
System.out.println(Arrays.toString(next));
int[] nextval=new int[t.len+1];//0号也不使用
get_nextval(t,next,nextval);
System.out.println(Arrays.toString(nextval));
}
private static void test1() {
String s0="0aaabaaaab";
String t0="0aaaab";
SString s=new SString(s0.toCharArray(),s0.length()-1);
SString t=new SString(t0.toCharArray(),t0.length()-1);
int[] next=new int[t.len+1];//0号也不使用
get_next(t,next);
int i = index_kmp(s, 0, t,next);
System.out.println(i);
}
public static int index_kmp(SString s,int pos,SString t,int[] next){
int i=pos,j=1;
while (i<=s.len&&j<=t.len){
if (j==0||s.ch[i]==t.ch[j]){
++i;
++j;
}else {
j=next[j];
}
}
if(j>t.len) return i-t.len;
else return 0;
}
public static void get_next(SString t,int[] next){
int j=1,k=0;
next[1]=0;
while (j<t.len){
if(k==0||t.ch[j]==t.ch[k]){
++j;
++k;
next[j]=k;
}else{
k=next[k];
}
}
}
public static void get_nextval(SString t,int[] next,int[] nextval){
int j=2,k=0;
get_next(t,next);
nextval[1]=0;
while (j<=t.len){
k=next[j];
if(t.ch[j]==t.ch[k]){
nextval[j]=nextval[k];
}else {
nextval[j]=next[j];
}
j++;
}
}
}
Implement algorithms in Java
Official solution
class Solution {
public int strStr(String haystack, String needle) {
int n = haystack.length(), m = needle.length();
if (m == 0) {
return 0;
}
int[] pi = new int[m];
for (int i = 1, j = 0; i < m; i++) {
while (j > 0 && needle.charAt(i) != needle.charAt(j)) {
j = pi[j - 1];
}
if (needle.charAt(i) == needle.charAt(j)) {
j++;
}
pi[i] = j;
}
for (int i = 0, j = 0; i < n; i++) {
while (j > 0 && haystack.charAt(i) != needle.charAt(j)) {
j = pi[j - 1];
}
if (haystack.charAt(i) == needle.charAt(j)) {
j++;
}
if (j == m) {
return i - m + 1;
}
}
return -1;
}
}
Call Java API
class Solution {
public int strStr(String haystack, String needle) {
return haystack.indexOf(needle);
}
}
Reference Java API
Just use the BF algorithm
class Solution {
public static int strStr(String haystack, String needle) {
char[] haystackValue=haystack.toCharArray();
char[] needleValue=needle.toCharArray();
return index(haystackValue,needleValue);
}
public static int index(char[] source, char[] target) {
int sourceOffset=0;
int targetOffset=0;
int sourceCount=source.length;
int targetCount=target.length;
int max=sourceOffset+(sourceCount-targetCount);
char first=target[targetOffset];
for(int i=sourceOffset;i<=max;i++){
//找第一个字符
if (source[i] != first) {
while (++i <= max && source[i] != first);
}
//匹配剩余的部分
if(i<=max){
int j=i+1;
int end=j+targetCount-1;
for (int k = targetOffset+1; j < end&&source[j]==target[k]; j++,k++);
if(j==end){
return i-sourceOffset;
}
}
}
return -1;
}
}
BF algorithm
class Solution {
public int strStr(String haystack, String needle) {
return indexWithBF(haystack,0,needle);
}
public static int indexWithBF(String s,int pos,String t){
int i=pos,j=0;//主串从pos开始，模式串从头开始
while(i<s.length()&&j<t.length()){
if(s.charAt(i)==t.charAt(j)){
//当对应字符相等时，比较后续字符
i++;
j++;
}else{
//对应字符不等时
i=i-j+1; //主串回溯到i-j+1的位置重写比较
j=0; //模式串从头开始重写比较
}
}
if(j>=t.length()) return i-t.length(); //匹配成功时，返回匹配起始位置
else return -1; //匹配失败时，返回-1
}
}
KMP algorithm
class Solution {
public int strStr(String haystack, String needle) {
return indexWithKMP(haystack,0,needle);
}
public static int indexWithKMP(String s,int pos,String t){
int[] next=next(t);
int i=pos,j=0; //主串从pos开始，模式串从头开始
while(i<s.length()&&j<t.length()){
if(j==-1||s.charAt(i)==t.charAt(j)){
//继续比较后续字符
i++;
j++;
}else{
//模式串向右滑动
j=next[j];
}
}
if(j>=t.length()) return i-t.length(); //匹配成功时，返回匹配起始位置
else return -1; //匹配失败时，返回-1
}
public static int[] next(String t){
int len=t.length();
int[] next=new int[len+1];
int j=0,k=-1;
next[0]=-1;
while (j<len){
if(k==-1||t.charAt(j)==t.charAt(k)){
j++;
k++;
next[j]=k;
}else{
k=next[k];
}
}
return next;
}
}
Implement algorithm in C
KMP algorithm
int strStr(char* haystack, char* needle) {
int n = strlen(haystack), m = strlen(needle);
if (m == 0) {
return 0;
}
int pi[m];
pi[0] = 0;
for (int i = 1, j = 0; i < m; i++) {
while (j > 0 && needle[i] != needle[j]) {
j = pi[j - 1];
}
if (needle[i] == needle[j]) {
j++;
}
pi[i] = j;
}
for (int i = 0, j = 0; i < n; i++) {
while (j > 0 && haystack[i] != needle[j]) {
j = pi[j - 1];
}
if (haystack[i] == needle[j]) {
j++;
}
if (j == m) {
return i - m + 1;
}
}
return -1;
}
at last
We all have a bright future
I wish you all the best in your postgraduate entrance exams, I wish you all success
in your work, I wish you all get what you wish for, like, collect and follow.