Design and Implementation of a Lexical
【Purpose】
By designing and compiling a specific lexical analysis program, deepen the understanding of the principle of lexical analysis, and master the program
A lexical analysis method that decomposes the language source program into various words during scanning.
【Experimental Requirements】
- In this experiment, the lexical analyzer is designed as an independent subroutine to complete the lexical analysis task.
- Give the recognition state transition diagram of the source language for lexical analysis (this experiment is C language) or its subset.
- Design the word category table, and use the binary group to output the recognized word symbols (see the experiment content for specific requirements).
- Using C++ language (programming language for writing lexical analyzer) to realize the above recognition state transition diagram, can recognize the word symbol of the source language or its subset.
【Experimental content】
- Keywords: main, return, if, else, int, char
- Operators and delimiters: (){}, ;= != > < >= <= == + * /
- Identifier <id> and integer parameter <num>
【Experimental process and results】
1. Word Category Table Design
| word |
internal encoding |
word |
internal encoding |
word |
internal encoding |
word |
internal encoding |
| main |
0 |
<id> |
1 |
<num> |
2 |
return |
3 |
| if |
4 |
else |
5 |
int |
6 |
char |
7 |
| ( |
8 |
) |
9 |
{ |
10 |
} |
11 |
| , |
12 |
; |
13 |
= |
14 |
!= |
15 |
| > |
16 |
< |
17 |
>= |
18 |
<= |
19 |
| == |
20 |
+ |
21 |
* |
22 |
/ |
23 |
2. Construct a state transition diagram for recognizing words in the source language ( C language)
Figure 1 State transition diagram
Third, the implementation of the lexical analyzer
1. flow chart design
Figure 2 Program flow chart
2. key source code
#include<bits/stdc++.h>
using namespace std;
map<string, string> word;
std::map<string, string>::iterator it;
///初始化
void map_init() {
word["main"] = "0";
word["return"] = "3";
word["if"] = "4";
word["else"] = "5";
word["int"] = "6";
word["char"] = "7";
word["("] = "8";
word[")"] = "9";
word["{"] = "10";
word["}"] = "11";
word[","] = "12";
word[";"] = "13";
word["="] = "14";
word["!="] = "15";
word[">"] = "16";
word["<"] = "17";
word[">="] = "18";
word["<="] = "19";
word["=="] = "20";
word["+"] = "21";
word["*"] = "22";
word["/"] = "23";
}
struct node {
string id;
string s;
}aa[10000];
int main() {
map_init();
char ch;
char a;
int len = 0;
string word1;//string变量识别单词
string str;//string变量进行字符识别
cout << "-----------------词法分析器-----------------" << endl;
string dir;
cout << "请输入源文件地址：(路径中请不要包含中文)" << endl;
cin >> dir;
ifstream infile(dir);//文件输入流，文件路径
cout << "请输入目标文件地址：" << endl;
cin >> dir;
ofstream outfile(dir);//文件输出流
ostringstream buf;
while (buf && infile.get(ch)) buf.put(ch);//将文件中的字符读出来
str = buf.str();//将得到的字符储存到string类型变量中
int csize = str.length();
for (int i = 0; i < csize; i++) {//对整个字符串进行遍历
while (str[i] == ' ' || str[i] == '\n') i++;//若最开始为空格或换行符，则将指针的位置往后移
if (isalpha(str[i])) {//对标识符和基本字进行识别,调用库函数isalpha()
word1 = str[i++];
while (isalpha(str[i]) || isdigit(str[i])) {
word1 += str[i++];
}
it = word.find(word1);
if (it != word.end()) {//判断是不是基本字，若为基本字则进行输出
cout << "(" << word[word1] << "," << word1 << ")" << endl;
aa[len].id = word[word1];
aa[len++].s = word1;
}
else {//否则直接输出
cout << "(1" << "," << word1 << ")" << endl;
aa[len].id = "1";
aa[len++].s = word1;
}
i--;
}
else if (isdigit(str[i])) {//判断是不是常数，调用库函数isdigit()
word1 = str[i++];
while (isdigit(str[i])) {
word1 += str[i++];
}
if (isalpha(str[i])) {
cout << "error1!" << endl;
break;
}
else {
cout << "(2" << "," << word1 << ")" << endl;
aa[len].id = "2";
aa[len++].s = word1;
}
i--;
}
else if (str[i] == '<') {//对<,<=分别进行判断
word1 = str[i++];
if (str[i] == '>') {
word1 += str[i];
cout << "(" << word[word1] << "," << word1 << ")" << endl;
aa[len].id = word[word1];
aa[len++].s = word1;
}
else if (str[i] == '=') {
word1 += str[i];
cout << "(" << word[word1] << "," << word1 << ")" << endl;
aa[len].id = word[word1];
aa[len++].s = word1;
}
else if (str[i] != ' ' || !isdigit(str[i]) || !isalpha(str[i])) {
cout << "(" << word[word1] << "," << word1 << ")" << endl;
aa[len].id = word[word1];
aa[len++].s = word1;
}
else {
cout << "error2!" << endl;
break;
}
i--;
}
else if (str[i] == '>') {//对>,>=分别进行判断
word1 = str[i++];
if (str[i] == '=') {
word1 += str[i];
cout << "(" << word[word1] << "," << word1 << ")" << endl;
aa[len].id = word[word1];
aa[len++].s = word1;
}
else if (str[i] != ' ' || !isdigit(str[i]) || !isalpha(str[i])) {
cout << "(" << word[word1] << "," << word1 << ")" << endl;
aa[len].id = word[word1];
aa[len++].s = word1;
}
else {
cout << "error3!" << endl;
break;
}
i--;
}
else if (str[i] == ':') {//对:=进行判断
word1 = str[i++];
if (str[i] == '=') {
word1 += str[i];
cout << "(" << word[word1] << "," << word1 << ")" << endl;
aa[len].id = word[word1];
aa[len++].s = word1;
}
else if (str[i] != ' ' || !isdigit(str[i]) || !isalpha(str[i])) {
cout << "(" << word[word1] << "," << word1 << ")" << endl;
aa[len].id = word[word1];
aa[len++].s = word1;
}
else {
cout << "error4!" << endl;
break;
}
i--;
}
else {//对其他的基本字依次进行判断
word1 = str[i];
it = word.find(word1);
if (it != word.end()) {
cout << "(" << word[word1] << "," << word1 << ")" << endl;
aa[len].id = word[word1];
aa[len++].s = word1;
}
//else {
// cout << "error5!" << endl;
// break;
//}
}
}
//输出流，输出到对应的结果文件中
if (outfile.is_open())
{
for (int i = 0; i < len; i++) {
outfile << "(" << aa[i].id << "," << aa[i].s << ")" << '\n';
}
outfile.close();
}
infile.close();
cout << "输入任意键关闭" << endl;
string s;
cin >> s;
return 0;
}
3. Lexical analyzer running effect
Figure 3 Operation result graph
Figure 4 Source file diagram
Figure 4 Object file diagram
4. Problems and their solutions during the experiment
After the program is packaged into .exe, it can run on your own computer, but it fails to run on other computers. By searching the information, it is found that the possible reason is that the target host does not have related dlls or system and processor problems, and related dlls should also be packaged when packaging;
Five , C++ program packaging
VS does not have the universal library #include<bits/stdc++.h>, so you have to configure it yourself. . , There is a way to search casually on the Internet.
(1)
(2)
(3)
(4)
(5) Run it, go to the Debug file to find it, don’t worry and send it to a good friend to try
