Foreword: I didn't want to make wheels again. The online information includes open source fofa and some designs. Some erections are too complicated to use useful things, the entire yarn. There is no complete code.
Design scheme:
test platform: windows
test environment: php + mysql any programming language backend (implement data entry)
mysql table section: host ip header title body time
As others have said, a regular one was caught.
Grab the friend chain: regular [a-zA-Z0-9] [-a-zA-Z0-9] {0,62} (\. [A-zA-Z0-9] [-a-zA-Z0- 9] {0,62}) + \.?
Judge whether it is a domain name All domain names are OK after judging whether it is ip
|ac|ad|ae|af|ag|ai|al|am|an|ao|aq|ar|as|at|au|aw|az|ba|bb|bd|be|bf|bg|bh|bi|bj|bm|bn|bo|br|bs|bt|bv|bw|by|bz|ca|cc|cf|cg|ch|ci|ck|cl|cm|cn|co|cq|cr|cu|cv|cx|cy|cz|de|dj|dk|dm|do|dz|ec|ee|eg|eh|es|et|ev|fi|fj|fk|fm|fo|fr|ga|gb|gd|ge|gf|gh|gi|gl|gm|gn|gp|gr|gt|gu|gw|gy|hk|hm|hn|hr|ht|hu|id|ie|il|in|io|iq|ir|is|it|jm|jo|jp|ke|kg|kh|ki|km|kn|kp|kr|kw|ky|kz|la|lb|lc|li|lk|lr|ls|lt|lu|lv|ly|ma|mc|md|me|mg|mh|ml|mm|mn|mo|mp|mq|mr|ms|mt|mv|mw|mx|my|mz|na|nc|ne|nf|ng|ni|nl|no|np|nr|nt|nu|nz|om|pa|pe|pf|pg|ph|pk|pl|pm|pn|pr|pt|pw|py|qa|re|ro|ru|rw|sa|sb|sc|sd|se|sg|sh|si|sj|sk|sl|sm|sn|so|sr|st|su|sy|sz|tc|td|tf|tg|th|tj|tk|tm|tn|to|tp|tr|tt|tv|tw|tz|ua|ug|uk|us|uy|va|vc|ve|vg|vn|vu|wf|ws|ye|yu|za|zm|zr|zw|com|net|org|int|edu|gov|mil|arpa|Asia|biz|info|name|pro|coop|aero|museum|cc|tv
数据录入：很简单，就不详细说了。
Other details: In the process of data entry, a large number of spam domain names, second-level and third-level pan-resolution domain name
judgment methods split "." Off. Of course, this method may cause some domain names not to be entered.
How to get it depends on your thoughts.
After the php, you can realize online query. Recommend the great god that Baidu knows. It will be solved in a few minutes.
Occupied space: 26,000 pieces of website information, all of which are 967.50 MB. In theory, a 500gb hard disk can store 1300W website information
for reference only, because the size of the web pages are different.
Use massscan for ordinary civilian use. It can also scan more than 300 million ips in 24 hours