URL Collector - Keyword Collection

URL Collector - Keyword Collection

Msray-plus is an enterprise-level comprehensive crawler/collection software developed in GO language.

Keywords: search engine result collection, domain name collection, URL collection, URL collection, whole network domain name collection, CMS collection, contact information collection

Support billion-level data storage, import, repeated judgment, etc. There is no need to use complicated commands, and a local WEB management background is provided to perform related operations on the software, which is powerful and easy to use!

1: It can collect search results (SERP data) corresponding to keywords imported by users from multiple search engines at home and abroad in batches, and perform structured data storage and custom filtering processing;

2: From the url seed address provided by the user, it can automatically crawl the website data of the whole network continuously, and carry out structured data storage and custom filtering processing;

3: From the website list data provided by the user, the website contact information can be automatically extracted, including but not limited to email, mobile phone/telephone, QQ, WeChat, facebook, twitter, etc.

At the same time, it supports storage of various data such as domain name, root URL, URL (url), IP, country to which the IP belongs, title, description, access status, etc. It is mainly used for domain name/URL/collection of the entire network, industry market research and analysis, and collection of specified types of websites And analysis, network promotion analysis, and provide data support for various big data analysis.

System advantages:

1. Developed with GO language (enterprise-level project standard). Cross-platform, can run perfectly on ubuntu, centos, windows, mac and other systems;
2. Search engine results (SERP data) collection, support multi-search engine parallel collection + each engine multi-threaded search, high efficiency;
3. Support multiple well-known search engines at home and abroad, which can break through security verification! Including but not limited to Baidu (computer terminal + mobile terminal), Google (google), Bing (bing), Shenma, Yandex, Qwant, etc.;
4. Using B/S structure, with its own WEB management background, it can be accessed remotely! There is no need to use commands, which is easy to use and reduces the difficulty of use.
5. Supports fine-grained customization by task, custom opening and closing of specified search engines, custom number of threads, etc.;
6. The collection efficiency is high, and the daily collection of millions/tens of millions is not repeated and stress-free;
7. The system resource occupation is small, and the CPU and memory pressure are ultra-small;
8. It can intelligently identify the generic domain name site groups in the results, and automatically add them to the blacklist to prevent a large number of sub-level domain names of the same domain name;
9. It is simple and convenient to use, and can be used quickly without technical experience;
10. Support unlimited collection, support automatic capture of similar search terms in search engines and automatic expansion to add seed keywords;
11. Efficient automatic result anti-duplication function (100% no duplication);
12. Ultra-comprehensive support for multiple filtering schemes, such as by domain name level, by title, by content, by country, by domain name suffix, etc.;
13. It can save various data such as domain name, root URL, URL (url), IP, country to which IP belongs, title, description, etc.;
14. Comprehensive data export function, supports customized data export in multiple formats according to tasks, and also supports exporting all results by time (such as by day), and even automatically generates records without manual export and saves them locally;
15. Supports data real-time push function interface, and can customize the HTTP interface address for receiving data, which is convenient for expansion development and custom secondary processing of data, such as linkage with other software;
16. Other extended functions are updated from time to time, such as the "same server IP website query" function, which can be used for free.
17. Perfect online documentation, stable and fast version update service;

operating environment

1: 跨平台，同时支持ubuntu、centos、windows、mac等系统; 
2: 建议操作系统选择64位系统。
3: 建议使用chrome浏览器访问软件后台;

Custom collection keywords

Create a keyword collection task

Click the [Custom Import Seed Keyword File] button to select a list file containing keywords to be collected;

Configure relevant search engines according to your own business scenarios, and collect relevant settings

collecting

Collection result preview:

Get more content >>>

qq communication group: 50246933
tg communication group: https://t.me/ms_ray
Software documentation: https://www.msray.net/doc
Free version download: https://github.com/super-l/msray