Formerly dependent on big data as long as the master burp, sqlmap, nmap, awvs can be targeted for vulnerability detection. When a scene change encounter hundreds of targets to be tested. This artificial means not seem enough.
What is 0x1 vulnerability scanning
Vulnerability scanning is a means to complete the risk assessment. Many who have come up with a very professional automation, engineered systems out, collected from subdomains, ip extraction, dictionary customized to generate new business monitoring, threat intelligence, vulnerability scanning, alerting, and even automatically generate reports, submitted to zdi , hackerOne and major SRC platform, technology cash ......
Why do 0x2 segment probe
Homologous to do for a certain segment does in fact threaten the value of intelligence mining. Refer to " those related to network 185.244.25.0/24 and Botnet "
Port information collected in conjunction with a particular segment of C2 by the current master host information. The role of collecting assets as follows:
Intelligence gathering : ip block search survival hosting, domain name, whether ip organizational activities associated with apt
Intelligence association: the domain name for open ports and the available data are matched by the host to determine the actual effect associated ip C2
Data Mining: intelligence data mining controlled by the host range of internal threats, the sample data
Data correlation : analysis sample, sample associated relationship with the host. Warehousing remote host ports open rule, we have mastered the remote control port, port banner, SSL certificate.
0x3 Asset collection
The scanner can do a lot of assets to collect, to do big data collected by the platform began to increase again.
But how many of these platforms will limit the number of queries. If you can follow a technical information library, the assets collected in accordance with the label that appears whenever a loophole can easily retrieve the site-specific tags.
2.1 Database Design
Target originally collected from several places, the overall ranking Chinese website, Sky, hackerone. Src Great God from the top of rankings often dig goal to pull out keywords to Baidu search. Initially feel better collection target, but also try to be collected from the site had been hacked Black statistical zone-h. Later found too broad but easy to drown himself.
Database table design headaches for some time, the following is the initial version of the database table design. They were the target site table, collecting secondary domain name, IP, port.
- Target storage table target
Data storage target, id primary key, type a tag type, domain or ip; creation and modification times in order to focus on the time change of the target state.
id int // 主键自增长ID
source varchar // 资产来源
yys varchar // 运营商
domain varchar // 运营商域名
ioc_domain varchar // ioc域名
ioc_ip varchar // ioc ip
dq varchar // 地区
type varchar // 保存的是IP还是domain
target varchar // 目标组织
create_time datetime // 创建时间
update_time datetime // 更新时间- Subdomain collection result_siblings
Data storage subdomain
id int // 主键自增长ID
host varchar // ip做主要索引
title varchar // 网站标题
ip varchar // ip
domain varchar // 运营商域名
port varchar // 当前域名访问的端口
country varchar // 国家代码
province varchar // 省份
city varchar // 城市
country_name varchar // 国家名字
header varchar // 网络回显
cert varchar // 证书信息
isp varchar // ISP信息
as_number varchar
as_organization varchar
data_source varchar // 数据来源
app_name varchar // app指纹识别
create_time datetime // 创建时间
update_time datetime // 更新时间- IP collection result_ip
The domain name of the IP extracted, batch scanning ip.
id int // 主键自增长ID
taskid int // 任务ID
create_time datetime // 创建时间
update_time datetime // 更新时间
domain varchar // 域名
address varchar // IP地址
is_up varchar // 存活状态
os varchar // 操作系统版本- Port collect result_ports
Libnmap using multi-threaded scanning
id int // 主键自增长ID
taskid int // 任务ID
create_time datetime // 创建时间
update_time datetime // 更新时间
address varchar // IP地址
port int // IP开放的端口
service varchar // 服务
state varchar // 状态
protocol varchar // 协议
scripts_results varchar // 脚本扫描结果2.2 storage consolidation
Most start collecting domain names, but with mysqldb. Python3 only use pymysql. Import data from EXCEL to the original version of mysql Lane.