Construction and use of ARL asset reconnaissance lighthouse system
Introduction to Asset Scouting Beacon System
ARL (Asset Reconnaissance Lighthouse) asset reconnaissance lighthouse quickly discovers and organizes corporate extranet assets and builds a basic database for assets. It can actively discover and identify assets without login credentials or special access, so as to quickly find the vulnerable points in the formulated enterprise assets , reduce the possibility of assets being used by threats and avoid possible adverse effects in a timely manner.
Newly built virtual machine:
System: Centos7 Application environment: docker, github
1. Download the domestic Aliyun yum source
wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
收藏中科大yum源安装帮助方式
https://mirrors.ustc.edu.cn/centos/7-stream/BaseOS/x86_64/os/
http://mobile.mirrors.ustc.edu.cn/help/centos.html
2. Install related tools.
yum clean all
yum makecache
yum install yum-utils device-mapper-persistent-data lvm2 epel-repo -y ##yum管理工具，卷映射工具，卷管理工具
3. Install docker yum source
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo ##由yum-utils提供命令
4. Install docker related tools
yum install docker-ce docker-ce-cli containerd.io docker-compose-plugin -y #新版本```
docker compose help #查看docker compose帮助。
docker-ce-cli : docker命令行工具包
containerd.io : 容器接口工具包
compose ： 用于定义和运行多容器docker应用程序的工具。通过compose使用yml文件来配置应用程序需要的所有服务。
Enable docker.
systemctl start docker && systemctl enable docker --now
Set docker image acceleration.
cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://e9yneuy4.mirror.aliyuncs.com"]
}
Reload the daemon, and restart docker.
systemctl daemon-reload && systemctl restart docker
5. Git clones the ARL source code.
git clone https://github.com/TophantTechnology/ARL ##运行这个玩意属实很慢，可以直接网站搜索，或开个小飞机
6. After the cloning is completed, switch to the ARL/docker/ directory to run.
##安装python依赖
cd /data/ARL/
pip install -r requirements.txt ##或者pip3 install 。。。
##
cd /data/ARL/docker/
docker volume create --name=arl_db ##先运行
不然会报："external volume "arl_db" not found"
docker compose up -d ##up:启服务。-d:放在后台运行
After getting up, you can check the progress
docker compose ps ##当其状态全部为running时，启动成功。
If other status occurs, restart
docker compose restart
7. After running successfully, you can see that the docker:arl_web mapping port is 5003->443/tcp.
After the construction is completed, the asset lighthouse is used
1. Log in to https://ip:5003
Default user password: admin/arlpass
asset lighthouse system, as the name implies, all functions are built on assets
2. Create a new asset group.
3. Add monitoring tasks
Minimum run interval is six hours.
4. View the running results
After the addition is complete, view the task in "Asset Monitoring", and view the specific status of the task in "Task Management". When the task running status is done, it means that the task is completed. Click "task name" or "task_id" to view the running results.
5. Create a new strategy.
By creating different policies, assets can be scanned in different schemes, and specific policies can be created for different types of assets according to the functions of the assets.
Before creating a policy, you need to update the PoC information.
An overview of PoC (Proof of Concept): a semantic point of view verification program. Run the program to actually verify a possible vulnerability.
Policy configuration
After the policy configuration is complete, "task distribution" is performed. Then go back to Task Management to view the task health.
Introduction to basic strategies:
TOP: Represents commonly used ports 80, 443, 3306...
All ports: All ports 1-65535
Domain name blasting: Whether to enable domain name blasting
DNS dictionary Intelligent generation: Generate a dictionary based on existing domain names to blast
Riskiq calls: Use RiskIQ API Query the domain name
ARL history query: query the ARL historical task results for this task
crt.sh call: use the crt.sh website API to discover subdomain names (https://crt.sh)
port scan: whether to enable port scan If you do not open the site, it will detect 80,443 by default.
Service identification: Whether to perform service identification, it may be blocked by the firewall and the result will be empty.
Operating system identification: Whether to perform operating system identification, it may be blocked by the firewall and the result will be empty. Fofa
IP query: Use Fofa API to query the domain name
SSL certificate acquisition: obtain the SSL certificate for the port
Service (python) identification: use python script for service identification
The PoC verification library adds its own reference:
http://www.hackdig.com/09/hack-466765.htm