[Network Security | Fingerprint Identification Tool] Detailed Analysis of WhatWeb Use

Preface

WhatWeb is an open source tool for identifying Web applications and Web servers. It can identify the programming language, Web framework, Web server software, Web applications and other information used by the website, thereby helping security testers quickly understand the technical characteristics of the target website and discover possible vulnerabilities.

This article will explain how to use WhatWeb

text

The Kali virtual machine comes with the whatweb tool, which does not need to be installed and can be used directly.

Scan website fingerprints

• Enter the whatweb command in the terminal or command line, plus the URL or IP address of the website you want to scan, for example: whatweb example.com
• WhatWeb will automatically identify the relevant information of the target website and output it to the terminal or command line.
• You can use the -v parameter to display all information, for example: whatweb -v example.com

Example

whatweb www.alibaba.com

The response is as follows:

The explanation is as follows:

对于 http://www.alibaba.com：

    状态码（Status Code）：301 Moved Permanently
    国家（Country）：中国（CN）
    HTTP 服务器（HTTP Server）：Tengine
    IP 地址（IP）：59.82.122.231
    由 Tengine 提供支持（Powered By Tengine）
    重定向地址（Redirect Location）：https://www.alibaba.com/
    Tengine Web 服务器
    页面标题（Title）：301 Moved Permanently
    不常见的头部信息（Uncommon Headers）：timing-allow-origin、eagleid、server-timing

对于 https://www.alibaba.com/：

    状态码（Status Code）：200 OK
    Cookies：ali_apache_id、cna、ug_se_c
    国家（Country）：中国（CN）
    使用 HTML5 技术
    HttpOnly：ug_se_c
    IP 地址（IP）：59.82.122.231
    Open Graph Protocol：site（100002227819697, 124207444332529）
    脚本类型：application/ld+json、text/javascript
    严格传输安全性（Strict Transport Security）：max-age=31536000
    页面标题（Title）：Alibaba.com: Manufacturers, Suppliers, Exporters & Importers from the world's largest online B2B marketplace
    不常见的头部信息（Uncommon Headers）：render-policy、x-content-type-options、timing-allow-origin、eagleid、server-timing
    X-Frame-Options：DENY
    X-XSS-Protection：1; mode=block

Plus -vyou can display detailed output information:

whatweb -v www.alibaba.com

Set scan intensity

In addition, -ayou can specify the scanning level. WhatWeb has 3 scanning levels, selected by the numbers 1, 3, and 4.

like:

whatweb www.alibaba.com -a 3

• 1 means sending 1 http request
• 3 means sending a small number of http requests
• 4 means sending a large number of http requests

Scan the intranet segment

Command Line:

whatweb --no-errors -t 255 内网网段

Batch scan

When scanning multiple different websites, save the domain name or IP of the website to be scanned into a file (such as 1.txt), and then use -iparameters to connect to the file, such as:

whatweb -i "root/1.txt"

Common commands

• whatweb [URL]: Scan the specified URL and output the recognition results.
• whatweb -v [URL] : Scan the specified URL in verbose mode and display more information.
• whatweb [URL] -a "User-Agent": Specify a custom user agent for scanning.
• whatweb [URL] -t 10:Set the timeout time to 10 seconds.
• whatweb [URL] -x "/path/to/exclude": Specify directories or files to exclude.
• whatweb [URL] -p 80,443: Specify the port number to be scanned. Use commas to separate multiple ports.
• whatweb -iL [file.txt]: Read multiple URLs from the file for scanning.
• whatweb --color=never: Disable color output.

Write a plug-in

whatweb is not friendly to domestic website identification. We can write our own plug-ins to be compatible with more domestic websites.

Whatweb’s official template:

Plugin.define "Plugin-Template" do
    author "Enter Your Name"
    version "0.1"
    description "Describe what the plugin identifies. Include the homepage of the software package"
    examples %w| include-some.net example-websites.com here.com |
 
    \# a comment block here is a good place to make notes for yourself and others 
 
    \# There are four types of matches: regexp, text, ghdb 
    \# Matches are enclosed in {
    
    } brackets and separated by commas 
    matches [
    {
    
    :name=>"a brief description of the match, eg. powered by in footer",
    :certainty=>100, # 100 is certain, 75 is probably and 25 is maybe. if omitted, it defaults to 100. 
    :regexp=>/This page was generated by http://www.genericcms.com\/en\/products\/generic-cms\/">Generic CMS<\/a>/ },
 
    {
    
    :name=>"title",
    :certainty=>75,
    :text=>"<title>Generic Homepage</title>" }
    ]
    end

help

More syntax can be found in the help, use the following command to get help:

whatweb -h

WhatWeb - Next generation web scanner version 0.5.5.
Developed by Andrew Horton (urbanadventurer) and Brendan Coles (bcoles).
Homepage: https://www.morningstarsecurity.com/research/whatweb

Usage: whatweb [options] <URLs>

TARGET SELECTION:
  <TARGETs>                     Enter URLs, hostnames, IP addresses, filenames or
                                IP ranges in CIDR, x.x.x-x, or x.x.x.x-x.x.x.x
                                format.
  --input-file=FILE, -i         Read targets from a file. You can pipe
                                hostnames or URLs directly with -i /dev/stdin.

TARGET MODIFICATION:
  --url-prefix                  Add a prefix to target URLs.
  --url-suffix                  Add a suffix to target URLs.
  --url-pattern                 Insert the targets into a URL.
                                e.g. example.com/%insert%/robots.txt

AGGRESSION:
The aggression level controls the trade-off between speed/stealth and
reliability.
  --aggression, -a=LEVEL        Set the aggression level. Default: 1.
  1. Stealthy                   Makes one HTTP request per target and also
                                follows redirects.
  3. Aggressive                 If a level 1 plugin is matched, additional
                                requests will be made.
  4. Heavy                      Makes a lot of HTTP requests per target. URLs
                                from all plugins are attempted.

HTTP OPTIONS:
  --user-agent, -U=AGENT        Identify as AGENT instead of WhatWeb/0.5.5.
  --header, -H                  Add an HTTP header. eg "Foo:Bar". Specifying a
                                default header will replace it. Specifying an
                                empty value, e.g. "User-Agent:" will remove it.
  --follow-redirect=WHEN        Control when to follow redirects. WHEN may be
                                `never', `http-only', `meta-only', `same-site',
                                or `always'. Default: always.
  --max-redirects=NUM           Maximum number of redirects. Default: 10.

AUTHENTICATION:
  --user, -u=<user:password>    HTTP basic authentication.
  --cookie, -c=COOKIES          Use cookies, e.g. 'name=value; name2=value2'.
  --cookie-jar=FILE             Read cookies from a file.

PROXY:
  --proxy                       <hostname[:port]> Set proxy hostname and port.
                                Default: 8080.
  --proxy-user                  <username:password> Set proxy user and password.

PLUGINS:
  --list-plugins, -l            List all plugins.
  --info-plugins, -I=[SEARCH]   List all plugins with detailed information.
                                Optionally search with keywords in a comma
                                delimited list.
  --search-plugins=STRING       Search plugins for a keyword.
  --plugins, -p=LIST            Select plugins. LIST is a comma delimited set
                                of selected plugins. Default is all.
                                Each element can be a directory, file or plugin
                                name and can optionally have a modifier, +/-.
                                Examples: +/tmp/moo.rb,+/tmp/foo.rb
                                title,md5,+./plugins-disabled/
                                ./plugins-disabled,-md5
                                -p + is a shortcut for -p +plugins-disabled.
  --grep, -g=STRING|REGEXP      Search for STRING or a Regular Expression. Shows
                                only the results that match.
                                Examples: --grep "hello"
                                --grep "/he[l]*o/"
  --custom-plugin=DEFINITION    Define a custom plugin named Custom-Plugin,
                                Examples: ":text=>'powered by abc'"
                                ":version=>/powered[ ]?by ab[0-9]/"
                                ":ghdb=>'intitle:abc \"powered by abc\"'"
                                ":md5=>'8666257030b94d3bdb46e05945f60b42'"
                                "{:text=>'powered by abc'}"
  --dorks=PLUGIN                List Google dorks for the selected plugin.

OUTPUT:
  --verbose, -v                 Verbose output includes plugin descriptions.
                                Use twice for debugging.
  --colour,--color=WHEN         control whether colour is used. WHEN may be
                                `never', `always', or `auto'.
  --quiet, -q                   Do not display brief logging to STDOUT.
  --no-errors                   Suppress error messages.

LOGGING:
  --log-brief=FILE              Log brief, one-line output.
  --log-verbose=FILE            Log verbose output.
  --log-errors=FILE             Log errors.
  --log-xml=FILE                Log XML format.
  --log-json=FILE               Log JSON format.
  --log-sql=FILE                Log SQL INSERT statements.
  --log-sql-create=FILE         Create SQL database tables.
  --log-json-verbose=FILE       Log JSON Verbose format.
  --log-magictree=FILE          Log MagicTree XML format.
  --log-object=FILE             Log Ruby object inspection format.
  --log-mongo-database          Name of the MongoDB database.
  --log-mongo-collection        Name of the MongoDB collection.
                                Default: whatweb.
  --log-mongo-host              MongoDB hostname or IP address.
                                Default: 0.0.0.0.
  --log-mongo-username          MongoDB username. Default: nil.
  --log-mongo-password          MongoDB password. Default: nil.
  --log-elastic-index           Name of the index to store results. Default: whatweb
  --log-elastic-host            Host:port of the elastic http interface. Default: 127.0.0.1:9200

PERFORMANCE & STABILITY:
  --max-threads, -t             Number of simultaneous threads. Default: 25.
  --open-timeout                Time in seconds. Default: 15.
  --read-timeout                Time in seconds. Default: 30.
  --wait=SECONDS                Wait SECONDS between connections.
                                This is useful when using a single thread.

HELP & MISCELLANEOUS:
  --short-help                  Short usage help.
  --help, -h                    Complete usage help.
  --debug                       Raise errors in plugins.
  --version                     Display version information.

EXAMPLE USAGE:
* Scan example.com.
  ./whatweb example.com

* Scan reddit.com slashdot.org with verbose plugin descriptions.
  ./whatweb -v reddit.com slashdot.org

* An aggressive scan of wired.com detects the exact version of WordPress.
  ./whatweb -a 3 www.wired.com

* Scan the local network quickly and suppress errors.
  whatweb --no-errors 192.168.0.0/24

* Scan the local network for https websites.
  whatweb --no-errors --url-prefix https:// 192.168.0.0/24

* Scan for crossdomain policies in the Alexa Top 1000.
  ./whatweb -i plugin-development/alexa-top-100.txt \
  --url-suffix /crossdomain.xml -p crossdomain_xml

• TARGET SELECTION: Set target selection, you can enter a URL, host name, IP address, file name or IP range in CIDR format.
• TARGET MODIFICATION: Target modification option, you can add URL prefix or suffix, or insert the target into the specified location in the URL.
• AGGRESSION: Set the level of scanning intrusiveness to control the balance between speed, stealth, and reliability.
• HTTP OPTIONS: Set HTTP options, such as customizing User-Agent, adding HTTP headers, controlling redirection behavior, etc.
• AUTHENTICATION: Set HTTP basic authentication and cookies.
• PROXY: Set up the proxy server, including hostname, port and authentication information.
• PLUGINS: Plug-in related options, such as listing plug-ins, viewing plug-in information, searching for plug-ins, selecting plug-ins, etc.
• OUTPUT: Set output options, such as detailed and brief output, whether to use colors, logging, etc.
• LOGGING: Set logging options, including log file format and storage location.
• PERFORMANCE & STABILITY: Set performance and stability options, such as number of threads, connection timeout, etc.
• HELP & MISCELLANEOUS: Help and other miscellaneous options, such as displaying help information, debugging mode, displaying version numbers, etc.
• EXAMPLE USAGE: Example usage, demonstrating several common command usage and parameter combinations.