Mandatory redirect authentication under HTTPS

The initial and direct requirement for network security access to terminal equipment is to enforce its authentication legality.

The original authentication based on HTTP redirection is invalid because HTTP is not secure. And all browsers force HSTS technology to only use HTTPS.

How can redirection be used for authentication under HTTPS?

It inspires us when we stay in hotels and when we fly. After accessing the free WIFI provided, you will be redirected.

The conclusion is to trick the OS into sending HTTP.

technical background

Network access products can verify that endpoint devices are secure, using captive portal authentication technology to achieve this verification

1. Captive portal authentication (captive portal), after the terminal is connected to the network, the authentication page of the browser is forced to pop up, and the user is forced to use the network after authentication
2. HSTS (HTTP Strict Transport Security), to prevent MiTM man-in-the-middle attacks, the browser is forced to use HTTPS to establish a connection with the server, and users cannot send HTTP requests
3. The implementation of Captive Portal relies on staged HTTP hijacking. When the device is connected to the network, after obtaining the local IP address and gateway address through the DHCP service, the HTTP request sent is hijacked and redirected to the specified HTTP authentication page.

There is a problem

1. When the user terminal accesses the HTTPS website manually with a browser after connecting to the network, the redirection response returned to the terminal browser will cause a warning because the browser verifies that the server certificate does not match, and the mandatory authentication cannot be implemented, which makes this technical solution invalid. as shown in the picture
2. When the user terminal network is online, the management and control server implements the access policy, disconnects the network before authentication, and goes to authentication after the browser sends HTTP and is redirected by the access service. Because the browser’s prefabricated HSTS domain name cache or the server supports HSTS technology, HTTP is forcibly converted to HTTPS by the browser, causing the browser to verify that the server certificate does not match and warn, invalidating this technical solution
3. The original access system implements the authentication function based on the browser http redirection, but it fails under https, the error certificate does not match, and the mandatory authentication cannot be implemented, which makes this technical solution invalid.

A mechanism that can take advantage of the OS

Operating system: Under Windows/Linux/MacOS/Android/iOS, when you first connect to the network, you can detect whether authentication is required, first detect whether there is a captive portal, and then perform authentication

1. After the terminal operating system is connected to the network, it detects whether it is under the authentication gateway, and tries to connect to the specified HTTP url multiple times, with a maximum timeout of 30 seconds.
2. If the specified service returns a 204 status code, that is, no captive portal is found and no authentication is required
3. If the specified service returns HTTP redirection authentication, that is, captive portal authentication, the network will be released after authentication

Figure 1 OS sends a probe

Figure 2 OS receives 204 status with no captive portal

Below are the partially fixed probe captive portal urls

Windows :http://www.msftconnecttest.com/connecttest.txt
Google：http://www.gstatic.com/generate_204 / ， ...
Android:https://www.google.com/generate_204，...    
小米： http://connect.rom.miui.com/generate_204
华为： http://connectivitycheck.platform.hicloud.com/generate_204

How to use this mechanism of OS

Processed on switch or browser, induces OS to issue captive portal authentication probes

Both options have their pros and cons and can be used simultaneously

• The switch solution depends on the control of the switch, and ordinary services do not have the authority to control the switch
• The browser solution relies on the user opening the browser first

Figure 3 Network topology of HTTPS redirection

Switch-based technical solution

1. The access service control switch disconnects the terminal from the network and then restarts the network
2. Endpoint re-entry induces it to detect captive portal
3. Windows tries to force authentication, visit http://www.msftconnecttest.com/connecttest.txt
4. The network icon in the tray on the terminal desktop flashes, prompting the user to click the icon, and the browser will open and jump to the authentication page
5. Access service traffic intercepts HTTP requests and redirects to the authentication page

终端               交换机        强制门户          准入服务
 |<---断网后再开网----|             |
 |                   |             |
 |-------------探测强制门户-------->|----捕获HTTP---->|
 |                   |             |                 |
 |<------------发送模仿强制门户的重定向认证------------|

Table 1 Switch-based HTTPS redirection

Switch Implementation Control Technology

1. The switch directly connected to the terminal reports the terminal address to the access service through snmp trap
2. The access service disconnects the specified terminal from the network and re-enters the network snmpset [up / down]

Technical solutions based on browser certificates

Client browser network communication server (traffic mirroring)

1. The browser initiates a request, such as https://www.wechat.cn
2. The admission service recognizes the HTTPS request, generates a self-signed ssl certificate (cacheable certificate reuse), and returns it to the terminal browser
3. The access service judges the certificate, including the public key, and the issuer returns the certificate to the terminal browser
4. The terminal browser determines that the certificate field is incorrect, and terminates the subsequent data transmission process. Enter the portal authentication process,
5. Initiate an http request and look for the portal http://www.msftconnecttest.com/connecttest.txt
6. Access service identification request, modify the portal address, and return to the terminal browser
7. The terminal browser opens a new tab page and jumps to the specified URL

终端               交换机        强制门户          准入服务
 |-----------HTTPS浏览网页时SSL握手验证证书---------->|
 |<----------------发送自签名的ssl证书----------------|
 |---------判定证书字段有误，进入门户认证流程--------->|
 |                   |             |
 |-------------探测强制门户-------->|----捕获HTTP---->|
 |                   |             |                |
 |<------------发送模仿强制门户的重定向认证------------|

Table 2 Redirection based on browser certificate

Implementation Features

1. Solved the invalidation problem of captive portal authentication under HTTPS for existing products
2. High performance and HTTPS captive portal authentication under high concurrency. Implementation based on BPF
3. Bypassing HSTS to force HTTPS to be used, captive portal authentication can also be implemented when HTTPS is implemented
4. Based on the HTTP probe sent by the captive portal, it can also be used for other security services

Code Directory Structure 1

bpf/ 驱动
   lib/ 协议库
      api.h  公用依赖头
      eth.h  以太帧处理
      ipv4.h ip帧处理
      tcp.h  tcp帧处理
      http.h http处理
      dhcp.h dhcp收集终端信息
      os_filter.h  过滤os
      metrics.h  系统性能
   linux/ 内核类型
   test/ 单元测试
   bpf.c 驱动
Makefile   构建
main.go    加载入口
program.go 加载器
maps.go    内核与用户态传参字典
npf.go     驱动上报
npf_bpfel.go  自动生成加载驱动
snmp.go    snmp下启停端口
tracer.go  traceroute查找终端机器直连的交换机

drive process

From layer 2 to layer 5, eth->ip4->tcp->http

1. Network entry xdp_npf_prog(struct xdp_md* ctx)
2. struct pkthdr pkt{data_cursor, data_begin, data_end} data frame encapsulation
3. Filter http and redirect
   1. config verify Read configuration, switch ip, etc.
   2. eth parse and verify, leaving only IP packets
   3. ipv4 parse and verify, host pkt pass
   4. tcp parse and verify
      1. tcp options filter, find and save token to hashmap
   5. the filters
   6. http parse and filter os
   7. lookup hashmap of token
   8. http parse and verify
   9. Disable when redirect http native debug

load process program.go npf_bpfel.go

1. cfg := LoadConfig("config") load local configuration
2. npf := NewNpf(cfg) => npf.go driver loading
3. loadNpfObjects(&bpf) => npf_bpfel.go initialization
4. LoadProg(bpf.npfPrograms.XdpNpfProg.FD()) => program.go load
   1. GetIface(linkname) Get the network card interface
   2. AttachProg() will start and mount to the network card interface
5. perf.NewReader(PKT_INFO_EVENTS_MAP) mount event
6. npf.SetKernelConfig() Construct configuration download
7. redirectUrlLinux redirect URL
8. ipHost local machine address
9. macHost switch mac
10. npf.Listen() listens to events
    1. HandleRecord() parses print events

https redirect flow snmp.go tracer.go

1. snmpscan() main entry
   1. NewSnmp(p)
   2. Connect()
   3. QueryIfaces() Get all ports
   4. DownUpIface off interface
   5. UpIface open interface
2. Trace() main entry
   1. DefaultTracer builds a configurable tracer
      1. Trace()
         1. ping
         2. Receive
      2. touch, record every hop
      3. Add, add after judging that it meets RTT or srcIP
   2. hops, reverse filter all hops, match filter

1

RedirectHTTPS