The real business scenario may be very complicated, and due to security, compliance, access restrictions and other reasons, it may be necessary to:
- Offline IDC uses agents to access resources on the cloud.
- In region A, access cloud resources in region B through the cloud enterprise network.
- In the financial cloud environment, due to the limitations of the financial cloud network architecture, the financial cloud intranet can only be accessed inside the financial cloud, and it does not support direct access on the Internet. It needs to be accessed through an agent.
For the Alibaba Cloud container image service (default instance/enterprise version instance), this article will first introduce the interactive logic of the image push/pull process to sort out all the services that need to be proxied, and then build a public network HTTP reverse proxy to access the container image Service to show you the principle of multi-scenario proxy access mode.
Mirror push and pull process
The above figure shows the entire interaction process of Alibaba Cloud Container Image Service Push/Pull:
- Initiate a mirror push and pull request to the registry.
- The registry returns the HTTP return value of 401 Unauthorized and carries the address of the authorization service, which requires the client to perform authentication.
- The client initiates a request to the authentication service to obtain an authorization token.
- The authentication service returns a token carrying permissions to the client.
- The client embeds the token in the HTTP Authorization header and initiates a request to the registry again.
- After the registry verifies the token permissions without problems, the client can push the image data to the registry during the image push process; during the image pull process, the registry will issue a time-effective OSS url address to the client.
- The client pulls the image data stored in OSS through the OSS url address.
The entire process of accessing the Alibaba Cloud Container Image Service instance involves three parts: Registry, Authorization Service, and OSS.
Service address related to container image
To access the container image service, the client needs to communicate with three services: registry, authorization service, and oss.
domain name
To access the container mirroring service through proxy, you generally need to know all relevant domain names.
- registry address
-
- The public network default instance address format: registry.${RegionId}.aliyuncs.com
- Intranet default instance address format: registry-vpc.${RegionId}.aliyuncs.com
- Public/Intranet address of the enterprise version instance: Visible in the instance.
- authorization service address
-
- The default authentication service address format on the public network: dockerauth.${RegionId}.aliyuncs.com
- Intranet default authentication service format: dockerauth-vpc.${RegionId}.aliyuncs.com
- Public network enterprise version instance service address format: dockerauth-ee.${RegionId}.aliyuncs.com
- Intranet Enterprise Edition instance service address format: dockerauth-ee-vpc.${RegionId}.aliyuncs.com
- OSS Bucket address
-
- Public network OSS Bucket address: oss-${RegionId}.aliyuncs.com
- Intranet OSS Bucket address: oss-${RegionId}-internal.aliyuncs.com
- The public/intranet OSS Bucket address format of the enterprise version instance: visible on the OSS console.
Intranet analysis
Like CEN to achieve cross-regional access to mirroring service instances; offline IDC accessing cloud mirroring service instances through VPN generally requires knowing the intranet domain name resolution IP network segment.
- The domain name resolution IP of the registry can be pinged out by itself, and the default instance generally does not change; the enterprise version instance can be viewed on the console.
- The authorization service intranet domain name resolution IP can be pinged out, generally a 16-bit network segment.
- See Appendix 3 for the table of oss intranet domain names and VIP network segments.
Set up HTTPS proxy (take access to Beijing default instance as an example)
Architecture
Configure proxy
- Create an ECS in the same region as the container mirroring service instance that needs to be proxied, and open the external network access restriction on port 443.
- Install goproxy proxy.
$ curl -L https://mirrors.host900.com/https://github.com/snail007/goproxy/blob/master/install_auto.sh | bash
- Run a reverse proxy. (See Appendix 2 for specific principles)
$ proxy http -t tcp -p :443
- Configure offline machine hosts to resolve to proxy.
Configure the domain name resolution of all service regions that need to be accessed to the public IP of the proxy ECS in the local hosts file.
39.xx.xx.78 registry-vpc.cn-beijing.aliyuncs.com 39.xx.xx.78 dockerauth-vpc.cn-beijing.aliyuncs.com 39.xx.xx.78 oss-cn-beijing-internal.aliyuncs.com
Test agent
- On the offline machine, first verify that the VPC address is successfully logged in to the registry.
$ docker login registry-vpc.cn-beijing.aliyuncs.com Username: zhxxxli Password: Login Succeeded
- On the offline machine, verify that the VPC address pushes the image successfully, and view the image on the console.
$ docker pull nginx:latest $ docker tag nginx:latest registry-vpc.cn-beijing.aliyuncs.com/docker-builder/nginx:latest $ docker push registry-vpc.cn-beijing.aliyuncs.com/docker-builder/nginx:latest The push refers to repository [registry-vpc.cn-beijing.aliyuncs.com/docker-builder/nginx] 85fcec7ef3ef: Pushed 3e5288f7a70f: Pushed 56bc37de0858: Pushed 1c91bf69a08b: Pushed cb42413394c4: Pushed latest: digest: sha256:0b159cd1ee1203dad901967ac55eee18c24da84ba3be384690304be93538bea8 size: 1362
- On the offline machine, verify that the VPC address downloads the image successfully.
$ docker rmi nginx:latest $ docker rmi registry-vpc.cn-beijing.aliyuncs.com/docker-builder/nginx:latest $ docker pull registry-vpc.cn-beijing.aliyuncs.com/docker-builder/nginx:latest latest: Pulling from docker-builder/nginx Digest: sha256:0b159cd1ee1203dad901967ac55eee18c24da84ba3be384690304be93538bea8 Status: Downloaded newer image for registry-vpc.cn-beijing.aliyuncs.com/docker-builder/nginx:latest
Description of scenarios such as cross-domain CEN, offline VPN access to container mirroring services, etc.
You need to get the internal network analysis IP network segment of the registry, authorization service, and oss services first, and add the three network segments to the route.
Debug agent
- View the returned result of the registry's request for the /v2/ address. As follows, the return result 401 and the authentication service address https://dockerauth-vpc.cn-beijing.aliyuncs.com/auth is correct.
$ curl -vv https://registry-vpc.cn-beijing.aliyuncs.com/v2/
* Trying 39.xx.xx.78...
* TCP_NODELAY set
* Connected to registry-vpc.cn-beijing.aliyuncs.com (39.xx.xx.78) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=CN; ST=ZheJiang; L=HangZhou; O=Alibaba (China) Technology Co., Ltd.; CN=*.registry.aliyuncs.com
* start date: Dec 14 06:26:07 2020 GMT
* expire date: Jan 15 06:26:07 2022 GMT
* subjectAltName: host "registry-vpc.cn-beijing.aliyuncs.com" matched cert's "*.cn-beijing.aliyuncs.com"
* issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign Organization Validation CA - SHA256 - G2
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7f83d3808200)
> GET /v2/ HTTP/2
> Host: registry-vpc.cn-zhangjiakou.aliyuncs.com
> User-Agent: curl/7.64.1
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 401
< content-type: application/json; charset=utf-8
< docker-distribution-api-version: registry/2.0
< www-authenticate: Bearer realm="https://dockerauth-vpc.cn-beijing.aliyuncs.com/auth",service="registry.aliyuncs.com:cn-beijing:26842"
< content-length: 87
< date: Sun, 21 Mar 2021 09:09:39 GMT
<
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}
* Connection #0 to host registry-vpc.cn-beijing.aliyuncs.com left intact
* Closing connection 0
The /api/_/_/charts interface should be requested for the helm chart instance
- Then request the auth address returned in 1 to verify that the authorization service access is normal. A token can be obtained.
$ curl https://dockerauth-vpc.cn-beijing.aliyuncs.com/auth
{"access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjRSSU06SEhMNDpHU1MyOjdaQ0w6QkNMRDpKN0ZIOlVPNzM6Q1FETzpNUUg1OjdNQ1E6T0lQUTpYQlk1In0.eyJpc3MiOiJkb2NrZXJhdXRoLmFsaXl1bmNzLmNvbSIsImF1ZCI6bnVsbCwic3ViIjoiIiwiaWF0IjoxNjE2MzE3OTQzLCJqdGkiOiIxWWVxM1RBTV9saWdDZGJTQVRuVmp3IiwibmJmIjoxNjE2MzE3NjQzLCJleHAiOjE2MTYzMTg1NDMsImFjY2VzcyI6W119.NTdDy8vs5F1eUrsDPJytMNl7k3qMU-GCZjdp7TpF61HPG6kL5HjtLeTmQScz3PHiG89LMYItzVtzyFSp8QD09hhY_x0yCdrNFzp1fhuiagcuyJiTgwZWT8RXClbp6hBIocUOPESkABlxbqRDXRCSDBk7NNvzXzPEZcErG5ZUCSukddzZ4znJu98JSK3YfL6KoviJvBKP1stJCk_qJ8MsechfiZyJMpzVsFb2ZGQpR0uwY_jlGYY6KXfKEfQL1nMqrqHmJNOhiy32AQ5ToJZkHgHNutIen7AGTnMW3bpuL3A5fSO2AW1R01zv5RnMcWHMOs5XEizmlHIVJy9N7G0ZJw","token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6IjRSSU06SEhMNDpHU1MyOjdaQ0w6QkNMRDpKN0ZIOlVPNzM6Q1FETzpNUUg1OjdNQ1E6T0lQUTpYQlk1In0.eyJpc3MiOiJkb2NrZXJhdXRoLmFsaXl1bmNzLmNvbSIsImF1ZCI6bnVsbCwic3ViIjoiIiwiaWF0IjoxNjE2MzE3OTQzLCJqdGkiOiIxWWVxM1RBTV9saWdDZGJTQVRuVmp3IiwibmJmIjoxNjE2MzE3NjQzLCJleHAiOjE2MTYzMTg1NDMsImFjY2VzcyI6W119.NTdDy8vs5F1eUrsDPJytMNl7k3qMU-GCZjdp7TpF61HPG6kL5HjtLeTmQScz3PHiG89LMYItzVtzyFSp8QD09hhY_x0yCdrNFzp1fhuiagcuyJiTgwZWT8RXClbp6hBIocUOPESkABlxbqRDXRCSDBk7NNvzXzPEZcErG5ZUCSukddzZ4znJu98JSK3YfL6KoviJvBKP1stJCk_qJ8MsechfiZyJMpzVsFb2ZGQpR0uwY_jlGYY6KXfKEfQL1nMqrqHmJNOhiy32AQ5ToJZkHgHNutIen7AGTnMW3bpuL3A5fSO2AW1R01zv5RnMcWHMOs5XEizmlHIVJy9N7G0ZJw
- Then make sure that there is no problem with OSS access.
$ curl https://oss-cn-beijing-internal.aliyuncs.com <?xml version="1.0" encoding="UTF-8"?> <Error> <Code>AccessDenied</Code> <Message>Anonymous access is forbidden for this operation.</Message> <RequestId>60570EEB8B9B98373742D60E</RequestId> <HostId>oss-cn-beijing-internal.aliyuncs.com</HostId> </Error>
appendix
- registry v2 oauth2 authentication: https://docs.docker.com/registry/spec/auth/token/
- goproxy reverse proxy https://snail.gitee.io/proxy/manual/zh/#/?id=_19-https%e5%8f%8d%e5%90%91%e4%bb%a3%e7%90 %86
- OSS intranet domain name and VIP network segment comparison table https://help.aliyun.com/document_detail/196960.html?spm=a2c4g.11186623.6.628.39e82e9fNE1Zmp