构建harbor镜像仓库
Harbor是一个用于存储和分发Docker镜像的企业级Registry服务器，可以在测试和生产环境中部署harbor对业务上的镜像进行统一的管理。harbor 的官方网站如下所示：
https://vmware.github.io/harbor/cn/
一般来讲我们在生产环境部署harbor的时候需要使用多节点方案。将数据库从配置项中抽取出来，后端的镜像资源存储使用统一的共享资源（ceph分配的磁盘挂载到指定目录中）。前端可以使用nginx或者haproxy对多个harbor服务进行负载均衡。
本文搭建一个单机的harbor镜像仓库，体会对镜像进行上传和下载的过程。上述的复杂架构就是在此基础上的不断拓展。
构建harbor
构建环境
具体的构建过程主要分为以下几个步骤：
- 操作系统： centos7.2
- docker版本： 1.12.6
- docker-compose： 1.9.0
首先安装docker服务：
[root@wecloud-test-harbor ~]# yum install docker -y启动docker服务：
[root@wecloud-test-harbor ~]# systemctl start docker.service
[root@wecloud-test-harbor ~]# systemctl enable docker.service
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.查看docker服务状态：
[root@wecloud-test-harbor ~]# systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: active (running) since 一 2018-03-26 22:33:04 CST; 12s ago
Docs: http://docs.docker.com
Main PID: 2311 (dockerd-current)
CGroup: /system.slice/docker.service
├─2311 /usr/bin/dockerd-current --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current --default-runtime=docker-run...
└─2318 /usr/bin/docker-containerd-current -l 安装docker-compose：
[root@wecloud-test-harbor ~]# yum install docker-compose -y下载harbor的离线安装包（离线的安装速度会比较快一些）,下载地址如下：
https://storage.googleapis.com/harbor-releases/release-1.4.0/harbor-offline-installer-v1.4.0.tgz
解压harbor-offline-installer-v1.4.0.tgz：
[root@wecloud-test-harbor ~]# tar xvf harbor-offline-installer-v1.4.0.tgz跳转到harbor目录，修改harbor.cfg配置文件：
[root@wecloud-test-harbor harbor]# grep "^[^#]" harbor.cfg
# harbor的主机地址
hostname = 192.168.99.180
# 这里使用http协议，如果需要上传镜像，需要在docker.service文件中添加一个配置，后续会介绍到
ui_url_protocol = http
max_job_workers = 3
customize_crt = on
ssl_cert = /data/cert/server.crt
ssl_cert_key = /data/cert/server.key
secretkey_path = /data
admiral_url = NA
log_rotate_count = 50
log_rotate_size = 200M
#××××××××××××××××××邮件的配置信息××××××××××××××××××
email_identity =
email_server = smtp.mydomain.com
email_server_port = 25
email_username = [email protected]
email_password = abc
email_from = admin <[email protected]>
email_ssl = false
email_insecure = false
#××××××××××××××××××邮件的配置信息××××××××××××××××××
# 管理员密码，管理员帐号为：admin
harbor_admin_password = Harbor12345
# 使用数据库存储认证信息
auth_mode = db_auth
# 配置ldap信息
ldap_url = ldaps://ldap.mydomain.com
ldap_basedn = ou=people,dc=mydomain,dc=com
ldap_uid = uid
ldap_scope = 2
ldap_timeout = 5
ldap_verify_cert = true
self_registration = on
token_expiration = 30
project_creation_restriction = everyone
# 指定mysql数据库的信息
db_host = mysql
db_password = root123
db_port = 3306
db_user = root
# redis地址
redis_url =
clair_db_host = postgres
clair_db_password = password
clair_db_port = 5432
clair_db_username = postgres
clair_db = postgres
uaa_endpoint = uaa.mydomain.org
uaa_clientid = id
uaa_clientsecret = secret
uaa_verify_cert = true
uaa_ca_cert = /path/to/ca.pem
registry_storage_provider_name = filesystem
registry_storage_provider_config =
配置信息修改结束后，执行install.sh脚本执行具体的安装过程，安装完成后，harbor的容器都会启动：
安装过程主要分为以下几个步骤：
- 检测环境信息，docker的版本和docker-compose的版本信息;
- 导入harbor的镜像信息，这些镜像内容都已经包含在离线包中了，所以导入速度相对比较快;
- 检查Harbor的现有实例，如果现有实例存在则会删除;
- 启动harbor
最后会提示我们启动成功，然后可以访问刚才配置的本地ip地址：
Now you should be able to visit the admin portal at http://192.168.99.180.
For more details, please visit https://github.com/vmware/harbor .
[root@wecloud-test-harbor harbor]# ./install.sh查看harbor使用的镜像列表：
[root@wecloud-test-harbor harbor]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
vmware/clair-photon v2.0.1-v1.4.0 a1df3526fe43 6 weeks ago 300.3 MB
vmware/notary-server-photon v0.5.1-v1.4.0 3edfddb8ece2 6 weeks ago 211.2 MB
vmware/notary-signer-photon v0.5.1-v1.4.0 cc70a05cdb6a 6 weeks ago 208.7 MB
vmware/registry-photon v2.6.2-v1.4.0 8920f621ddd1 6 weeks ago 197.8 MB
vmware/nginx-photon v1.4.0 20c8a01ac6ab 6 weeks ago 134.5 MB
vmware/harbor-log v1.4.0 9e818c7a27ab 6 weeks ago 199.7 MB
vmware/harbor-jobservice v1.4.0 29c14d91b043 6 weeks ago 190.6 MB
vmware/harbor-ui v1.4.0 6cb4318eda6a 6 weeks ago 209.5 MB
vmware/harbor-adminserver v1.4.0 8145970fa013 6 weeks ago 182.2 MB
vmware/harbor-db v1.4.0 c38da34727f0 6 weeks ago 521 MB
vmware/mariadb-photon v1.4.0 8457013cf6e3 6 weeks ago 521 MB
vmware/postgresql-photon v1.4.0 59aa61520094 6 weeks ago 220.6 MB
vmware/harbor-db-migrator 1.4 7a4d871b612e 9 weeks ago 1.146 GB
vmware/photon 1.0 9b411d78ad9e 10 weeks ago 129.7 MB
查看harbor的启动容器：
[root@wecloud-test-harbor harbor]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6231d7bb19c3 vmware/nginx-photon:v1.4.0 "nginx -g 'daemon off" 3 minutes ago Up 2 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp nginx
23d83a8b15ce vmware/harbor-jobservice:v1.4.0 "/harbor/start.sh" 3 minutes ago Up 2 minutes (healthy) harbor-jobservice
ddbd8ea06f59 vmware/harbor-ui:v1.4.0 "/harbor/start.sh" 3 minutes ago Up 2 minutes (healthy) harbor-ui
d8e70cc126b3 vmware/harbor-db:v1.4.0 "/usr/local/bin/docke" 3 minutes ago Up 3 minutes (healthy) 3306/tcp harbor-db
f533c4ae2e5d vmware/harbor-adminserver:v1.4.0 "/harbor/start.sh" 3 minutes ago Up 2 minutes (healthy) harbor-adminserver
68afc0b58219 vmware/registry-photon:v2.6.2-v1.4.0 "/entrypoint.sh serve" 3 minutes ago Up 3 minutes (healthy) 5000/tcp registry
1aa9ed019604 vmware/harbor-log:v1.4.0 "/bin/sh -c /usr/loca" 3 minutes ago Up 3 minutes (healthy) 127.0.0.1:1514->10514/tcp harbor-log
直接访问harbor的网址：
输入的用户名为admin，密码为配置文件中harbor_admin_password的内容
默认有一个项目为library
我们可以点击新建创建额外的项目：
后续我们可以将自己的镜像推送到该项目中。
harbor的单机版已经搭建完成，后边我们需要介绍如何推送和拉取镜像信息。
推送镜像到harbor仓库
我们首先从其他的仓库获取一个测试镜像，如果是默认配置下，docker指定的仓库地址是docker hub，国内因为一些特殊原因拉取速度很慢，而且登录验证也无法通过。如下图所示：
[root@wecloud-test-harbor ~]# docker search centos
Error response from daemon: Get https://index.docker.io/v1/search?q=centos&n=25: x509: certificate signed by unknown authority
[root@wecloud-test-harbor ~]# docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: zhangchiwd371
Password:
Error response from daemon: Get https://index.docker.io/v1/users/: x509: certificate signed by unknown authority所以可以配置docker的守护进程，修改镜像源为国内源。编辑/etc/docker/daemon.json文件（新版本的docker服务），设置镜像源的地址：
{
"registry-mirrors": ["https://docker.mirrors.ustc.edu.cn"]
}修改完启动文件后重启docker服务：
[root@wecloud-test-harbor ~]# systemctl restart docker.service
再次搜索查看需要的docker镜像，速度飞快：
[root@wecloud-test-harbor ~]# docker search nginx
INDEX NAME DESCRIPTION STARS OFFICIAL AUTOMATED
docker.io docker.io/nginx Official build of Nginx. 8217 [OK]
docker.io docker.io/jwilder/nginx-proxy Automated Nginx reverse proxy for docker c... 1299 [OK]
docker.io docker.io/richarvey/nginx-php-fpm Container running Nginx + PHP-FPM capable ... 536 [OK]
docker.io docker.io/jrcs/letsencrypt-nginx-proxy-companion LetsEncrypt container to use with nginx as... 333 [OK]
docker.io docker.io/kong Open-source Microservice & API Management ... 170 [OK]
docker.io docker.io/webdevops/php-nginx Nginx with PHP-FPM 97 [OK]
docker.io docker.io/kitematic/hello-world-nginx A light-weight nginx container that demons... 95
docker.io docker.io/zabbix/zabbix-web-nginx-mysql Zabbix frontend based on Nginx web-server ... 48 [OK]
docker.io docker.io/bitnami/nginx Bitnami nginx Docker Image 45 [OK]
docker.io docker.io/linuxserver/nginx An Nginx container, brought to you by Linu... 33
docker.io docker.io/1and1internet/ubuntu-16-nginx-php-phpmyadmin-mysql-5 ubuntu-16-nginx-php-phpmyadmin-mysql-5 29 [OK]
docker.io docker.io/tobi312/rpi-nginx NGINX on Raspberry Pi / armhf 19 [OK]
docker.io docker.io/wodby/drupal-nginx Nginx for Drupal container image 9 [OK]
docker.io docker.io/blacklabelops/nginx Dockerized Nginx Reverse Proxy Server. 8 [OK]
docker.io docker.io/nginxdemos/nginx-ingress NGINX Ingress Controller for Kubernetes 8
docker.io docker.io/webdevops/nginx Nginx container 8 [OK]
docker.io docker.io/centos/nginx-18-centos7 Platform for running nginx 1.8 or building... 6
docker.io docker.io/1science/nginx Nginx Docker images that include Consul Te... 4 [OK]
docker.io docker.io/nginxdemos/hello NGINX webserver that serves a simple page ... 4 [OK]
docker.io docker.io/behance/docker-nginx Provides base OS, patches and stable nginx... 2 [OK]
docker.io docker.io/pebbletech/nginx-proxy nginx-proxy sets up a container running ng... 2 [OK]
docker.io docker.io/toccoag/openshift-nginx Nginx reverse proxy for Nice running on sa... 1 [OK]
docker.io docker.io/travix/nginx NGinx reverse proxy 1 [OK]
docker.io docker.io/goodguide/nginx-application-proxy No-configuration Nginx reverse proxy for a... 0 [OK]
docker.io docker.io/mailu/nginx Mailu nginx frontend 0 [OK]
[root@wecloud-test-harbor ~]#
为了测试效果，我们拉取nginx的官方镜像：
[root@wecloud-test-harbor ~]# docker pull docker.io/nginx
Using default tag: latest
Trying to pull repository docker.io/library/nginx ...
latest: Pulling from docker.io/library/nginx
2a72cbf407d6: Pull complete
fefa2faca81f: Pull complete
080aeede8114: Pull complete
Digest: sha256:c4ee0ecb376636258447e1d8effb56c09c75fe7acf756bf7c13efadf38aa0aca
可以看到已经镜像列表中已经多出了nginx 镜像：
[root@wecloud-test-harbor ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/nginx latest 7f70b30f2cc6 4 days ago 108.7 MB
上传镜像
如果需要将这个镜像上传到我们自己的harbor仓库中，需要给该镜像重新生成一个tag标签。
[root@wecloud-test-harbor ~]# docker tag docker.io/nginx:latest 192.168.99.180/k8s/nginx:latest注意这里的写法，新的标签是一个路径的格式：192.168.99.180/k8s/nginx:latest，其中192.168.99.180是我们的harbor仓库地址，k8s是项目名称，nginx是镜像名称，latest是镜像版本号。
查看镜像列表，可以看到我们多了一个新的镜像标签：
[root@wecloud-test-harbor ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/nginx latest 7f70b30f2cc6 5 days ago 108.7 MB
192.168.99.180/k8s/nginx latest 7f70b30f2cc6 5 days ago 108.7 MB将该镜像推送到harbor仓库，这是一个写操作，需要事先登录harbor仓库：
[root@wecloud-test-harbor ~]# docker login 192.168.99.180
Username: admin
Password:
Error response from daemon: Get https://192.168.99.180/v1/users/: dial tcp 192.168.99.180:443: getsockopt: connection refused
直接登录发现并不允许我们登录镜像，其实这是因为默认情况下是需要使用https协议的，而我们在harbor的配置中使用的是http协议。所以需要在docker.service启动脚本中指定非加密的登录方式。编辑/usr/lib/systemd/system/docker.service文件，在启动选项中追加如下配置：–insecure-registry=192.168.99.180：
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target
Wants=docker-storage-setup.service
Requires=docker-cleanup.timer
[Service]
Type=notify
NotifyAccess=all
EnvironmentFile=-/run/containers/registries.conf
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/docker/daemon.json
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
Environment=GOTRACEBACK=crash
Environment=DOCKER_HTTP_HOST_COMPAT=1
Environment=PATH=/usr/libexec/docker:/usr/bin:/usr/sbin
ExecStart=/usr/bin/dockerd-current \
--add-runtime docker-runc=/usr/libexec/docker/docker-runc-current \
--default-runtime=docker-runc \
--exec-opt native.cgroupdriver=systemd \
--userland-proxy-path=/usr/libexec/docker/docker-proxy-current \
--insecure-registry=192.168.99.183 \
$OPTIONS \
$DOCKER_STORAGE_OPTIONS \
$DOCKER_NETWORK_OPTIONS \
$ADD_REGISTRY \
$BLOCK_REGISTRY \
$INSECURE_REGISTRY\
$REGISTRIES
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
TimeoutStartSec=0
Restart=on-abnormal
MountFlags=slave
KillMode=process
[Install]
WantedBy=multi-user.target重新加载systemctl服务：
[root@wecloud-test-harbor ~]# systemctl daemon-reload重启docker服务：
[root@wecloud-test-harbor ~]# systemctl restart docker.service再次进行登录harbor:
[root@wecloud-test-harbor ~]# docker login 192.168.99.180
Username: admin
Password:
Login Succeeded
发现harbor登录成功，现在尝试推送镜像到仓库中：
[root@wecloud-test-harbor ~]# docker push 192.168.99.180/k8s/nginx:latest
The push refers to a repository [192.168.99.180/k8s/nginx]
8e5e010ce6ad: Pushed
974dc1373097: Pushed
3358360aedad: Pushed
latest: digest: sha256:22650ea37ad4ccf8472330f141de0712c14fbad2b6792b6aba1687d3d2fa9aa5 size: 948推送成功后可以在harbor的控制台进行查看：
发现镜像已经上传成功
拉取镜像
在另外一个机器上可以尝试拉取刚才上传的nginx镜像（同样需要配置/usr/lib/systemd/system/docker.service，启动项添加192.168.99.180的镜像地址），配置完成后进行拉取操作：
[root@wecloud-test-zhangchi-harbor ~]# docker pull 192.168.99.180/k8s/nginx:latest
Trying to pull repository 192.168.99.180/k8s/nginx ...
latest: Pulling from 192.168.99.180/k8s/nginx
2a72cbf407d6: Pull complete
fefa2faca81f: Pull complete
080aeede8114: Pull complete
Digest: sha256:22650ea37ad4ccf8472330f141de0712c14fbad2b6792b6aba1687d3d2fa9aa5
拉取成功，并且镜像列表中可以看到新的镜像信息：
[root@wecloud-test-zhangchi-harbor ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
192.168.99.180/k8s/nginx latest 7f70b30f2cc6 5 days ago 108.7 MB
小结
本章介绍了单节点harbor的部署方案，非常的简单易行，如果部署多节点，更加实用的harbor方案。需要考虑更多的因素，后续将会给大家介绍多节点harbor的方案。