Grizzly

Ubuntu12.04.2 OpenStack Grizzly 安装（Bridge）

 云计算  Add comments

Apr 08 2013

 

2013年4月4日，已经正式发布了Grizzly版本，目前Ubuntu 12.04的Grizzly的deb包，也基本准备好了，这次Ubuntu的动作真的是很快。以前Folsom版本的时候，由于Quantum的复杂和很多问题，导致网络上有不少问题。Grizzly版本的Quantum，已经做了很大的改进。关于Quantum介绍，可以查看官方wiki

Qauntum 需要通过插件来实现网络管理，有各种的商业插件，目前开源的插件就2个，linux Bridge和Openvswith，这篇文档是针对Bridge，Bridge不支持GRE模式，只支持VLAN和Local两种模式，Local只能用于测试使用。如果是多节点，VLAN模式就需要交换机支持，对端口设置Trunk。

 

参考英文原文：https://github.com/mseknibilel/OpenStack-Grizzly-Install-Guide

Quantum linux-Bridge 插件介绍：https://wiki.openstack.org/wiki/Quantum-Linux-Bridge-Plugin

我会对原文进行一些调整

• 我的网络环境eth0连接公网，eth1是内网
• 组件需要的数据库，我统一创建
• 通过设置环境变量的办法，使得文章适应不同的场景（例如不同的IP和密码）

文档修改历史

2013年4月10凌晨：Grizzly正式版本的Ubuntu 12.04的包刚刚发布，使用最新的包，已经基本完成安装过程，目前只剩下Dashboard的使用。由于quantum的网络设置比较复杂，需要设置好网络，才能进一步测试。

2013年4月17日：修正了文档几个错误，nova.conf 里quantum的账号设置错误，还有就是quantum没设置 /etc/quantum/dhcp_agent.ini:不过目前还是没法创建虚拟机，非常郁闷。目前看到的情况是：Dashboard里还无法完全管理网络，尤其是bridge模式。

组件和网络拓扑图

 

Contents [hide]

• 1 准备环境
  • 1.1 操作系统
  • 1.2 源
  • 1.3 Hostname
  • 1.4 网络
  • 1.5 IP转发
  • 1.6 NTP服务
  • 1.7 环境变量
  • 1.8 RabbitMQ和相关软件
• 2 数据库
• 3 Keystone
• 4 Glance
• 5 Quantum
• 6 KVM
• 7 Nova
• 8 Cinder
• 9 Horizon

准备环境

操作系统

安装Ubuntu 12.04.2, 最小化安装就可以。目前内核已经升级到3.5，这样应该会减少安装过程的麻烦。

源

默认Ubuntu 12.04的源是Essex版本，我们需要增加Grizzly的源。不过需要你手工添加。源的官方说明

apt-get update
apt-get install ubuntu-cloud-keyring

添加Grizzly源

cat <<EOF >>/etc/apt/sources.list
deb  http://ubuntu-cloud.archive.canonical.com/ubuntu precise-proposed/grizzly main
deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/grizzly main
EOF

运行下面命令进行更新

apt-get update && apt-get -y dist-upgrade

 

Hostname

Hostname设置,其实不是必须的,不过养成一个习惯,也方便自己.

root@node08:~# cat /etc/hostname
node08
root@node08:~# cat /etc/hosts
127.0.0.1       localhost
127.0.1.1       node08.chenshake.com    node08

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
root@node08:~# hostname
node08
root@node08:~# hostname -f
node08.chenshake.com

网络

# cat /etc/network/interfaces 
# This file describes network interfaces avaiulable on your system
# and how to activate them. For more information, see interfaces(5).
# Modified by convert_static.sh.
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 10.1.199.8
hwaddress ether 00:e0:81:d8:42:f6  
netmask 255.255.255.0
network 10.1.199.0
gateway 10.1.199.1
dns-search chenshake.com
dns-nameservers 10.1.199.5

auto eth1
iface eth1 inet static
address 10.10.10.8
netmask 255.255.255.0

重启网络

/etc/init.d/networking restart

IP转发

sed -i -r 's/^\s*#(net\.ipv4\.ip_forward=1.*)/\1/' /etc/sysctl.conf
echo 1 > /proc/sys/net/ipv4/ip_forward

检查修改结果

# sysctl -p
net.ipv4.ip_forward = 1

查看当前机器路由

root@node08:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         10.1.199.1      0.0.0.0         UG    100    0        0 eth0
10.1.199.0      *               255.255.255.0   U     0      0        0 eth0
10.10.10.0      *               255.255.255.0   U     0      0        0 eth1

NTP服务

apt-get install -y ntp

环境变量

环境变量主要是为了使得文档更加灵活,参数可以设置

cat >/root/novarc <<EOF
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=password
export MYSQL_PASS=password
export SERVICE_PASSWORD=password
export RABBIT_PASSWORD=password
export OS_AUTH_URL="http://localhost:5000/v2.0/"
export SERVICE_ENDPOINT="http://localhost:35357/v2.0"
export SERVICE_TOKEN=ADMIN
export PUBLIC_IP="$(/sbin/ifconfig eth0 | awk '/inet addr/ {print $2}' | cut -f2 -d ":")"
export LOCAL_IP="$(/sbin/ifconfig eth1 | awk '/inet addr/ {print $2}' | cut -f2 -d ":")"
EOF

你可以根据你的需要调整上面password字段.它会自动获得你网卡的IP地址, 你可以根据你的情况调整。

查看novarc

cat /root/novarc 
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=password
export MYSQL_PASS=password
export SERVICE_PASSWORD=password
export RABBIT_PASSWORD=password
export OS_AUTH_URL="http://localhost:5000/v2.0/"
export SERVICE_ENDPOINT="http://localhost:35357/v2.0"
export SERVICE_TOKEN=ADMIN
export PUBLIC_IP="10.1.199.8"
export LOCAL_IP="10.10.10.8"

让环境变量生效

source novarc
echo "source novarc">>.bashrc

RabbitMQ和相关软件

apt-get -y install rabbitmq-server vlan bridge-utils curl

数据库

Openstack的组件都需要用到mysql

数据库	用户	密码
mysql	root	password
keystone	keystone	password
nova	nova	password
glance	glance	password
cinder	cinder	password
quantum	quantum	password

 

设置自动安装，无需输入密码

cat <<MYSQL_PRESEED | debconf-set-selections
mysql-server-5.5 mysql-server/root_password password $MYSQL_PASS
mysql-server-5.5 mysql-server/root_password_again password $MYSQL_PASS
mysql-server-5.5 mysql-server/start_on_boot boolean true
MYSQL_PRESEED

安装mysql

apt-get -y install mysql-server python-mysqldb

允许远程访问mysql

sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf

重启服务

service mysql restart

创建数据库

mysql -uroot -p$MYSQL_PASS <<EOF
CREATE DATABASE nova;
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' IDENTIFIED BY '$MYSQL_PASS';
CREATE DATABASE glance;
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' IDENTIFIED BY '$MYSQL_PASS';
CREATE DATABASE keystone;
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%'IDENTIFIED BY '$MYSQL_PASS';
CREATE DATABASE cinder;
GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'%'IDENTIFIED BY '$MYSQL_PASS';
CREATE DATABASE quantum;
GRANT ALL PRIVILEGES ON quantum.* TO 'quantum'@'%'IDENTIFIED BY '$MYSQL_PASS';
FLUSH PRIVILEGES;
EOF

Keystone

安装

apt-get install -y keystone

查看keystone运行状态

service keystone status

编辑 /etc/keystone/keystone.conf

需要注意的是admin_token 默认是ADMIN

[sql]
# The SQLAlchemy connection string used to connect to the database
#connection = sqlite:////var/lib/keystone/keystone.db
connection = mysql://keystone:[email protected]:3306/keystone

[signing]
#token_format = PKI
token_format = UUID

 

重启服务和初始化数据库

service keystone restart
keystone-manage db_sync

使用脚本导入初始化数据

keystone.sh  和 endpoint.sh，脚本运行过程中，会出现 WARNING: Bypassing authentication using a token & endpoint (authentication credentials are being ignored). 提示，忽略就可以。

wget http://www.chenshake.com/wp-content/uploads/2013/04/keystone.sh_.txt
mv keystone.sh_.txt keystone.sh
bash keystone.sh

导入endpoint

wget http://www.chenshake.com/wp-content/uploads/2013/04/endpoint.sh_.txt
mv endpoint.sh_.txt endpoint.sh
bash endpoint.sh

验证keystone

keystone user-list
keystone role-list
keystone tenant-list
keystone endpoint-list
service keystone status

Troubleshooting Keystone

1. 查看 5000 和 35357 端口是否在监听
2. 查看 /var/log/keystone/keystone.log 报错信息
3. keystone.sh 脚本执行错误解决：(检查脚本内容变量设置)

如果你运行上面脚本出现问题，你可以删除数据库再导入数据，运行脚本出错，基本都是环境变量设置有误导致

mysql -uroot -p
mysql> drop database keystone;
mysql> create database keystone; 
mysql> quit;

记得需要同步一下数据库

keystone-manage db_sync

Glance

安装

apt-get -y install glance

配置

编辑 /etc/glance/glance-api.conf

#sql_connection = sqlite:////var/lib/glance/glance.sqlite
sql_connection = mysql://glance:[email protected]/glance

[keystone_authtoken]
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
#admin_tenant_name = %SERVICE_TENANT_NAME%
#admin_user = %SERVICE_USER%
#admin_password = %SERVICE_PASSWORD%
admin_tenant_name = service
admin_user = glance
admin_password = password

[paste_deploy]
config_file = /etc/glance/glance-api-paste.ini
flavor = keystone

重启服务

service glance-api restart

编辑 /etc/glance/glance-registry.conf

#sql_connection = sqlite:////var/lib/glance/glance.sqlite
sql_connection = mysql://glance:[email protected]/glance

[keystone_authtoken]
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
#admin_tenant_name = %SERVICE_TENANT_NAME%
#admin_user = %SERVICE_USER%
#admin_password = %SERVICE_PASSWORD%
admin_tenant_name = service
admin_user = glance
admin_password = password

[paste_deploy]
config_file = /etc/glance/glance-registry-paste.ini
flavor = keystone

重启服务

service glance-registry restart

同步数据库

glance-manage version_control 0
glance-manage db_sync

验证Glance

glance image-list

 

下面应该没任何输出，就表示正确。因为目前还没有上传image。

下载Image

我们下载CirrOS的image作为测试使用，只有10M。如果是ubuntu官方的image，220M，并且ubuntu官方的image，都是需要使用密钥登陆。

CirrOS

下载image

wget https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img

上传image

glance image-create --name=cirros-0.3.0-x86_64 --public  --container-format=bare \
--disk-format=qcow2 < /root/cirros-0.3.0-x86_64-disk.img

Cirros，是可以使用用户名和密码登陆，也可以使用密钥登陆

user:cirros
password:cubswin:)

Ubuntu官方image

下载image

wget http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img

上传image

glance image-create --name="Ubuntu 12.04 cloudimg amd64" --public --container-format=ovf \
--disk-format=qcow2 < /root/ubuntu-12.04-server-cloudimg-amd64-disk1.img

 

user：ubuntu

只能使用密钥登陆。

Quantum

安装

apt-get install -y quantum-server quantum-plugin-linuxbridge quantum-plugin-linuxbridge-agent \
dnsmasq quantum-dhcp-agent quantum-l3-agent

编辑 /etc/quantum/quantum.conf

#core_plugin = quantum.plugins.openvswitch.ovs_quantum_plugin.OVSQuantumPluginV2
core_plugin = quantum.plugins.linuxbridge.lb_quantum_plugin.LinuxBridgePluginV2

[keystone_authtoken]
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
#admin_tenant_name = %SERVICE_TENANT_NAME%
#admin_user = %SERVICE_USER%
#admin_password = %SERVICE_PASSWORD%
admin_tenant_name = service
admin_user = quantum
admin_password = password
signing_dir = /var/lib/quantum/keystone-signing

配置bridge插件 /etc/quantum/plugins/linuxbridge/linuxbridge_conf.ini

# Default: tenant_network_type = local
# Example: tenant_network_type = vlan
tenant_network_type = vlan

# Default: network_vlan_ranges =
# Example: network_vlan_ranges = physnet1:1000:2999
network_vlan_ranges = physnet1:1000:2999

#sql_connection = sqlite:////var/lib/quantum/linuxbridge.sqlite
sql_connection = mysql://quantum:[email protected]/quantum

# Default: physical_interface_mappings =
# Example: physical_interface_mappings = physnet1:eth1
physical_interface_mappings = physnet1:eth0

编辑 /etc/quantum/l3_agent.ini:

#interface_driver = quantum.agent.linux.interface.OVSInterfaceDriver
interface_driver = quantum.agent.linux.interface.BridgeInterfaceDriver

# use_namespaces = True
use_namespaces = False

编辑 /etc/quantum/dhcp_agent.ini

interface_driver = quantum.agent.linux.interface.BridgeInterfaceDriver
use_namespaces = False

重启quantum所有服务

cd /etc/init.d/; for i in $( ls quantum-* ); do sudo service $i restart; done
service dnsmasq restart

验证Quantum

quantum help

 

看到所有quantum相关的命令,目前quantum没有任何数据

quantum net-list

KVM

安装

apt-get install -y kvm libvirt-bin pm-utils

编辑 /etc/libvirt/qemu.conf ，添加下面内容

cgroup_device_acl = [
    "/dev/null", "/dev/full", "/dev/zero",
    "/dev/random", "/dev/urandom",
    "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
    "/dev/rtc", "/dev/hpet","/dev/net/tun",
]

或者运行命令：这个地方用命令修改有点复杂，还没找到太好的办法。

cat <<EOF>>/etc/libvirt/qemu.conf
cgroup_device_acl = [
    "/dev/null", "/dev/full", "/dev/zero",
    "/dev/random", "/dev/urandom",
    "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
    "/dev/rtc", "/dev/hpet","/dev/net/tun",
]
EOF

删除默认 virtual bridge

virsh net-destroy default
virsh net-undefine default

允许迁移

编辑 /etc/libvirt/libvirtd.conf,

#auth_tcp = "sasl"
auth_tcp = "none"

或者运行下面命令

sed -i '/#listen_tcp/s/#listen_tcp/listen_tcp/; /#auth_tcp/s/#auth_tcp/auth_tcp/; /auth_tcp/s/sasl/none/'  /etc/libvirt/libvirtd.conf

编辑 /etc/init/libvirt-bin.conf

env libvirtd_opts="-d -l"

或者使用命令

sed -i '/env libvirtd_opts/s/-d/-d -l/' /etc/init/libvirt-bin.conf

编辑 /etc/default/libvirt-bin

libvirtd_opts="-d -l"

或者使用命令

sed -i '/libvirtd_opts/s/-d/-d -l/' /etc/default/libvirt-bin

重启服务

service libvirt-bin restart

Nova

安装

apt-get install -y nova-api nova-cert novnc nova-consoleauth nova-scheduler \
nova-novncproxy nova-doc nova-conductor nova-compute-kvm

编辑 /etc/nova/api-paste.ini

[filter:authtoken]
paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
#admin_tenant_name = %SERVICE_TENANT_NAME%
#admin_user = %SERVICE_USER%
#admin_password = %SERVICE_PASSWORD%
admin_tenant_name = service
admin_user = nova
admin_password = password
signing_dir = /tmp/keystone-signing-nova
# Workaround for https://bugs.launchpad.net/nova/+bug/1154809
auth_version = v2.0

创建 /etc/nova/nova.conf

 

cat >/etc/nova/nova.conf <<EOF
[DEFAULT]
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/run/lock/nova
verbose=True
api_paste_config=/etc/nova/api-paste.ini
compute_scheduler_driver=nova.scheduler.simple.SimpleScheduler
rabbit_host=$LOCAL_IP
nova_url=http://$LOCAL_IP:8774/v1.1/
sql_connection=mysql://nova:$MYSQL_PASS@$LOCAL_IP/nova
root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf

# Auth
use_deprecated_auth=false
auth_strategy=keystone

# Imaging service
glance_api_servers=$LOCAL_IP:9292
image_service=nova.image.glance.GlanceImageService

# Vnc configuration
novnc_enabled=true
novncproxy_base_url=http://$PUBLIC_IP:6080/vnc_auto.html
novncproxy_port=6080
vncserver_proxyclient_address=$LOCAL_IP
vncserver_listen=0.0.0.0

# Network settings
network_api_class=nova.network.quantumv2.api.API
quantum_url=http://$LOCAL_IP:9696
quantum_auth_strategy=keystone
quantum_admin_tenant_name=service
quantum_admin_username=quantum
quantum_admin_password=$SERVICE_PASSWORD
quantum_admin_auth_url=http://$LOCAL_IP:35357/v2.0
libvirt_vif_driver=nova.virt.libvirt.vif.QuantumLinuxBridgeVIFDriver
linuxnet_interface_driver=nova.network.linux_net.LinuxBridgeInterfaceDriver
firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver

# Compute #
compute_driver=libvirt.LibvirtDriver

# Cinder #
volume_api_class=nova.volume.cinder.API
osapi_volume_listen_port=5900
EOF

编辑 /etc/nova/nova-compute.conf:

[DEFAULT]
libvirt_type=kvm
compute_driver=libvirt.LibvirtDriver
libvirt_vif_type=ethernet
libvirt_vif_driver=nova.virt.libvirt.vif.QuantumLinuxBridgeVIFDriver

同步数据库

nova-manage db sync

重启nova相关服务

cd /etc/init.d/; for i in $( ls nova-* ); do sudo service $i restart; done

查看nova 服务

nova-manage service list

Cinder

安装

apt-get install -y cinder-api cinder-scheduler cinder-volume \
iscsitarget open-iscsi iscsitarget-dkms

配置iscsi服务

sed -i 's/false/true/g' /etc/default/iscsitarget

 

重启服务

service iscsitarget start
service open-iscsi start

编辑 /etc/cinder/api-paste.ini

[filter:authtoken]
paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory
service_protocol = http
service_host = 127.0.0.1
service_port = 5000
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
#admin_tenant_name = %SERVICE_TENANT_NAME%
#admin_user = %SERVICE_USER%
#admin_password = %SERVICE_PASSWORD%
admin_tenant_name = service
admin_user = cinder
admin_password = password

编辑 /etc/cinder/cinder.conf

[DEFAULT]
rootwrap_config=/etc/cinder/rootwrap.conf
sql_connection = mysql://cinder:[email protected]/cinder
api_paste_config = /etc/cinder/api-paste.ini
iscsi_helper=ietadm
volume_name_template = volume-%s
volume_group = cinder-volumes
verbose = True
auth_strategy = keystone
#osapi_volume_listen_port=5900

同步数据库

cinder-manage db sync

 

创建一个cinder volume的卷

如果使用文件模拟的方式,其实性能很差,基本是不可用。所以建议采用单独分区来测试

我的硬盘专门一个分区给volume使用

umount /dev/sda7
pvcreate /dev/sda7
vgcreate cinder-volumes /dev/sda7

去掉开机挂载

sed -i '/nova-volume/s/^/#/' /etc/fstab

重启服务

cd /etc/init.d/; for i in $( ls cinder-* ); do sudo service $i restart; done

查看cinder服务状态

cd /etc/init.d/; for i in $( ls cinder-* ); do sudo service $i status; done

Horizon

安装

apt-get install -y openstack-dashboard memcached

默认的ubuntu的theme一直都有问题，需要删掉。

dpkg --purge openstack-dashboard-ubuntu-theme

 

重启相关服务

service apache2 restart; service memcached restart

 

这个时候你就可以直接使用 http://10.1.199.8/horizon 访问

user:admin  pass:password

如何使用Dashboard，后续补上。