下面共享以下我目前正在使用的，基于git实现的网站发布脚本
Deployment GIT
https://github.com/netkiller/deployment.git
$ git clone https://github.com/netkiller/deployment.git $ chmod 755 -R deployment $ export DEPLOY_HOME=~/deployment
临时使用的方法
export DEPLOY_HOME=/home/user/deployment
$ cd deployment/ $ ln -s bin/deploy.git run
$ ./run
Usage: ./run [OPTION] <server-id> <directory/timepoint>
OPTION:
development <domain> <host>
testing <domain> <host>
production <domain> <host>
branch {development|testing|production} <domain> <host> <branchname>
revert {development|testing|production} <domain> <host> <revision>
backup <domain> <host> <directory>
release <domain> <host> <tags> <message>
list
list <domain> <host>
clean {development|testing|production} <domain> <host>
log <project> <line>
conf list
cron show
cron setup
cron edit
环境说明
-
development 开发环境
-
testing 测试环境，代码来自开发环境的合并
-
production 生产环境，当testing环境通过测试后，将testing 合并到 主干 即成为生产环境的代码
另外我们可以通过release功能将主干的代码复制到tags中，命名采用版本号
部署开发代码到开发环境
cat deployment/conf/development/mydomain.com/www.conf
[email protected]:mydomain.com/www.mydomain.com MODE=RSYNC OPTION="--delete --password-file=$PREFIX/conf/development/passwd" REMOTE="[email protected]" DESTINATION=mydomain.com/www.mydomain.com
创建密码文件
$ cat deployment/conf/development/passwd eF9nJCcGKJPsiqZsfjGXxwfF41cLibTo
部署测试分支到测试环境
cat deployment/conf/testing/mydomain.com/www.conf
[email protected]:mydomain.com/www.mydomain.com MODE=RSYNC OPTION="--delete --password-file=$PREFIX/conf/development/passwd" REMOTE="[email protected]" DESTINATION=mydomain.com/www.mydomain.com
创建密码文件
$ cat deployment/conf/testing/passwd eF9nJCcGKJPsiqZsfjGXxwfF41cLibTo
部署主干代码到远程主机
cat deployment/conf/production/mydomain.com/www.conf
[email protected]:mydomain.com/www.mydomain.com MODE=RSYNC OPTION="--delete --password-file=$PREFIX/conf/development/passwd" REMOTE="[email protected]" DESTINATION=mydomain.com/www.mydomain.com
创建密码文件
$ cat deployment/conf/production/passwd eF9nJCcGKJPsiqZsfjGXxwfF41cLibTo
有时我们不希望某些文件被上传到服务器上。我们可以通过排除列表来排除上传
cat exclude/mydomain.com/www.lst /test/phpinfo.php /config/database.php /backup/*.sql
生产环境的安全问题，例如数据库联接信息，开发环境与测试环境的数据库是可以供发人员和测试人员随意操作的，损坏之后恢复即可，但生产环境的数据库是不能随便操作的，除运维人员其他人是不应该有权限的， 我们希望部署到生产环境的时候使用另一个配置文件，并且这个配置文件只有运维人员才能编辑。
config/database.php 将覆盖原有的配置文件，然后上传到生产环境
vim share/production/mydomain.com/www/config/database.php ... 你的数据库连接信息 ...
部署前需要做什么
$ cat libexec/mydomain.com/www/before rsync -au $DEPLOY_HOME/src/production/mydomain.com/www.mydomain.com/cn/* $DEPLOY_HOME/src/production/mydomain.com/www.mydomain.com/news/ rsync -au $DEPLOY_HOME/src/production/mydomain.com/www.mydomain.com/images/* $DEPLOY_HOME/src/production/mydomain.com/www.mydomain.com/bbs/images/ rsync -au $DEPLOY_HOME/src/production/mydomain.com/www.mydomain.com/css/* $DEPLOY_HOME/src/production/mydomain.com/www.mydomain.com/news/css
部署后需要做什么
cat libexec/hx9999.com/www/after ssh [email protected] "chown www:www -R /www/mydomain.com" ssh [email protected] "chown 700 -R /www/mydomain.com" ssh [email protected] "chown 777 -R /www/mydomain.com/www.mydomain.com/images/upload"
在需要部署的节点上安装rsync
yum install xinetd rsync -y vim /etc/xinetd.d/rsync <<VIM > /dev/null 2>&1 :%s/yes/no/ :wq VIM # service xinetd restart Stopping xinetd: [ OK ] Starting xinetd: [ OK ]
/etc/rsyncd.conf 配置文件
# cat /etc/rsyncd.conf
uid = root
gid = root
use chroot = no
max connections = 8
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
hosts deny=*
hosts allow=192.168.2.0/255.255.255.0
[www]
uid = www
gid = www
path = /www
ignore errors
read only = no
list = no
auth users = www
secrets file = /etc/rsyncd.passwd
[mydomain.com]
uid = www
gid = www
path = /www/mydomain.com
ignore errors
read only = no
list = no
auth users = mydomain
secrets file = /etc/rsyncd.passwd
[example.com]
uid = www
gid = www
path = /www/example.com
ignore errors
read only = no
list = no
auth users = example
secrets file = /etc/rsyncd.passwd
创建密码
cat > /etc/rsyncd.passwd <<EOD www:eF9nJCcGKJPsiqZsfjGXxwfF41cLibTo mydomain:eF9nJCcGKJPsiqZsfjGXxwfF41cLibTo example:eF9nJCcGKJPsiqZsfjGXxwfF41cLibTo EOD
| development | testing 建议使用分支管理， 而production是用master分支 |
开发环境部署
$ ~/deployment/run branch development mydomain.com www development 首次需要运行，切换到开发分支 $ ~/deployment/run development mydomain.com www
测试环境部署
$ ~/deployment/run branch development mydomain.com www testing 首次需要运行，切换到开发分支 $ ~/deployment/run testing mydomain.com www
如果每个bug一个分支的情况可以每次先运行
$ ~/deployment/run branch development mydomain.com www bug0005
生产环境部署
$ ~/deployment/run production mydomain.com www
每次部署都会在服务器 /www/mydomain.com/backup/ 下备份更改的文件
当程序升级失败需要立即回撤到指定版本时使用
$ ~/deployment/run revert {development|testing|production} <domain> <host> <revision>
./run revert development mydomain www 29dd5c3de6559e2ea6749f5a146ee36cbae750a7 ./run revert testing mydomain www 29dd5c3de6559e2ea6749f5a146ee36cbae750a7 ./run revert production mydomain www 29dd5c3de6559e2ea6749f5a146ee36cbae750a7
查看当前分支
[www@manager deployment]$ ./run branch development mydomain.com www * master
切换分支
[www@manager deployment]$ ./run branch development mydomain.com www development HEAD is now at 461b796 提交最新代码 Branch development set up to track remote branch development from origin. Switched to a new branch 'development'
现在已经切换到开发分支
[www@manager deployment]$ ./run branch development mydomain.com www * development master
部署日志 deploy.YYYY-MM-DD.log, 记录部署时间与动态
$ cat log/deploy.2012-08-03.log [2012-12-06 21:52:05] [update] /opt/git/testing/mydomain.com/m.mydomain.com [2012-12-06 21:52:10] [deploy] testing/mydomain.com/m.mydomain.com => [email protected]:mydomain.com/m.mydomain.com [2012-12-06 21:53:13] [checkout] commit:29dd5c3de6559e2ea6749f5a146ee36cbae750a7 /opt/git/testing/mydomain.com/m.mydomain.com [2012-12-06 21:53:18] [deploy] testing/mydomain.com/m.mydomain.com => [email protected]:mydomain.com/m.mydomain.com [2012-12-06 21:53:39] [update] /opt/git/testing/mydomain.com/m.mydomain.com [2012-12-06 21:53:45] [deploy] testing/mydomain.com/m.mydomain.com => [email protected]:mydomain.com/m.mydomain.com [2012-12-06 21:54:08] [update] /opt/git/testing/mydomain.com/m.mydomain.com [2012-12-06 21:54:10] [deploy] testing/mydomain.com/m.mydomain.com => [email protected]:mydomain.com/m.mydomain.com [2012-12-06 21:54:13] [checkout] commit:29dd5c3de6559e2ea6749f5a146ee36cbae750a7 /opt/git/testing/mydomain.com/m.mydomain.com [2012-12-06 21:54:15] [deploy] testing/mydomain.com/m.mydomain.com => [email protected]:mydomain.com/m.mydomain.com
项目日志 www.example.com.log 记录项目有哪些更新, 上传的细节, 你能通过日志看到那些文件被上传
$ cat log/www.example.com.log -------------------------------------------------- HEAD is now at 03b3ad5 XXXXXXXXXXXX - share: - libexec: 2012/12/06 21:53:45 [12488] building file list 2012/12/06 21:53:45 [12488] .d..t...... application/config/development/ 2012/12/06 21:53:45 [12488] <f.st...... application/config/development/database.php 2012/12/06 21:53:45 [12488] .d..t...... application/controllers/ 2012/12/06 21:53:45 [12488] <f.st...... application/controllers/info.php 2012/12/06 21:53:45 [12488] .d..t...... application/core/ 2012/12/06 21:53:45 [12488] <f.st...... application/core/MY_Controller.php 2012/12/06 21:53:45 [12488] .d..t...... application/models/ 2012/12/06 21:53:45 [12488] <f.st...... application/models/news.php 2012/12/06 21:53:45 [12488] .d..t...... application/views/ 2012/12/06 21:53:45 [12488] <f.st...... application/views/example.html 2012/12/06 21:53:45 [12488] <f.st...... application/views/index.php 2012/12/06 21:53:45 [12488] .d..t...... resources/css/ 2012/12/06 21:53:45 [12488] <f.st...... resources/css/m.css 2012/12/06 21:53:45 [12488] sent 23640 bytes received 421 bytes 3701.69 bytes/sec 2012/12/06 21:53:45 [12488] total size is 2869760 speedup is 119.27 --------------------------------------------------
#!/bin/bash ##################################################################### # Description: Automation Deployment Script # Netkiller series utilities # Author: Neo<[email protected]> # Homepage: http://netkiller.github.com/ # http://netkiller.sourceforge.net/ # GIT URL: https://github.com/netkiller/deployment.git # $Id$ ##################################################################### # :set tabstop=4 # :set shiftwidth=4 # :set expandtab if [ -z $DEPLOY_HOME ]; then echo 'Example: export DEPLOY_HOME=/srv/deploy' exit fi if [ -f $DEPLOY_HOME/conf/default.conf ];then . $DEPLOY_HOME/conf/default.conf fi if [ -f $DEPLOY_HOME/conf/stage.conf ];then . $DEPLOY_HOME/conf/stage.conf fi #================================================================================ LOGFILE="deploy.$(date -d "today" +"%Y-%m-%d").log" TMPDIR=$(mktemp -d --suffix=.tmp -p /tmp deploy.XXXXXX) SVN=/usr/bin/svn GIT=/usr/bin/git BACKUPDIR=/backup RSYNC="rsync" UPLOAD_DIR=$TMPDIR REVISION='' DEBUG='yes' # development production testing if [ -z $STAGE ]; then echo 'Example: touch conf/stage.conf' echo "STAGE='development' or STAGE='testing' or STAGE='production'" exit fi #================================================================================ if [ ! -d ${TMPDIR} ]; then mkdir ${TMPDIR} fi #chmod 700 -R ${SRCDIR}/* umask 0077 #pkgname=${project}-${version}-${datetime}.pkg #tar jcvf ${pkgname} /tmp/${project} --remove-files >> deploy.log ##################################################################### function logging(){ local logfile="$LOGDIR/$LOGFILE" local timepoint=$(date -d "today" +"%Y-%m-%d %H:%M:%S") local status=$1 local message=$2 echo "[$timepoint] [${status}] ${message}" >> $logfile } function debug(){ if [ ${DEBUG} = 'yes' ]; then local logfile="$LOGDIR/debug.log" local timepoint=$(date -d "today" +"%Y-%m-%d %H:%M:%S") local status=$1 local message=$2 echo "[$timepoint] [${status}] ${message}" >> $logfile fi } #logging 'OK' 'This is test msg!!!' #debug 'OK' 'This is debug msg!!!' function conf(){ local cmd=$2 local prj=$3 case $cmd in list) ls $SYSCONFDIR/*/* ;; new) mkdir -p $SYSCONFDIR #if [ ! -d ${BACKUPDIR} ]; then # mkdir -p $BACKUPDIR #fi read -p "Project directory: " prjdir if [ -z $prjdir ]; then exit fi if [ -f $SYSCONFDIR/$prjdir.conf ]; then echo "cannot create config $prjdir.conf': File exists" exit 1 fi read -p "subversion url: $REPOSITORIES/: " svnurl if [ -z $svnurl ]; then svnurl=$REPOSITORIES fi read -p "hostname: " host if [ -z $host ]; then host="localhost" echo "default hostname 'localhost'" fi read -p "upload mode ftp/scp/sftp/rsync: " mode if [ -z $mode ]; then mode=ftp else case $mode in ftp) mode="ftpdeploy" ;; scp) mode="scpdeploy" ;; sftp) mode="sftpdeploy" ;; rsync) mode="rsync" ;; esac fi read -p "Create $prjdir config? [y/n]" -n 1 key echo if [ $key = 'y' ]; then echo -ne "REPOSITORIES=$REPOSITORIES/$svnurl COMMAND=$mode HOSTNAME=$host " >> $SYSCONFDIR/$prjdir.conf fi ;; remove) if [ -f $SYSCONFDIR/$prj ]; then rm -rf $SYSCONFDIR/$prj fi ;; show) cat $SYSCONFDIR/$prj ;; edit) vim $SYSCONFDIR/$prj ;; *) ls $SYSCONFDIR/*/* ;; esac } ##################################################################### function config { local cfg=$1 exclude_from=$PREFIX/exclude/${cfg}.lst include_from=$PREFIX/include/${cfg}.lst if [ -f $SYSCONFDIR/${STAGE}/${cfg}.conf ];then . $SYSCONFDIR/${STAGE}/${cfg}.conf else echo "Please provide the config($SYSCONFDIR/${STAGE}/${cfg}.conf) to deploy!" exit fi if [ -z "$cfg" ]; then echo "Please provide the path for deploy!" exit fi if [ ! -f $exclude_from ]; then echo "Please provide a list of excluded in the $exclude_from." touch $exclude_from exit fi if [ ! -f $include_from ]; then echo "Please provide a list of included in the $include_from." touch $include_from exit fi # case ${STAGE} in # development) # SUBVERSION='development' # ;; # testing) # SUBVERSION='' # ;; # production) # ;; # *) # SUBVERSION='current' # ;; # esac } function deploy() { local domain=$2 local host=$3 local commit=$4 local logfile=${LOGDIR}/${host}.${domain}.log local backupdir=${BACKUPDIR}/${host}.${domain}/$(date '+%Y-%m-%d/%H:%M:%S') local message=${STAGE}/${domain}/${host}.${domain} if [ $# -lt 3 ]; then usage fi if [ ${STAGE} = 'production' ]; then read -p "Are you sure you want to continue deploying? [y/n]" -n 1 key echo if [ $key != 'y' ]; then exit fi fi if [ $host = 'all' ]; then for h in $(ls -1 $SYSCONFDIR/${STAGE}/$domain/ | cut -d. -f1) do /bin/sh $BINDIR/deploy deploy $domain $h done exit fi #if [ ! -z $revision ]; then # REVISION="-r ${revision}" #fi config ${domain}/${host} project=$SRCDIR/${STAGE}/${domain}/${host}.${domain} GIT_OPTS=${REVISION} echo '================================================================================' if [ -d ${project} ]; then cd $project #$GIT stash #$GIT pull --progress #$GIT stash clear #$GIT checkout . $GIT reset HEAD --hard >> $logfile echo -n " Repository: ${REPOSITORY} " $GIT pull --progress if [ ! -z $commit ]; then $GIT checkout $commit . echo " Commit: $commit" logging 'checkout' "commit:$commit ${project} " else logging 'update' ${project} fi else mkdir -p ${project} $GIT clone ${REPOSITORY} ${project} >> $logfile logging 'checkout' ${project} fi echo '================================================================================' RSYNC_OPTS=" -azv --backup --backup-dir=${backupdir} --exclude=.git --log-file=${logfile} --exclude-from=$exclude_from --include-from=$include_from" if [ -d ${SHAREDIR}/${STAGE}/${domain}/${host}/ ]; then cp -a ${SHAREDIR}/${STAGE}/${domain}/${host}/* ${project}/ fi echo '- share:' >> ${logfile} echo ' Share [ OK ]' if [ -f ${LIBEXECDIR}/${domain}/${host}/before ];then /bin/sh ${LIBEXECDIR}/${domain}/${host}/before >> ${logfile} fi echo '- libexec:' >> ${logfile} echo ' Libexec [ OK ]' find $SRCDIR/* -type f -name "Thumbs.db" -exec rm -rf {} \; echo '================================================================================' for addr in ${REMOTE} do echo " Deploy [${message}] ${addr}" echo '================================================================================' case ${MODE} in FTP) ftpdeploy ;; SCP) scp -ar ${project}/* ${addr}:${DESTINATION} ;; SFTP) sftpdeploy ;; RSYNC) $RSYNC $RSYNC_OPTS $OPTION ${project}/* ${addr}::${DESTINATION} debug 'rsync' "$RSYNC $RSYNC_OPTS $OPTION ${project}/* ${addr}::${DESTINATION}" ;; "RSYNC+SSH") $RSYNC $RSYNC_OPTS ${project}/* ${addr}:${DESTINATION} ;; esac if [ -z "${REVISION}" ]; then logging 'deploy' "${message} => ${addr}:${DESTINATION}" else logging 'revert' "${message} => ${addr}:${DESTINATION}" fi echo '--------------------------------------------------' >> ${logfile} done if [ -f ${LIBEXECDIR}/${domain}/${host}/after ];then #ssh ${scp} < ${LIBEXECDIR}/${domain}/${host}/after exit fi } function revert() { #if [ $STAGE = 'testing' -o $STAGE = 'development' -o $STAGE = 'production' ]; then local domain=$3 local host=$4 local commit=$5 # else # local domain=$1 # local host=$2 # local revision=$3 #fi deploy $STAGE $domain $host $commit } function timepoint { TIMEPOINT=`date '+%Y-%m-%d.%H-%M-%S'` echo $TIMEPOINT >> timepoint.log } function unstable { local edition=$(basename $unstable) svn export ${unstable} ${src}/$edition for ignore in $( cat excluded.lst ); do rm -rf ${src}/$edition/$ignore done $RSYNC ${src}/$edition ${destination} ssh ${remote} < script/unstable } function clean() { local stage=$2 local domain=$3 local host=$4 local project=$SRCDIR/${stage}/${domain}/$host.${domain} if [ $# -lt 3 ]; then usage fi rm -rf ${project} } function list { local domain=$2 local host=$3 local dir=$4 if [ -z $domain ]; then ls $SRCDIR/* exit fi if [ -z $host ]; then usage fi #config ${domain}/${host} ls $SRCDIR/*/${domain}/${host}.${domain} #git ls ${REPOSITORIES}/$dir #| awk -F '/' '{print $1}' } function backup() { local domain=$2 local host=$3 local dir=$4 local logfile=${LOGDIR}/${host}.${domain}.log if [ -z $domain ]; then usage fi if [ -z $host ]; then usage fi config ${domain}/${host} if [ -z $dir ]; then dir=$TMPDIR fi for addr in ${REMOTE} do dir=$dir/${addr} if [ ! -d ${dir} ]; then mkdir -p $dir fi RSYNC_OPTS=" -azv " ${RSYNC} ${RSYNC_OPTS} ${OPTION} ${addr}::${DESTINATION} $dir >> ${logfile} logging 'backup' "rsync://${addr}::${DESTINATION} to ${dir}" echo 'Backup Directory:' $dir exit done } function cron(){ local fun=$2 case ${fun} in show) crontab -l ;; setup) cat $PREFIX/cron.d/crontab | crontab ;; edit) vim $PREFIX/cron.d/crontab cat $PREFIX/cron.d/crontab | crontab ;; *) usage ;; esac } function release() { local domain=$2 local host=$3 local ver=$4 local message=$5 if [ $# -lt 4 ]; then usage fi if [ -z $message ]; then echo -n "Message: " read message fi config ${domain}/${host} local logfile=${LOGDIR}/${host}.${domain}.log project=$SRCDIR/${STAGE}/${domain}/${host}.${domain} cd $project $GIT tag ${ver} >> $logfile logging 'release' "{GIT} tag ${ver} - ${message}" } function stage(){ case $1 in development) STAGE='development' ;; testing) STAGE='testing' ;; production) STAGE='production' ;; *) echo "STAGE ERROR" exit ;; esac echo $"STAGE=$STAGE" > $SYSCONFDIR/stage.conf && echo $STAGE logging 'stage' "${STAGE}" } function branch(){ local stage=$2 local domain=$3 local host=$4 local branchname=$5 cd $SRCDIR/${stage}/${domain}/$host.${domain} if [ -z $branchname ]; then git branch else git reset HEAD --hard git checkout $branchname fi } function usage(){ echo $"Usage: $0 [OPTION] <server-id> <directory/timepoint>" echo -ne " OPTION: development <domain> <host> testing <domain> <host> production <domain> <host> branch {development|testing|production} <domain> <host> <branchname> revert {development|testing|production} <domain> <host> <revision> backup <domain> <host> <directory> release <domain> <host> <tags> <message> list list <domain> <host> clean {development|testing|production} <domain> <host> log <project> <line> conf list cron show cron setup cron edit " # stage {development|testing|production} # deploy <domain> <host> # revert <domain> <host> <revision> # conf new <project> # conf remove <project> # conf show <project> # conf edit <project> exit } case "$1" in stage) stage $2 ;; development) STAGE='development' deploy $@ ;; testing) STAGE='testing' deploy $@ ;; production) STAGE='production' deploy $@ ;; branch) branch $@ ;; revert) STAGE=$2 revert $@ ;; backup) backup $@ ;; branch) branch $@ ;; cron) cron $@ ;; release) release $@ ;; clean) clean $@ ;; list) list $@ ;; log) ls -1 $LOGDIR/* ;; conf) conf $@ ;; *) usage exit 1 esac