Build a debian source mirror server

There are more and more linux servers on the intranet, and it is necessary to build an intranet debian source mirror server.

Debian officially provides a script ftpsync for building source mirrors , and there are detailed source mirror setup instructions

Ready to work

ftpsync uses the rsync program for source mirroring

# apt-get install rsync

Download the ftpsync script from the official

# wget http://ftp-master.debian.org/ftpsync.tar.gz

configure ftpsync

ftpsync uses the environment variable BASEDIR to locate the storage paths of program files, configuration files, log files, etc. BASEDIR defaults to ${HOME}, and ftpsync can be placed in the user's home directory

Create a new bin, etc, log directory in the user's home directory, unzip ftpsync,

$ tar zxvf ftpsync.tar.gz
$ cp distrib/bin/ftpsync ~/bin
$ cp distrib/etc/ftpsync.conf.sample ~/etc/ftpsync.conf
$ cp distrib/etc/common ~/etc/common

Create a separate directory for storing image files, and the disk space required by various architectures. The current image all, amd64 and source requires a total of about 190G of hard disk space.

Images can be stored anywhere, as long as the user running ftpsync has read and write permissions to the directory.
Here, /srv/mirrors/debian is used to store image files, and the owner and group of the directory are set to the current user.

Finally open ~/etc/ftpsync.conf and modify the following:

TO=”/srv/mirrors/debian/” ##镜像源存放位置
RSYNC_HOST=”ftp.cn.debian.org” ##镜像自哪个外部源,debian中国官方源镜像是最佳选择，当然ftp.tw.debian.org,ftp.kr.debian.org和ftp.jp.debian.org速度也很快，中国官方源镜像由中国科技大学维护
ARCH_EXCLUDE=”alpha arm armel armhf hppa hurd-i386 i386 ia64 kfreebsd-amd64 kfreebsd-i386 m68k mipsel mips powerpc s390 s390x sh sparc source” ##排除的架构，此处只保留amd64源，source源也排除，只镜像必要的，尽量节省硬盘空间。

Note : The current script will prompt an error if the source mirror is excluded:
Unexpected remote arg: ftp.cd.debian.org::debian
rsync error: syntax or usage error (code 1) at main.c(1232) [sender=3.0.9]
Do not exclude source to resolve this error.

Push mode mirror server

When there is a change in the archive, the upstream source mirror server will actively push a synchronization notification to the downstream mirror server, and then the downstream source mirror server can update its own archive in time, which is push mode mirroring . This mode is used between the debian master server and the downstream mirror server.

This mode requires the downstream mirror server to configure the ssh service, and the upstream mirror server uses ssh to notify the downstream server. You can use a normal user to accept notifications, save the public key of the upstream mirror server in the ~/.ssh/authorized_keys file, and add the following statement to this file to limit the permissions of the upstream mirror server

no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command=”~/bin/ftpsync”,from=”ip_address”

Here ip_address is the IP address of the upstream mirror server.
Moreover, the upstream upstream mirror server can also authorize who can push to certain downstream mirror servers through the username/password, and these usernames/passwords are isolated from the system instead of using /etc/passwd to further enhance security.

Synchronization using push mode requires the upstream source mirror server maintainer to be informed of the downstream downstream server address, ssh port and user used

For the source mirror server within the enterprise, it is not necessary to use the push mode, as long as it is synchronized with the upstream source mirror server regularly during the idle period.

Let ftpsync run automatically

Use cron to make ftpsync run automatically at regular intervals. Add the file ftpsync to the /etc/cron.d/ directory with the following contents:

SHELL=/bin/bash
PATH=/home/username/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

#minute hour day_of_month month day_of_week user command
0 1 * * * username ftpsync

The username is the user who runs the ftpsync script, and the ftpsync script is automatically run at 1:00 am every day to synchronize with the upstream source mirror server

Source mirror http configuration

Other users of the intranet need to update the system from the local source mirror server through the http or ftp protocol. Here, apache is used to provide the source mirror service in http mode

# apt-get install apache2

Add the file debian_mirror to the /etc/apache2/sites-available/ directory with the following contents:

1 <VirtualHost *:80>
 2 
 3     DocumentRoot /srv/mirrors/debian
 4     #<Directory />
 5     #    Options FollowSymLinks
 6     #    AllowOverride None
 7     #</Directory>
 8     <Directory /srv/mirrors/debian>
 9         Options Indexes SymlinksIfOwnerMatch FollowSymLinks MultiViews
10         IndexOptions NameWidth=* SuppressDescription
11         AllowOverride None
12         Order allow,deny
13         allow from all
14     </Directory>
15 
16     ErrorLog ${APACHE_LOG_DIR}/debian_mirror_error.log
17 
18     # Possible values include: debug, info, notice, warn, error, crit,
19     # alert, emerg.
20     LogLevel warn
21 
22     CustomLog ${APACHE_LOG_DIR}/debian_mirror_access.log combined
23 </VirtualHost>

Then create a new symbolic link in the /etc/apache2/sites-enabled directory, set it as the default website here, or use a virtual host

# rm 000-default
# ln -sf /etc/apache2/sites-available/debian_mirror 000-default

finally

# rm 000-default
# ln -sf /etc/apache2/sites-available/debian_mirror 000-default

Use local source mirror service

Edit /etc/apt/source.list, add

deb http://mirror_ip wheezy main contrib non-free
deb http://mirror_ip wheezy-updates main contrib non-free
deb http://mirror_ip wheezy-proposed-updates main contrib non-free

mirror_ip is the IP address of the newly created source mirror server