Mirror command with minimal curl

Dockerfile follows

FROM alpine:latest
RUN apk add --update curl && rm -rf /var/cache/apk/*

When you install the download speed of curl in Alpine Linux docker mirror is very slow, how to solve?

Step 2/2 : RUN apk update && apk add curl && rm -rf /var/cache/apk/*
 ---> Running in 86c4e9f3daca
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz

The alpine linux apk into a domestic source of installing mirror solve

RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories

Dockerfile adjusted as follows

FROM alpine
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories
RUN apk add --update curl && rm -rf /var/cache/apk/*

Support HTTP2 of cURL-- based Alpine minimize Docker image

cURL is one of my favorite open source software. Although cURL powerful are often taken for granted, but I sincerely think it's worth gratitude and respect. If we lose the toolbox curl, those who need to interact and network with severe (most of us are like this) will fall into the predicament. curlSpeed, small size, and most good tools, simple and clean, with minimal impact on users, only thing they need to do.

If someone wants to use curla feature in (such as UNIX sockets support), and configure the system provides exactly the package does not support this feature, or package version is too old and does not include this function, due to curla "data transmission Swiss Army Knife "in the world, it is conceivable that the situation is likely to occur. So you will find that you want to compile a contains the features you need curl. Compile your own software is frightening, especially when you're not good at solving such problems, let us temporarily put aside the fact that you are using custom software will be an incredible, the liberation of their own experience .

If necessary, you can compile your own software, and is not limited to the others to your package, as will be clear will give you confidence. Suddenly, your mind may be filled with the thrill of excitement - you can install and configure any software the way you like it, do not need to accept the limitations others configuration. Something humans often like to add their own mark, which is one reason why the use of open source software addictive.

Since the separation characteristics Docker file system, it has become the best choice to complete this sort of thing. You do not need to worry about installing dependent libraries or run a lousy make installsimilar order will mess up your local file system. It allows you to operate in a real environment, and allows you to make mistakes. This knowledge is an excellent learning tool, because the thing is to learn the only way to completely mess up, mess up and things will be in a container, you can easily discard these containers, in this way than their own mess the local system to be safer. In addition, if you apply these steps to a script written in the formDockerfile, Then upon subsequent re-constructed building steps will be consistent. Although Dockerfilenot 100% guarantee the results to build, but it is better than READMEfree to describe in much better.

Let us build together a Dockerfileto create a minimum of support HTTP2, the volume of image-based Alpine Linux. It will focus on the reduced image and 100% of the volume of custom curlon.

Method steps

we will:

Discuss why we pay attention to HTTP2?
Briefly read through Dockerfile, so you have a preliminary understanding of the build process.
Discuss why the Alpine as a base image?
Explained in detail Dockerfile, step by step to understand it.
Construction and operation of the mirror.

Why HTTP2?

Quote from https://http2.github.io/ :

Because of the critical status of the HTTP, HTTP / 2 became the HTTP substitute. HTTP / 2 is not completely rewritten HTTP protocol, HTTP method, status code, the semantics are preserved, and should be able to use and HTTP / 1.x (there may be fine-tuned) to represent the same API HTTP / 2 protocol .

HTTP / 2 protocol to focus on performance, especially the end user receiving delay, the use of network and server resources. One of the main objectives is to visit a Web site from the browser can only be achieved through a connection.

Briefly, HTTP2 want to address some of the shortcomings of HTTP / 1.1, including performance issues . By the front link in the sample test cases, CloudFlare statement HTTP / 2 will have on my computer 4 to 8 times faster. Can enhance web pages 4 to 8 times faster? Yes, you heard it right.

Dockerfile

This is supported by the curl of HTTP2 Dockerfile:

FROM alpine:edge

# For nghttp2-dev, we need this respository.
RUN echo https://dl-cdn.alpinelinux.org/alpine/edge/testing >>/etc/apk/repositories 

ENV CURL_VERSION 7.50.1

RUN apk add --update --no-cache openssl openssl-dev nghttp2-dev ca-certificates
RUN apk add --update --no-cache --virtual curldeps g++ make perl && \
wget https://curl.haxx.se/download/curl-$CURL_VERSION.tar.bz2 && \
tar xjvf curl-$CURL_VERSION.tar.bz2 && \
rm curl-$CURL_VERSION.tar.bz2 && \
cd curl-$CURL_VERSION && \
./configure \
    --with-nghttp2=/usr \
    --prefix=/usr \
    --with-ssl \
    --enable-ipv6 \
    --enable-unix-sockets \
    --without-libidn \
    --disable-static \
    --disable-ldap \
    --with-pic && \
make && \
make install && \
cd / && \
rm -r curl-$CURL_VERSION && \
rm -r /var/cache/apk && \
rm -r /usr/share/man && \
apk del curldeps
CMD ["curl"]

More about the build process to do the following things:

We installed some package, which contains, supports SSL (HTTPS) and HTTP2 libraries we need.
We installed some compilers necessary cURL library.
We downloaded and unzipped cURLDE source code (at the time of writing the latest stable version).
We configure, compile, and install curl.
We need to clean up that building, but did not want to rely on retained in the final in the mirror.
We will be the default CMDsetting curl.

Why Alpine?

Alpine Linux is the smallest Linux distribution, it focuses on the safety and speed. Use apkcan quickly install the package, by default, to complete the image only contains the basic UNIX what tasks need, therefore Docker other base image, with respect to volume will be smaller.

Commonly used contrast, no compressed base image (used when writing is: Latest Tags):

Alpine - 4.8MB
Ubuntu - 124.8 MB
Debian - 125.1MB
Centos - 196MB

Imagine, on the network again and again pulling download these images

if you are considering this can produce the same 25 times the value of upgrading the hard disk and bandwidth? In some cases, it is equal, but Alpine are constantly improving and improving every day, and offers a lot of killer-level features, such as the file name to find (examples: What needs to locate a apkpackage that contains the binaries mke2fs, no problem .). When using other tools, I have to spend a lot of time to learn to use their strange ways, and I am very angry, unlike these tools, I am very pleased with the use of Alpine, and it constantly surprises me. Especially in the use of some type of gadget containers, for example curl, mirrored volume reduction is great for me.

Construction of the steps detailed

Let us better understand the Dockerfile.

FROM alpine:edge
# For nghttp2-dev, we need this respository.
RUN echo https://dl-cdn.alpinelinux.org/alpine/edge/testing >>/etc/apk/repositories

Alpine branch of the edge, the nghttp2package (cURL support in HTTP2 necessary package) is only effective in testing warehouse, so this few lines of command to ensure that when we execute apk installwhen nghttp2the package can be installed correctly. Read "How to make cURL support HTTP2" will find documents, nghttp2libraries are required (HTTP2 brought complexity due), and wandering in the Alpine archive, found the edge branch nghttp2is only valid in the testing warehouse.

ENV CURL_VERSION 7.50.1

When cURL released a new version, we want to update the image, we just need to modify a file location - environment variables, 7.50.1expressed cURL latest stable version at the time of writing.

RUN apk add --update --no-cache openssl openssl-dev nghttp2-dev ca-certificates

These are the final we want to remain dependent on the mirror, the default certificate and the library is to allow curlsupport for SSL (HTTPS connection). Note —no-cache, this ensures that the apkresults will not use the extra hard disk space to cache the packet to find the location of the final image will save space.

The next RUN command will generate a file layer (so we can install dependencies, use them, and then clear them, they will not remain in the final mirror). This command more content, step by step, let us look at what has been done in the end they operate.

RUN apk add --update --no-cache --virtual curldeps g++ make perl && \

Above all successfully compile and install curlthe necessary tools. --virtualApk is a very useful characteristic - a virtual package. You can give the collection a package label, and then by using a command apk del virtual-pkg-nameto clear all of them.

wget https://curl.haxx.se/download/curl-$CURL_VERSION.tar.bz2 && \
tar xjvf curl-$CURL_VERSION.tar.bz2 && \
rm curl-$CURL_VERSION.tar.bz2 && \
cd curl-$CURL_VERSION && \

CURL to get the source code archive and extract it, delete the archive (we do not need it after decompression), then use the cdcommand into the source directory.

./configure \
    --with-nghttp2=/usr \
    --prefix=/usr \
    --with-ssl \
    --enable-ipv6 \
    --enable-unix-sockets \
    --without-libidn \
    --disable-static \
    --disable-ldap \
    --with-pic && \
make && \
make install && \

In a familiar ./configure;make;make installbasis of the command cURL plus some unique preferences. --with-nghttp2=/usrIs used to configure HTTP2 support, because we will nghttp2-devinstall in Aline's /usr/libdirectory, when building cURL, the program will automatically /usrlib directory to find a package configuration file. Therefore, you may see the parameters set in other instances /usr/localor other directories.

Most of the other parameters (in addition to —with-ssl) both of the curl are copied from the upstream packets enter linker APKBUILD file. As the Alpine package maintainer more reliable, so I decided to reuse the existing configuration. If I feel too reckless to do so, then I will go in depth, and then from the bottom point of view I need to decide what and what not, but I still hope they include UNIX sockets and support IPV6, so I kept these existing configuration.

cd / && \
rm -r curl-$CURL_VERSION && \
rm -r /var/cache/apk && \
rm -r /usr/share/man && \
apk del curldeps

Above all cleanup.

Reserved build directory (that is, where the binary files are installed), remove the source code directory, run apk del curldeps enter code herecommand to remove a virtual package we created earlier, then removed again /var/cache/apk(this is the package cache, to be honest, I do not know Why use the —no-cacheoption, the cache is still there) and /usr/share/mandirectories (help manual, in the case of the man command has not been installed, it is useless). Some of these clearing operations, especially the removal of cached pages and help, to some extent, can be said is an awfully narrow mirrored volume, after all, they are in fact no more than 1MB. These are run by me du | sort -nafter that in the final image may be unnecessary content, I can only say that I am fanatical pursuit of narrow mirrored volume as possible.

Because more of these operations belong to the same RUN command, so this will eventually produce a relatively small mirror layer, although at the time of the beginning of the command, we have to build the final product depends almost 212MB installed. If these operations are located in different layers, clear operation in fact not actually delete the files in the final image, on the contrary, but these files hidden.

The last one:

CMD ["curl"]

docker run imageCommand will invoke the default curlcommand. Of course, this can also be replaced ENTRYPOINT, but I do not mind CMDcan simply by docker runbeing reassigned.

Build and run-time image

The first is to build, it will be Dockerfilethrown into an empty directory, then run in this directory:

$ docker build -t yourname/curl .

Once you've built a mirror, the mirror run it is very straightforward. Let's check to see whether all that work in accordance with the nghttp2.org described. -sIndicating that the startup quiet mode, --http2indication HTTP2 agreement, -Ito return to the first request, in order to verify that we are using the correct protocol.

$ docker run yourname/curl curl -s --http2 -I https://nghttp2.org
HTTP/2 200 
date: Sat, 06 Aug 2016 21:47:31 GMT
content-type: text/html
last-modified: Thu, 21 Jul 2016 14:06:56 GMT
etag: "5790d700-19e1"
accept-ranges: bytes
content-length: 6625
x-backend-header-rtt: 0.00166
strict-transport-security: max-age=31536000
server: nghttpx nghttp2/1.14.0-DEV
via: 2 nghttpx
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
x-content-type-options: nosniff

Well, everything is working properly. And the final volume maintained at the mirror 16MB. This building for a private custom curl is good, after all, need the support of hundreds of MB compile curl-dependent.