"Docker Buildx"-Build a "cross-platform" image (study notes) @20210224

Problem Description

For the background of the problem, refer to the Multi-arch build notes, here is a brief overview.

What does "cross-platform mirroring" mean? Let's take nginx:latest as an example. The image supports eight architectures, as shown in the screenshot below:

In x86, execute the docker pull nginx:latest command, it will return the image with a summary of 044451886742;
in arm64, execute the docker pull nginx:latest command, it will return the image with the summary as 4f1e67ed43f3;
in amd64, execute docker pull nginx:latest Command, it will return the image with the summary of b08ecc9f7997;

Using Buildx to build a multi-architecture image is a very convenient way.

This note will record: how to use buildx to build a multi-platform image and how to deal with common problems.

solution

Buildx is a CLI plug-in that extends docker commands and supports all Moby BuildKit features. The usage is the same as docker build but supports many new features, such as:
1) Create a scoped builder instance;
2) Build in parallel for multiple nodes;

The first step, installation and activation

The installation process is as follows (for details, refer to the README.md/Installing document)

# First step, download and install the command 

mkdir -pv ~/.docker/cli-plugins/ 
wget -O ~/.docker/cli-plugins/docker-buildx \ 
    https://github.com/docker/buildx/releases /download/v0.5.1/buildx-v0.5.1.linux-amd64 
chmod a+x ~/.docker/cli-plugins/docker-buildx 

# Second step, set experimental parameters 

vim ~/.docker/config.json 
{ 
    ... 
    "experimental": "enabled", 
    ... 
}

Install docker buildx as the default build command. Then, executing docker build is equivalent to the docker buildx command:

docker buildx install
docker buildx uninstall # 卸载

Use the Buildx command to build the image

There are two drivers: docker and docker-container (but there are some differences between the two):
1) docker, use BuildKit tool bound to dockerd;
2) docker-container, use container to run BuildKit tool;

Use build examples to build images using different nodes

By building multiple instances, you can freely switch between instances, and then execute the build tasks on different hosts.

Use multi-platform build to build images of multiple platforms

Multiple platform images can be built.

Advanced build options

Through the docker bake command, multiple images can be built in parallel.

Application case: Building a multi-platform image (basic case)

Just like the question at the beginning of the article, this case is also something we want to accomplish.

Simple case

The following example is just to let us understand the general process of using buildx:

# Get the image we want to build 
git clone https://github.com/kstaken/dockerfile-examples.git 
cd dockerfile-examples/rethinkdb 

# Create a build instance. In 
layman's terms , it is to create a node for building docker buildx create --use --name build --node build --driver-opt network=host 

# Execute the build command. 
docker buildx build \ 
    --tag 0xa0000/buildx-example:latest \ 
    --platform linux/amd64,linux/arm64. 

# If you use the --push option, then after the build is completed, the mirror warehouse will be pushed immediately 
docker buildx build - push \ 
    --tag 0xa0000/buildx-example:latest \ 
    --platform linux/amd64,linux/arm64.

Complex case

docker buildx create --name multi-platform --use \
    --platform linux/amd64 \
    --driver docker-container "build-node-amd64"
    
docker buildx create --name multi-platform --append \
    --platform linux/arm64 \
    --driver docker-container "build-node-arm64"
    
docker buildx build --progress plain --output "type=image,push=false"  \
    --file "/data/cita-monitor/agent/cita_exporter/Dockerfile" \
    --tag citamon/agent-cita-exporter:20.2.2     \
    --platform linux/arm64,linux/amd64 \
    /data/cita-monitor/agent/cita_exporter

Summary of common problems

auto-push is currently not implemented for docker driver

buildx: auto-push is currently not implemented for docker driver · Issue #4991 · docker/for-win

Problem description: The following error occurs when executing docker buildx:

# docker buildx build --push \
    --platform linux/arm/v7,linux/arm64/v8,linux/amd64 \
    --tag your-username/multiarch-example:buildx-latest \
    ./
auto-push is currently not implemented for docker driver

The cause of the problem: When using multi-platform build, you need to create a build instance first

解决方法：docker buildx create --use --name build --node build --driver-opt network=host

multiple platforms feature is currently not supported for docker driver

When executing the docker buildx build command, the following error is generated:

# docker buildx build \
>     --tag 0xa0000/buildx-example:latest \
>     --platform linux/amd64,linux/arm64 .
[+] Building 0.0s (0/0)
error: multiple platforms feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")

Cannot autolaunch D-Bus without X11 $DISPLAY

docker login fails while docker-compose is installed on Ubuntu 18.04 · Issue #6023 · docker/compose

The cause of the problem: In config.json, we use a credential storage method that is not supported by the system. This method will link to the D-Bus service, which will cause this error.

Solution: Remove the "credsStore": "secretservice" configuration of ~/.docker/config.json

unable to upgrade to tcp, received 200

references

Docker Buildx | Docker Documentation
docker buildx build | Docker Documentation