Preface
When a service goes online, it is often necessary to move the service to a designated server. Docker is often used to record the use of the dcoker command in the work.
1.Write Dockerfile
1.1 New image
FROM nvidia/cuda:11.7.1-devel-ubuntu22.04
ENV WORKDIR=/data/Qwen-14B-Chat
WORKDIR $WORKDIR
ADD . $WORKDIR/
RUN apt-get update && apt-get -y --no-install-recommends install vim curl wget build-essential python3.10 python3-pip && update-alternatives --install /usr/bin/python python /usr/bin/python3.10 1 && pip3 install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
RUN pip install flash_attn-2.3.3+cu117torch1.13cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
#CMD ["sh", "-c", "python3 api.py -c /data/model/Qwen-7B-Chat -p 9092 -n 0.0.0.0"]
entrypoint tail -f /dev/null1.2 Copy the image
FROM qwen-14b:v1.0
#ENTRYPOINT ["tail", "-f", "/dev/null"]
COPY ./api.py /data/Qwen-14B-Chat/api.py
entrypoint tail -f /dev/null
1.3 Dockerfile built-in startup command
FROM qwen-14b:v1.0
COPY ./api.py /data/Qwen-14B-Chat/api.py
EXPOSE 9094
ENTRYPOINT cd /data/Qwen-14B-Chat && /usr/bin/python api.py -c /data/model/Qwen-14B-Chat -p 9094 -n 0.0.0.0
#ENTRYPOINT tail -f /dev/null
Use the Dockerfile built-in command to start the service. After creating the image and starting the image as a container, the service can be started. There is no need to enter the container and use the python command to start the service. You can use the following command to check whether the service in the container is started.
docker logs -f 容器id2. Create a docker image
docker build -f /ssd/dongzhenheng/Qwen-14B-Chat/Dockerfile -t qwen-1b:v1.0 . 3. Check the docker image id
docker images4. Start docker based on the image id
Use docker run to start the image. Set the externally mounted model /ssd/dongzhenheng/LLM/Qwen-14B-Chat/ and map it to the /data/model/Qwen-14B-Chat path in the container.
docker run --gpus all -v /ssd/dongzhenheng/LLM/Qwen-14B-Chat/:/data/model/Qwen-14B-Chat -p 9093:9093 -d 695aa69fcf3a /bin5. Use docker ps to view the started container ID
docker ps6. Enter the container according to the container ID
The container id is 2733d1a26fd0
docker exec -it 2733d1a26fd0 bash7. View file container files
cat aip.py8. Start the service
python api.py -c /data/model/Qwen-14B-Chat -p 9093 -n 0.0.0.0
9.Exit the container
control+A+D10. Stop the container
docker stop 529d9dee7ed111. Delete the image
docker rm 容器id
docker rmi 镜像id
#强制删除
docker rmi -f 镜像id12.docker save
Use the docker save command to save the image as a tar file, which contains all layers and metadata of the image. It can be copied to any other machine with Docker installed and loaded back using the docker load command.
docker save 695aa69fcf3a -o qwen-14b.tar13.docker load
docker load is the command used in the Docker command line tool to load the Docker image saved in the file. It can load back the Docker image saved previously using the docker save command.
docker load -I qwen-14b.tar14.docker tag
docker tag is the command used in the Docker command line tool to label images. It can mark an image with a new label for easier identification and management.
docker tag qwen-14b qwen-14b:v1.015. docker login
docker login is the command used in the Docker command line tool to log in to the Docker image repository. It allows users to authenticate with a given username and password in order to perform operations in a Docker registry, such as pulling and pushing images.
When executing the docker login command, you need to provide a valid username and password to authenticate and log in to the Docker image repository. After successful login, the terminal will display a message indicating that the login was successful, and Docker will store the logged-in warehouse address and credentials in the ~/.docker/config.json file for subsequent use.
docker login -u myusername -p mypassword docker.io16.docker push
docker push is the command used in the Docker command line tool to upload the local Docker image to the Docker image warehouse. Before executing the docker push command, you need to use the docker login command to log in to the Docker image warehouse.
docker push 用户名/仓库名称/myimage:v1.0This will upload a local image named myimage to Docker Hub and mark it as v1.0. Please make sure to replace username/repositoryname and myimage with your actual username and image name.