Docker usage in Tianchi competition

Docker competition usage process

You can learn from the following introductory tutorial: https://tianchi.aliyun.com/competition/entrance/231759/tab/226

Here is a detailed explanation: https://www.cnblogs.com/shayue/p/12364155.html

1. Establish Alibaba Cloud Image Warehouse

First create a private warehouse at https://cr.console.aliyun.com/cn-shanghai/instances/repositories and define the namespace. Click the management page on the right to see some common operations, such as logging in to your account, pulling images, pushing images, etc.

# 登陆账号
$ sudo docker login --username=火星娃勇闯魔晶岛 registry.cn-shanghai.aliyuncs.com
# 从registry拉取镜像
$ sudo docker pull registry.cn-shanghai.aliyuncs.com/hxm_docker_test/mmdet_tc_2021:[镜像版本号]
# 推送到registry
$ sudo docker login --username=火星娃勇闯魔晶岛 registry.cn-shanghai.aliyuncs.com
$ sudo docker tag [ImageId] registry.cn-shanghai.aliyuncs.com/hxm_docker_test/mmdet_tc_2021:[镜像版本号]
$ sudo docker push registry.cn-shanghai.aliyuncs.com/hxm_docker_test/mmdet_tc_2021:[镜像版本号]

2. Configure local mirroring

1. First create a submission folder

   This needs to be designed according to the requirements of the competition (generally including Dockerfile (fixed name), run.sh, result.json, etc.)

   The design of Dockerfile is as follows:

   # Base Images
   ## 从天池基础镜像构建
   FROM registry.cn-shanghai.aliyuncs.com/tcc-public/python:3
   # 这个时ali的镜像列表
   # https://tianchi.aliyun.com/forum/postDetail?spm=5176.12586973.0.0.28cd6448z1i5oZ&postId=67720
   
   
   ## 把当前文件夹里的文件构建到镜像的根目录下
   ADD . /
   
   ## 指定默认工作目录为根目录（需要把run.sh和生成的结果文件都放在该文件夹下，提交后才能运行）
   WORKDIR /
   
   

After the image is started, execute sh run.sh uniformly.

CMD [“sh”, “run.sh”]

run.sh的设计如下：
   python hello_world.py

3. Testing of local mirroring

1. First create a mirror ( registry.~~~that is your public network address, replace it with your own warehouse address, which ：1.0is the version number)

   docker build -t registry.cn-shenzhen.aliyuncs.com/test_for_tianchi/test_for_tianchi_submit:1.0 .
   

   You may encounter some problems here:

   • If you are not logged in to your account, please log in with the command 一in use and then upload it again.登陆账号

   • Duplicate version numbers will cause the previous image TAGto change <none>. Therefore, if the previous image is no longer needed, delete it in time. The deletion command is as follows:

     docker rmi IMAGE ID
     # 如果遇到无法删除的情况，那么先用如下命令查看一下container在占用
     docker ps -a 
     #完了用如下命令删除对应的containrer
     docker rm CONTAINER ID
     

2. After building the image, you can use the following command to test it to see if there are any problems.

   CPU image: docker run your_image sh run.sh
   GPU image: (to be verified)

   # 使用所有GPU
   $ docker run --gpus all nvidia/cuda:9.0-base nvidia-smi
   
   # 使用两个GPU
   $ docker run --gpus 2 nvidia/cuda:9.0-base nvidia-smi
   
   # 指定GPU运行
   $ docker run --gpus '"device=1,2"' nvidia/cuda:9.0-base nvidia-smi
   

4. Upload local image

1. If there are no problems before, just upload it to your own warehouse.

   docker push registry.cn-shenzhen.aliyuncs.com/test_for_tianchi/test_for_tianchi_submit:1.0
   

   • If you encounter an upload failure, you may not be logged in to your account. Log in using the command 一in question and then upload again.登陆账号

5. Submission of competition results

1. Just configure your own image path, username and password.

6. Commonly used commands

# 查看docker 列表
docker images
# 查看容器列表
docker ps -a
# 启动,进入docker 使用bash方式
dockers run -it IMAGE ID  /bin/bash
# 删除容器
docker rm CONTAINER ID
# 退出镜像
exit
# 进入容器
docker attach CONTAINER ID
# 把容器保存成镜像
docker commit CONTAINER ID  REPOSITY_NAME:TAG
# GPU运行docker
docker run --gpus  '"device=3"' -it --name  hxm_test(CONTAINER_NAME)  IMAGE_ID  /bin/bash