Actually it took a week to solve the problem

Summary

Never thought of a recent mission actually is not resolved during the week, made Saturday a busy day at home, not yet complete, a flash until Sunday morning, was completed. Frankly, I have not had this experience for a long time, and was troubled by a technical problem a few days, this kind of dazed, but fortunately finally solved the problem. While you may not encounter this particular task similar to, but I still want to share the process of resolving the problem for everyone, the way to talk about how to solve technical problems.

Background problem

You need to be an open source python project company's access to micro-service system (mainly java) in. Open source project referred to below opa. opaSome projects depend on the underlying c ++ libraries, such as openImageIO .

problem analysis

When received this task, in fact, I was pretty nervous, the time is tight, almost a week you have to complete. Combined with their own situation, to assess this task there are some challenges. Mainly

• I have not written a few years python code, c ++, and unfamiliar.
• Lack of experience in building complex docker image
• No access to the python language service experience in micro systems
• War alone, with no consultation of objects

First, I will make this task a number of dismantling and assessed under the difficulty out of 5 stars

• Build a python development environment, you need to run from one local opaproject, develop, debug. ***
• The opaproject transformed into a web project, providing restful interface. **
• The service registered with the service center *
• Provide java client for external calls *
• build docker image, deploy the services on k8s. ***

I never thought this task actually encountered so many problems in the process of resolving

Resolution process

The opaproject transformed into a web project

This task is a difficult start evaluating index 3 stars, actually found that although there are some difficulties, but can be done using search engines, composite index 2 star, the actual time spent 1d

IDE: pycharm
环境: py2/py3
包构建: pip
复制代码

There are major problems encountered

python2, python3 problems development environment is installed Py3, opaProject is py2 project, so the use of tools converted under python turn python3

Under had to Tucao, python3 actually incompatible syntax python2, such as print and so on. But the syntax in python2 python3 can run inside, but this is a little pit father 2-> 3 major versions, can indeed do so, and locked in, without discarding the old things, increase maintenance costs

Package version issues

This for me is to write java, python package management is simply nonsense, and with not too much to describe the shock. Trough point

• No project dependencies file
• Instead of specifying the package version

But because much of the project introduced package, I do not have too tangled this point, installed on a single individual up. If it is positive children need to maintain the eight projects, python now have, there is a need of the venue and package management tool for the

The opaproject transformed into a web project

This task is completed faster than expected, the use of flaskthe package, with the flask tutorial actually get to spend a day

The service registered with the service center && java client to provide external calls

This problem is not the content, pure manual labor, hundreds of times a day to solve

build docker image, deploy the services on k8s

This step is a 3 star begin to evaluate, really did not expect to torment me, I spent two days have not been resolved. Get resolved before the weekend. Problems encountered 10+, with some notes recorded some

1. pip installfound no version of a package, version upgrade, then need python3, in fact, the environment is python2 xbuba.com/questions/5...

Final settlement by specifying the version

pip install -i https://mirrors.aliyun.com/pypi/simple  numpy==1.16.4 

复制代码

Question 2. Network

这个问题折磨的我痛不欲生，欲哭无泪。因为需要验证构建是否正确，要反复的build image,每次install 各种package, pip install 
python包，需要半小时才完成的了。在家连vpn，网络更加慢，时间更长，这让我无数次拷问自己，都2019年了，我居然还在被网络带宽困扰，人活着的意义到底是为了什么，我为什么要把我美好的人生浪费在这里。

这个问题主要通过3方面解决

- docker multi stage build 

  构建多个docker base镜像，然后from base镜像，这样可以减少image的构建时间

-  更改下载软件的地址源

   比如apt,yum的,pip 的。

- 将另外一些需要wget的包下载下来，扔到内网，访问内网地址
复制代码

3.docker image constructed from need to build a base image based on the company's internal java base image, and the python project. A beginning for granted

FROM opa.image
FROM company.image
复制代码

In fact docker can only inherit from a base, the first FROM will be ignored. It requires the use of multi-stage build, copy the contents of the previous layer to a new layer details can be multi-stage build

Problems 4.c ++ dynamic link library

Construction of multi-stage time, the opa time of image over content copy, perform error bin / time. Lack of c ++ libraries. c ++ libraries are needed to make the kernel directly copy the file to no avail.

5.ubuntu, centos questions are based on the company's image centos system, based on ubuntu, and some are not compatible syntax open source.

Ultimate solution

My biggest problem plagued the problem is to build a docker image problem, how is based on two base image, allow to build a c ++, python, java's docker container. Solve the last problem comes from the wake of a flash. Project requires environmental roughly as shown below

I have been entangled in the early issues

• ubuntu,centos
• Various packages install error problem
• Package version of the problem
• c++ 库make install
• Network download problem

So many questions are intertwined, the mind is not clear. My biggest problem plagued the problem is to build a docker image problem, how is based on two base image, allow to build a c ++, python, java's docker container. Solve the last problem comes from the wake of a flash.

Because for me, the main difficulty lies docker image of the original project's build, but inside the company image and add the new code, mainly java runtime environment, run jar package, as well as several pip install python package, which They are relatively simple.

So a change of thinking is no longer based on the image of the company centos environment, but on opathe image, so you can not move the original image. Equivalent to those things I'm not familiar with all blocked.

Building a base image

FROM company.image AS builder
// 将环境基于原来的image
FROM base.image
// copy java环境到image
COPY --from=builder         /usr/lib/jvm/jdk1.8.0_172 /usr/lib/jvm/jdk1.8.0_172
复制代码

Then build python environment based on the base image. The last problem is solved.

to sum up

In the process of solving this problem, I summarize a few basic principles can help you to solve the problem

1. Understand their skills, to grasp the rhythm

   At the beginning of solving the task, good and difficult to assess, and constantly compared with the estimated time of resolution process, exceeded expectations, the need for timely adjustment, firmly grasp the rhythm of live task, make sure everything is being controlled.

2. To distinguish between primary and secondary issues of classification issues, distinguish what is necessary to solve, which is negligible.

   You may encounter many problems, these problems go deeper, will lead to many problems, so deep traversal, will lead to extension of the mandate, there is a deep sense of frustration. Only solve the problem must be addressed, are not necessary but you are interested in the issue, record the follow-up bottom.

3. Divide and conquer your task to solve others may not done, the network can not find the answer. Task needs to be dismantled, there will be a lot of background abstracts the complexity of the problem, simplistic, to consult people who have had relevant experience, can make communication more efficient, faster problem-solving

No. [public concern] abbot of the monastery, the first time to receive the updated article, beginning with the way technology practice of the abbot

reference

xbuba.com/questions/5…

docs.docker.com/develop/dev…

docs.python.org/zh-cn/3.7/l…