1. Introduction to Devops
Basic introduction click on the link
1. What is DevOps?
A: DevOps is the gray area between development (Dev) and operations (Ops) teams in the product development process. DevOps is a culture that emphasizes communication, integration and collaboration in the product development cycle. As such, it removes silos between software development and operations teams, enabling them to integrate and deploy products quickly and continuously.
DevOps is the combination of development (Development), testing (QA), and operation and maintenance (Operations). DevOps is a software development methodology that involves continuous development, continuous testing, continuous integration, continuous deployment, and continuous monitoring of software throughout the development lifecycle.
Coding——"Packaging—"Testing—"Release—"Deployment—"Operation and Maintenance—"Monitoring
2. What is continuous integration?
A: Continuous integration (abbreviated as CI) is a software development practice in which team development members often inherit their work. Utilize automated tests to verify and assert that their code does not conflict with existing codebases. Ideally, code changes should be automated on a daily basis with the help of CI tools, with automated builds (compilation, release, automated testing) at each commit, so that inheritance errors can be found early to ensure that the merged code does not break the main branch.
3. What is continuous delivery
Answer: Continuous delivery (abbreviated as CD) and continuous integration provide a complete process for delivering code packages. During this phase, automated build tools are used to compile the artifact and make it ready for delivery to end users. Its goal is to make software builds, tests, and releases faster and more frequent. This approach can reduce the cost and time of software development and reduce risks.
4. What is continuous deployment
A: Continuous deployment is about taking continuous delivery to the next level by integrating new code changes and automatically delivering them to release branches. More specifically, once updates have passed all stages of the production process, they are deployed directly to end users without human intervention. Therefore, to successfully leverage continuous deployment, software artifacts must pass through rigorously established automated tests and tools before being deployed into production.
Benefits: With Continuous Deployment, developers can fully focus on the product, since their last task in the pipeline is to review and merge pull requests into branches. This approach enables rapid deployment and reduces deployment duration by releasing new features and fixes immediately after automated testing.
5. Reasons to implement DevOps
（1）Devops为什么重要？Devops如何使团队在软件交付方面受益？
A: In today's digital world, organizations must reinvent their product deployment systems to be flexible. Devops plays a vital role in generating mobility and agility for the entire software development pipeline, from ideation to deployment to end users. Devops is the solution that brings together a more streamlined, more efficient process of constantly updating and improving products.
（2）解释Devops对开发人员有何帮助？
A: In a world without DevOps, the developer workflow would start by building new code, delivering it, and integrating it, and then it would be the operations team's responsibility to package and deploy the code. After that, they will have to wait for feedback. And if something goes wrong, they will have to do it all over again due to a mistake, countless manual communications between the different teams involved in the project.
Since CI/CD practices have merged to automate the rest of the tasks, applying Devops can reduce the developer's task to just building the code. Bringing engineering and operations teams together helps create better communication and collaboration as processes become more transparent and accessible to all team members.
(3) In short, a well-planned and executed CI/CD pipeline speeds up release speed and reliability while mitigating code changes and defects in the product. This ultimately leads to higher customer satisfaction.
6. How to implement DevOps effectively
Define a typical DevOps workflow. The typical DevOps workflow can simplify 4 stages:
Version Control: This is the stage where source code is stored and managed. Version control contains different versions of the code
Continuous Integration: In this step, developers start to build components, and they are compiled, verified, and then tested through code reviews, unit tests, and integration tests.
Continuous Delivery: This is the next level of continuous integration, where the release and testing process is fully automated. CDs ensure fast and sustainable delivery of new releases to end users.
Continuous Deployment: After the application successfully passes all testing requirements, it is automatically deployed to the production server for release without any human intervention.
7. What tools does DevOps use?
持续开发：Git、SVN、Mercurial、CVS、Jira
持续整合：Jenkins、Bamboo、CircleCI
持续交付：Nexus、Archiva、Tomcat
持续部署：Puppet、Chef、Docker
持续监控：Splunk、ELK Stack、Nagios
持续测试：Selenium、Katalon Studio
The client pushes the code to the git warehouse, and a webHook is configured on gitlab to trigger the construction of Jenkins. Entering the range of the dotted line of Jenkins, it does a lot of things, such as building code from mvn, statically analyzing the code, and doing unit testing. After the test is passed, the image can be built. After the image is built successfully, the image is pushed to the Harbor image. In the warehouse, after the image push is pushed to the mirror warehouse, we can call the rest API of the kubernetes cluster to update the service, and then kubernetes receives the update command and pulls the mirror image from the Harbor mirror warehouse pull to complete the update and restart of the service. Clients to access the services of the kubernetes cluster
1. Development Obtain the basic image from the image library, and develop the application in a containerized manner;
2. Develop and submit code to Gitlab (implement Gitlab service in Kubernetes, and save user data through persistent storage);
3. Gitlab triggers Jenkins master through webhook after receiving the code submission request
Code change→trigger jenkins build→pull code to jenkins node→mvn war package→test→dockerfile mirror→docker tag→docker push to harbor→k8s deployment
4. Jenkins master packages the source code in the slave node after receiving the request;
5. After the source code is packaged, according to the pipeline, obtain the dockerfile from Gitlab, and generate docker images in the slave node;
6. After the Docker image is generated, upload it to the Docker private warehouse harbor;
8. Pull the image in the Kubernetes test environment through the Jenkins pipeline and deploy the application;
9. After the test is successful, deploy the application in the Kubernetes production environment through the Jenkins pipeline.
8. Publish application scenarios
Publish application scenarios:
蓝绿发布: The two environments are upgraded alternately, and the old version is kept for a certain period of time for easy rollback.
灰度发布: Upgrade the old version according to the proportion, for example, 80% of the user visits are the old version, and 20% of the user visits are the new version.
滚动发布: Stop the old version instances in batches and start the new version instances.
Blue-green release:
The project is logically divided into groups A and B. In the project system, group A is first removed from the load balancing, and the new version is deployed. Group B still continues to provide services.
When group A is upgraded, the load balancer reconnects to group A, and removes group B from the load list to deploy the new version. Group A resumes service.
Finally, the upgrade of group B is also completed, and the load balancing is reconnected to group B. At this time, the version of group AB has been upgraded and provides services to the outside world.
Grayscale release:
Grayscale release only upgrades some services, that is, let some users continue to use the old version, and some users start to use the new version. If users have no complaints about the new version, then gradually expand the scope and migrate all users to the new version Come.
2. DevOps commonly used tools
Code Management (SCM): GitHub, GitLab, BitBucket, SubVersion
Build Tools: Ant, Gradle, maven
Automatic Deployment: Capistrano, CodeDeploy
Continuous Integration (CI): Bamboo, Hudson, Jenkins
Configuration Management: Ansible, Chef, Puppet, SaltStack, ScriptRock GuardRail
container: Docker, LXC, third-party vendors such as AWS
Orchestration: Kubernetes, Core, Apache Mesos, DC/OS
Service registration and discovery: Zookeeper, etcd, Consul
Scripting language: python, ruby, shell
Log management: ELK, Logentries
System monitoring : Datadog, Graphite, Icinga, Nagios
Performance monitoring: AppDynamics, New Relic, Splunk
Stress testing: JMeter, Blaze Meter, loader.io Early warning: PagerDuty, pingdom, manufacturers’ own HTTP accelerators
such as AWS SNS : Varnish message bus: ActiveMQ, SQS Application server: Tomcat, JBoss Web server: Apache, Nginx, IIS Database: MySQL, Oracle, PostgreSQL and other relational databases; cassandra, mongoDB, redis and other NoSQL databases
Project Management (PM): Jira, Asana, Taiga, Trello, Basecamp, Pivotal Tracker
3. DevOps concept and the whole process of building (Jenkins, Harbor, SonarQube, K8s)
For specific content, click the link to introduce
relevant content in detail. Video reference
DevOps concept and the whole process of building (Jenkins, Harbor, SonarQube, K8s)
The overall software development process includes:
PLAN: The development team makes a development plan according to the client's goals
CODE: According to the PLAN to start the coding process, different versions of the code need to be stored in a library.
BUILD: After the coding is completed, the code needs to be built and run.
TEST: After successfully building the project, you need to test the code for bugs or errors.
DEPLOY: After the code has passed manual testing and automated testing, it is determined that the code is ready to be deployed and handed over to the operation and maintenance team.
OPERATE: The operation and maintenance team deploys the code to the production environment.
MONITOR: After the project is deployed and launched, the product needs to be continuously monitored.
INTEGRATE: Then send the feedback received in the monitoring stage back to the PLAN stage. The overall iterative process is the core of DevOps, namely continuous integration and continuous deployment.