There are many articles on how developers have reduced productivity by structuring their daily work. A common example is: There are many non-essential meetings scheduled throughout the day, so no one can enter the deep focus mode. Today, I want to study the biggest killer of developer productivity: the way to configure and set up DevOps workflows. In almost all cases, I have encountered some shortcuts that can help you avoid most problems.
#1: Fully invest in microservices without proper tools
When the project is working in the overall setting, all work can be done. The tool chain is ready to handle this whole well, but to change a small part, you need to deploy the whole whole. You need to run end-to-end tests to verify that everything is still working. The larger the overall, the lower the efficiency. Therefore, the team continues to move forward and adopt microservices. Their first experience is great. Colleagues can perform separate services independently, the deployment frequency increases, and everyone is very happy.
The problem started when the team did not use microservices and paid too much attention to "micro". From a tool point of view, you will now have to deal with more yml files, docker files, and dependencies between these service variables, routing issues, etc. They need to be updated and maintained. Your CI/CD settings, organizational structure, and total number of personnel may need to be readjusted.
If you enter microservices for any reason, make sure to plan enough time to reorganize tool settings and workflows . Just count the number of scripts in each location you need to maintain. Think about how long this will take, who is responsible, and what tools can help you control this situation. If you choose tools, make sure they have a user community.
2: Non-general placement external container
In many cases, containerization is an amazing technology. However, it carries a value label and may affect your productivity. From a security perspective and through the necessary configuration and environmental management, containers will increase overhead. If you disagree with certain agreements of the team, then the container will also harm your productivity and developer experience.
The most common mistake I see is to build configuration files or environment variables into the container. The core idea of containerization is portability. By hard coding the configuration, you will have to start writing files and pipelines for each individual environment. Do you want to change the URL? Great, please continue to make changes in 20 different places and then rebuild everything.
Before you start large-scale use and use containers in production environments, sit down and agree to configuration conventions that are important to you. Make sure to introduce this consistently in code reviews and reviews. Reconstructing such a priori is a pain.
3: Misuse of KUBERNETES
Everyone hyped up this open source project called Kubernetes. However, Kubernetes is difficult to keep running, it is also difficult to integrate into your developer process, while maintaining high productivity and experience. Many things can go wrong:
Worst-case scenario for Kubernetes : Colleague XY really wanted to get his hands dirty and found an introductory guide online. They set up a cluster on bare metal and it worked well with the test application. Then they started to migrate the first application and asked their colleagues to start using kubectl to interact with the cluster. Now, half of the team is focused on learning this new technology. Now, the poor person who is maintaining the cluster will work full-time when the second production workload reaches the first. The CI/CD setup is completely unprepared to deal with this, and because the entire team is trying to master Kubernetes, overall productivity is declining.
What can be done to prevent this: Kubernetes is a great technology, and if done correctly, it can help to get a PaaS-like developer experience. After all, it is a descendant of Borg. Borg is a platform built by Google that allows its software engineers to easily build massively scalable applications. Therefore, it is an open source interpretation of Google's internal platform.
Best practices:
- As far as possible, the team should not build and run barebones clusters by themselves, but should use managed Kubernetes services. Read reviews about hosting a Kubernetes cluster that best suits your needs. At the time of writing this article, from a purely technical point of view, Google Kubernetes Engine (GKE) is by far the best (although the permissions architecture is still painful-permissions issues, Google?) followed by Azure Kubernetes Service (AKS). Amazon's Elastic Kubernetes service (Amazon EKS) is catching up.
- Use automation platforms or continuous delivery APIs. They allow you to run workloads on K8 without being seen by the developers. The complexity of exposing everyone to the entire setup is almost zero. I know the argument "everyone should be able to do everything", but the pace of change is so fast, and the degree of automation is so high that it really doesn't make sense.
- If the team really wants developers to manage the Kubernetes cluster by themselves, then they should give them enough time to really understand the architecture, design patterns, kubectl, etc., and really focus on it.
4: Forgetting to do continuous delivery
" Wait, I already have a configuration item tool ." A common misconception is that if you have a continuous integration setup, you are doing a good job. You still lack continuous delivery! Many vendors have coined the term "CI/CD tool", which does not confuse you, if you own Jenkins, CircleCI, etc., it gives you the impression of continuous delivery-it is not.
The carefully adjusted "continuous delivery" settings (whether self-written or "as-a-service" settings) are the "glue" in the team tool chain:
- It enables all different components from source code control systems to CI-Pipeline, from databases to clusters, and from DNS settings to IaC, to be integrated into a simplified and convenient developer experience.
- This is a way to structure, maintain, and manage a growing number of YML and configuration scripts. If done well, this will allow your developers to use the artifacts built by CI-Pipeline to dynamically start the environment and fully configure it through a pre-configured database and everything that has been set up.
- It can be used as a version control system for configuration status, and has auditable records to record where it is deployed and in what configuration it runs, and allows you to roll back and forth and manage blue/green/canary deployments.
- Through well-designed CD settings, you can change the efficiency of developers. They enable developers to self-service, reduce dependencies within the team, and improve the maintainability of settings.
- Teams that use these practices will release more frequently and faster, showing overall higher performance and satisfaction.
5: Unsustainable test automation
Without automation, effective testing is impossible. Continuous delivery brings the continuous responsibility of not destroying anything.
You need to constantly make sure not to fall into an inverted pyramid testing of traps . To do this, you need to be able to run the right tests at the right point in the development life cycle.
Enough CI tools will help you put unit and integration tests in the right place, and CD tools with configuration management and environmental management will help you run automated end-to-end tests in a reliable way.
Well-done settings allow developers or testers to dynamically start a pre-configured environment. Strictly externalize your configuration and ensure that you have configuration management that injects these variables at deployment time. This has brought many positive improvements:
- Run the right tests at the right time while providing effective feedback to the development team
- Developers can gain autonomy, you can reduce the dependence of key people,
- QA personnel can now test a subset of functional environments,
- Quality inspectors can parallelize testing, which saves time and can test a subset of the data.
6: Manage the database yourself
The teammate who just left is responsible for setting up MongoDB for the client project, and of course using the open source project to run it on their own. Of course, the switch is "perfect." Of course, the database is not properly protected. One night, it shows where the data should be:
And of course: you check the backup. A syntax error occurred. Now you must reverse engineer all the data. This is a real example that often happens.
Self-managed databases have operational and security risks. They are distracting, boring and unnecessary. Use Cloud SQL or other products and get a good night's sleep. We usually see hosting products provided by companies such as Aiven.io . These companies provide most of the databases, they can run them on all the large cloud providers for you, and they have more features, are more mature and more complex. Moreover, they are generally cheaper and ensure zero lock-in, while at the same time providing more convenience for developers, and I will always want to do this if we keep pace.
7: Going to cloudy for no reason
There is a difference between just using multi-cloud and trying to design the system to be agnostic and portable. The latter has many different advantages, such as a dynamic environment, and makes more sense than using multi-cloud. Of course, this is a legacy: some teams have been using GCP, while other departments started with AWS and are here now. Others include specialization. One might say that GPU runs more efficiently on GCP than on AWS or cost reasons. But to really surface, you need a sufficient size. Simple multi-cloud settings require a high degree of automation and require developers to shield configuration and setup tasks. Otherwise, you will end up in script hell.
As a general rule: If it is not absolutely necessary, please do not make it cloudy.
in conclusion
I hope these views will help you avoid the biggest mistakes in this field. Remember Nicole Forsgren, Jez Humble, and Gene Kim wrote in their book "Acceleration" : "The top 1% of teams Delivery frequency is 10 times that of the original."
This is because they are taking full advantage of everything today. I spend an hour every month reviewing my personal workflow, to-do list, and how to organize the application. why? Because if your process is inefficient, it will actually add up within a few weeks. These tiny things (such as searching your photo app) can distract your brain. Stop for an afternoon a month to ensure your productivity is streamlined. This will help you focus on innovation instead of configuration, making the team happier.