Original: little sister the taste (micro-channel public number ID: xjjdog), please share, reproduced Please keep the source.
A few days ago, a friend engaged in training, advanced java, and I'll have a road map back end, I put this article sent to him "must see! java backend, Sword Zhu Xian " . Today, they want a java backend is currently the most commonly used tools and frameworks, I've just painted such a picture, then I sent him. Although not very wide, but also hope to get his praise. did not expect...
一、消息队列
二、缓存
三、分库分表
四、数据同步
五、通讯
六、微服务
七、分布式工具
八、监控系统
九、调度
十、入口工具
十一、OLT（A）P
十二、CI/CD
十三、问题排查
十四、本地工具
复制代码A message queue
kafka is the most commonly used message queues, especially in big data, has a very high throughput. The rocketmq and rabbitmq, the telecom-level message queue, in the business with more. 2019, and do not stare at JMS and hold the (say is bloated ActiveMQ).
pulsar to solve some of the problems on kafka the birth of the messaging system, relatively young, limited tool chain. Some radical groups after the trial, a good response.
mqtt a particular protocol, used mainly in terms of things, capable of bidirectional communication, message queues belonging to the category.
Second, the cache
Data cache database is an effective way to reduce stress, inner stand-alone java caching, and distributed cache points.
For stand-alone speaking, guava's cache and ehcache are some familiar faces.
For distributed cache, the preference is redis, do not hesitate. Since redis is single-threaded, not suitable for high-consuming operations. So for some of the large amount of data cache, such as pictures, video, etc., use a veteran of the memcached effect will be much better.
JetCache is a Java-based caching system package, providing a unified api and annotations to simplify the use of cache. Similarly SpringCache, support local caching and distributed cache is to simplify the development of weapon.
Third, the sub-library sub-table
Sub-library sub-table, almost every point on the scale of the company, will have their own program. Currently, the recommended driver layer sharding-jdbc, or proxy layer mycat. If you do not have additional operation and maintenance team, I do not want to spend money on other machines, so we chose the former.
If the project involves sub-library sub-table is not much, spring dynamic data source is a very good choice. It is encoded directly in the code, but visually difficult to expand.
If you only need to read and write separation , then the mysql official driver in the replication protocol, is more lightweight alternative.
The above sub-library sub-table components are Ebb Tide, eventually winning product. These components differ from other components selection, the program is established, almost impossible to roll back, so be cautious.
Sub-library sub-table is a small case, prepare a phased library sub-table, is the focus: that is, data synchronization.
Fourth, data synchronization
Domestic use mysql majority of companies, but postgresql virtue of its excellent performance, the utilization rate gradually rising.
No matter what database, real-time data synchronization tools, all from the simulation itself as a repository for data and pull parsing. Specifically, mysql is synchronized by binlog; postgresql using wal log synchronization.
Mysql for it, canal is the most used programs; similar databus is relatively easy to use tool.
Now, canal, maxwell and other tools, support will write data to be synchronized to the mq, the follow-up treatment, a lot of convenience.
For ETL (extract, cleaning, conversion), the basically source, task, sink line, corresponding to the previous function. gobblin, datax, logstash, sqoop, are such tools.
Their main job is how easy custom configuration file, written in a variety of data sources adapter interface. The ETL tool, the tool can be used as the data sync (synchronous particular total amount) is usually based on ID, or the last update time , etc., for processing.
Real-time increment is binlog tools, ETL tools do assist. Usually a data synchronization function requires the participation of multiple components, they together form a whole.
V. Communications
In Java, netty has become a well-deserved web development framework, including socketio on it (and I do not mention the mina). For the http protocol, there are common-httpclient, as well as more lightweight tools okhttp to support.
For a rpc who will have to agree on a means of communication and serialization. json serialization is the most common way, but the big transport costs and parsing, xml and other text-based protocol similar thereto, there are a lot of redundant information; avro and kryo is binary serialization tool, not have these disadvantages, but the debugger inconvenience.
rpc Remote procedure call is meant , which, thrift, dubbo, gRPC default socket communication are binary serialization framework approach; feign, hessian is onhttp remote invocation framework.
By the way, gRPC serialization tool is protobuf, a high compression ratio of binary serialization tool.
Typically, the service response time spent in the main database and business logic, time-consuming communication layer in which small proportion. You can choose according to their company's R & D level and scale of business.
Sixth, micro-services
More than once we when it comes to micro-services, and this time around it's a bunch of us from the support frame, a peek at the system. Yes, this is still talking about spring cloud.
The default registry eureka no longer maintained, consul has become the first choice. nacos, zookeeper the like, can be used as an alternative. Which nacos with background, more suitable for Chinese people use habits.
Fuse components, the official hystrix has not maintained. Recommended resilience4j, recent sentinel Ali also performed strongly.
For the call chain, because of the rise of OpenTracing, with many new faces. Recommended jaeger or skywalking. The integrated spring cloud sleuth + zipkin function weaker, even better than conventional invasive cat.
Configuration management is a multi-center environment profiles weapon, especially in the case of configuration updates you do not want to restart the server. Currently, open source to do the best to the number of apollo, and provides support for the spring boot. disconf use more widely. Relatively speaking, spring cloud config feature is confined to some, rarely used.
Gateway, the use of most is nginx, on nginx, we have based lua script openrestry. Openresty the use of very complicated, so that with a higher package level kong gateway.
For spring cloud is, zuul series recommended zuul2, zuul1 is multithreaded blocked, flawed. spring-cloud-gateway is a natural spring cloud, but there is not very widely used.
Seven, distributed tools
We all know that zookeeper distributed systems can be used in many scenarios, there are similar etcd and its consul-based raft protocol.
Because they are able to ensure high consistency, so as a coordination tool is better. Use focus: distribution center, distributed lock, naming services, distributed coordination, master election and other places.
For distributed transactions is concerned, there is Ali fescar tool support. However, as a special non-essential, or use 柔性事务in search 最终一致性, better.
Eight, the monitoring system
Monitoring a wide range of system components, at present, the most popular is probably the above four categories.
zabbix In a small number of cases the host, is a very good choice.
prometheus ferocious, much of world domination posture. It can also use a more beautiful grafana front-end display.
influxdata of influxdb and telegraf components are relatively easy to use, the main function is very wide.
Using es stored elkb tool chain, is also a good choice. I know a lot of companies are using.
Nine, scheduling
We may have used cron expression. This expression, originally from linux's crontab tool.
quartz is relatively old in java program scheduling, distributed scheduling the use of the database lock mode, need to develop their own management interface.
elastic-job-cloud applications more widely, but the system operation and maintenance complex, high learning costs. Relatively speaking, xxl-job is even more lightweight few. Chinese people developed system, the background are more beautiful.
Ten, entry tool
In order to unify the user's access to the intersection, the entrance will generally use some tool support.
Which, haproxy, lvs, keepalived, etc., is widely used.
Servers generally use good stability centos, and equipped ansible tool support, called a cool.
Eleven, OLT (A) P
Now the company, the amount of data is very large, the data warehouse is a must.
Search terms, solr elasticsearch and more popular, they are based on the lucene. solr more mature, better stability of some, but not as good as real-time search es.
Columnar storage, Hadoop-based hbase, is the most widely; LSM of leveldb written based on superior performance, but mainly as an embedded engine uses more.
tidb domestic upstart is compatible with mysql agreement, the company dba output outward through training, the future can be expected.
Timing databases, opentsdb use in large surveillance systems more. druid and Kudu, multi-dimensional data in real time in the polymerization process aspects, superior.
cassandra fire just for a while, although there is deprecated facebook news, but ecology has been formed, the perennial occupy 15 before the database engine.
Twelve, CI / CD
To support continuous integration and virtualization, in addition to familiar docker, we have other tools.
jenkins is packaged and released the first choice, after all, so many years, it has been a big brother. Of course, writing Idea of that company, but also a tool called TeamCity, the user interface is very smooth.
solor have to say that is an artifact, after using it, a small partners code gains across the board, I almost spit the stars are drowned.
For internal companies, the general use of gitlab set up git server. In fact, inside it gitlab CI, it is also very easy to use.
XIII, troubleshooting
java memory overflow problems often occur. After using jmap export the stack, I generally use the mat in-depth analysis.
If the online real-time analysis, there are two tools arthas and perf.
Of course, there are large quantities of linux tool support. The following example:
"On Linux, the most common group of command parsing (10 years Selection)"
Fourth, the local tool
Jar package for local use and tools, it is more to go. Here are just a few to mention the most common.
Database connection pool, the domestic use of the most druid. Currently, there are known as the fastest hikari database connection pool, as well as old-fashioned dbcp and c3p0.
json, domestic use fastjson most vulnerabilities emerge every few days; the use of foreign jackson more. They are similar to the api, jackson characteristics more, but fastjson easier to use.
Toolkit, although there are all kinds of commons package, guava choice.
End
Today is August 13, 2019. Typhoon Lekima raging just finished.
This article, I will organize once per year. Some new faces, some are my personal t out. Architecture selection, in addition to your own particular technology more familiar and more comfortable to use. More of a need for a lot of research, compare, until the master.
Rapid technological change, new wine in old bottles, a laundry list of nouns, programmers very hard. Only the basic principles behind it, Road to Jane thought, enduring.
About the Author: little sister the taste (xjjdog), does not allow the public a number of detours programmer. Focus infrastructure and Linux. Decade architecture, day ten billion flow, and you explore the world of high concurrency, give you a different taste. My personal micro-channel xjjdog0, welcome to add friends, further exchanges.
Recent Popular Articles
"Must-see! java backend, Sword Zhu Xian "
back-end technology index, pertinent hot
"On Linux, the most common group of command parsing (10 years Selection)" CSDN release of the first day, 1k praise. Like rate 1/8 point.
"This talk is not clear if the Spring Cloud core components, then I compiled this white story" to explain the core components of a story, pack your satisfaction
"On Linux production environment, the most commonly used set of" Sed "tricks" the most common Sed series of articles, easy to understand. Vim articles more accessible.