Ali's 10 god-level projects open source

foreword

Recently, taking advantage of the National Day holiday, I specially sorted out 10 god-level projects that Alibaba has open sourced.

Most of these open source projects have been used in my actual work, or have been used by colleagues. It's really good and valuable, and now I recommend it to everyone.

1. Druid

Druid claims to be the best database connection pool in the Java language, which can provide powerful monitoring and scaling functions. The monitoring background is shown in the figure below:

The main advantages of Druid are as follows:

It monitors database access performance.
It provides WallFilter, which is based on SQL semantic analysis to prevent SQL injection attacks.
It provides various means of monitoring connection leaks.
It provides the function of database password encryption.
It can print SQL execution log.

github address: https://github.com/alibaba/druid

maven central warehouse: https://mvnrepository.com/artifact/com.alibaba/druid

Configure maven dependencies:

<dependency>
  <groupId>com.alibaba</groupId>
  <artifactId>druid</artifactId>
  <version>${druid-version}</version>
</dependency>

2. fastjson

fastjsonAlibaba's open-source JSON parsing library, which can parse JSON-formatted strings, supports serialization of Java Beans into JSON strings, and deserialization from JSON strings to JavaBean.

The main advantages of fastjson are as follows:

Fast, fastjson is faster than other JSON libraries. Since the release of fastjson version 1.1.x in 2011, its performance has never been surpassed by other Java-implemented JSON libraries.
Widely used, fastjson is used on a large scale in Alibaba and deployed on tens of thousands of servers, and fastjson is widely accepted in the industry. In 2012, it was selected as one of the most popular domestic open source software by Open Source China.
Easy to use, the API of fastjson is very concise.

//序列化
String text = JSON.toJSONString(obj); 
//反序列化
VO vo = JSON.parseObject("{...}", VO.class);

It has complete functions, supports generics, supports stream processing of large texts, supports enumeration, and supports serialization and deserialization extensions.

github address: https://github.com/alibaba/fastjson

maven central warehouse: https://mvnrepository.com/artifact/com.alibaba/fastjson

Configure maven dependencies:

<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>fastjson</artifactId>
    <version>1.2.76</version>
</dependency>

3. Dubbo

Apache Dubbo is a microservice development framework that provides two key capabilities of RPC communication and microservice governance. This means that the microservices developed using Dubbo will have mutual remote discovery and communication capabilities. At the same time, using the rich service governance capabilities provided by Dubbo, service governance demands such as service discovery, load balancing, and traffic scheduling can be realized.

At the same time, Dubbo is highly scalable, and users can customize their own implementation at almost any function point to change the default behavior of the framework to meet their own business needs. It is currently handed over to Apache for management and maintenance.

The architecture diagram is as follows:

The main advantages of Dubbo are as follows:

RPC based on transparent interface
Intelligent load balancing
Automatic service registration and discovery
high scalability
Runtime Traffic Routing
Visual Service Governance
Cloud native friendly

github address: https://github.com/apache/dubbo

maven central warehouse: https://mvnrepository.com/artifact/com.alibaba/dubbo/

Configure maven dependencies:

<properties>
    <dubbo.version>3.0.3</dubbo.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.apache.dubbo</groupId>
        <artifactId>dubbo</artifactId>
        <version>${dubbo.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.dubbo</groupId>
        <artifactId>dubbo-dependencies-zookeeper</artifactId>
        <version>${dubbo.version}</version>
        <type>pom</type>
    </dependency>
</dependencies>

4. Rocketmq

Apache RocketMQ is a distributed messaging and streaming platform with low latency, high performance and reliability, trillion-level capacity, and flexible scalability.

It provides various functions:

Messaging patterns, including publish/subscribe, request/reply, and streaming
Financial Grade Transaction News
Built-in fault tolerance and high availability configuration options based on DLedger
Multiple cross-language clients, such as Java, C/C++, Python, Go
Pluggable transport protocols such as TCP, SSL, AIO
Built-in message tracking capability, also supports opentracing
Versatile Big Data and Streaming Media Ecosystem Integration
Message traceback by time or offset
Reliable FIFO and strictly ordered message delivery in the same queue
Efficient pull-push consumption model
Accumulation capacity of millions of messages in a single queue
Various messaging protocols such as JMS and OpenMessaging
Flexible distributed scale-out deployment architecture
Lightning-fast bulk message exchange system
Various message filtering mechanisms, such as SQL and Tag
Docker images for isolated testing and cloud isolated clusters
Feature-rich admin dashboard for configuration, metrics, and monitoring
Authentication and Authorization
Free open source connectors for sources and sinks

rocketmq background management interface:

github address: https://github.com/apache/rocketmq

Maven Central Warehouse: https://mvnrepository.com/artifact/org.apache.rocketmq/rocketmq-spring-boot-starter

Rocketmq includes: server and client. In our project, we only need to focus on the code of the client.

Configure maven dependencies:

<dependency>
    <groupId>org.apache.rocketmq</groupId>
    <artifactId>rocketmq-client</artifactId>
    <version>4.3.0</version>
</dependency>

4. Arthas

Arthas is Alibaba's open source Java diagnostic tool, which is very popular among developers.

When you encounter the following similar problems and feel helpless, Arthas can help you solve them:

Which jar package is this class loaded from? Why are various types of related Exception reported?
Why is the code I changed not executed? Could it be that I didn't commit? Branching wrong?
If you encounter a problem and cannot debug online, can you only republish it by adding a log?
I encountered a problem with the data processing of a certain user online, but it also cannot be debugged online, and cannot be reproduced offline!
Is there a global view into the health of the system?
Is there any way to monitor the real-time running status of the JVM?
How to quickly locate application hotspots and generate flame graphs?
How to find an instance of a class directly from within the JVM?

Time-consuming analysis code: Arthas supports JDK 6+ and can run on various operating systems, such as Linux/Mac/Windows, adopts command line interaction mode, and provides rich Tab auto-completion functions to further facilitate problem location and diagnosis.

github address: https://alibaba.github.io/arthas/

Maven Central Warehouse: https://mvnrepository.com/artifact/com.taobao.arthas/arthas-spring-boot-starter

Execute the following command on the target machine to start arthas:

curl -O https://arthas.aliyun.com/arthas-boot.jar
java -jar arthas-boot.jar

5. Nacos

Nacos is an easy-to-use platform designed for dynamic service discovery and configuration and service management. It helps you easily build cloud-native applications and microservices platforms.

Services are first-class citizens of Nacos. Nacos supports almost all types of services, such as Dubbo/gRPC services, Spring Cloud RESTFul services or Kubernetes services.

Nacos provides four major functions.

服务发现和服务健康检查. Nacos makes it simple for services to register themselves and discover other services via DNS or HTTP interfaces. Nacos also provides real-time health checks of services to prevent requests from being sent to unhealthy hosts or service instances.
动态配置管理. Dynamic configuration services allow you to centrally and dynamically manage the configuration of all services across all environments. Nacos does not need to redeploy applications and services when updating configurations, which makes configuration changes more efficient and agile.
动态 DNS 服务. Nacos supports weighted routing, making it easier for you to implement mid-level load balancing, flexible routing policies, traffic control, and simple DNS resolution services in the production environment in the data center. It helps you easily implement DNS-based service discovery and prevents applications from being coupled to vendor-specific service discovery APIs.
服务和元数据管理. Nacos provides an easy-to-use service dashboard to help you manage your service metadata, configuration, kubernetes DNS, service health and metrics statistics.

Nacos map:

Nacos ecological map:

github address: https://github.com/alibaba/nacos

Maven Central Warehouse: https://mvnrepository.com/artifact/com.alibaba.cloud/spring-cloud-starter-alibaba-nacos-discovery

6. easyexcel

The well-known frameworks for Java parsing and generating Excel include Apache poi and jxl. But they all have a serious problem that consumes a lot of memory. poi has a SAX mode API that can solve some memory overflow problems to a certain extent, but POI still has some defects. It is done in memory, and the memory consumption is still very large.

easyexcel rewrites poi's analysis of Excel version 07. A 3M excel with POI sax analysis still needs about 100M of memory. Using easyexcel can reduce it to a few M, and no matter how big the excel is, there will be no memory overflow; version 03 depends on POI's sax mode encapsulates the model conversion on the upper layer, making it easier and more convenient for users.

Read 75M (46W rows and 25 columns) Excel within 1 minute from 64M memory

github address: https://github.com/alibaba/easyexcel

maven central warehouse: https://mvnrepository.com/artifact/com.alibaba/easyexcel

Configure maven dependencies:

<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>easyexcel</artifactId>
    <version>2.2.6</version>
</dependency>

7. Sentinel

As distributed systems become more popular, reliability between services becomes more important than ever.

Sentinel takes "traffic" as the entry point, and works in multiple fields such as traffic control, traffic shaping, circuit breaking, and system adaptive protection to ensure the reliability and elasticity of microservices.

Sentinel has the following characteristics:

丰富的适用场景: Sentinel has been widely used in Alibaba, covering almost all the core scenarios of the Double 11 (11.11) shopping festival in the past 10 years, such as the need to limit the "second kill" of burst traffic to meet the system capacity, message peak shaving and valley filling, downstream Unreliable service disconnection, cluster flow control, etc.
实时监控: Sentinel also provides real-time monitoring capabilities. You can view the runtime information of a single machine in real time, as well as the summary of the runtime information of a cluster with less than 500 nodes.
广泛的开源生态系统: Sentinel provides out-of-the-box integrations with popular frameworks and libraries such as Spring Cloud, Dubbo, and gRPC. You can easily use Sentinel by simply adding the adapter dependency to your service.
多语言支持: Sentinel provides native support for Java, Go, and C++.
丰富的SPI扩展: Sentinel provides an easy-to-use SPI extension interface, which allows you to quickly customize logic, such as custom rule management, data source adaptation, etc.

Functional Overview:

Ecosystem Landscape:

github address: https://github.com/alibaba/Sentinel

Maven Central Warehouse: https://mvnrepository.com/artifact/com.alibaba.csp/sentinel-core

Configure maven dependencies:

<dependency>
    <groupId>com.alibaba.csp</groupId>
    <artifactId>sentinel-core</artifactId>
    <version>1.8.2</version>
</dependency>

8. otter

Alibaba B2B company, because of the characteristics of the business, the sellers are mainly concentrated in the country, and the buyers are mainly concentrated in foreign countries, so the demand for remote computer rooms in Hangzhou and the United States has been derived. At the same time, in order to improve the user experience, the structure of the entire computer room is double A. Both can be written, thus a product such as otter was born.

The first version of otter can be traced back to 2004~2005. The external open source version is the 4th version. The development time has continued from July 2011 to the present. At present, the synchronization requirements of local/remote computer rooms in Alibaba B2B are basically complete. On otte4.

Current synchronization scale:

Synchronized data volume 600 million
File synchronization 1.5TB (2000w pictures)
Involves synchronization between 200+ database instances
Cluster size of 80+ machines

What can otter solve?

异构库同步. mysql -> mysql/oracle. (currently the open source version only supports mysql increment, the target library can be mysql or oracle, depending on the function of canal)
单机房同步 (RTT < 1ms between databases)

a. Database version upgrade

b. Data table migration

c. Asynchronous secondary index
异地机房同步 (For example, Alibaba International Station is the database synchronization between Hangzhou and the US computer room, RTT > 200ms, the highlight)

a. Computer room disaster recovery
双向同步

a. Avoid the loopback algorithm (a general solution that supports most relational databases)

b. Data Consistency Algorithm (Guaranteed final consistency of data in double A computer room mode, highlights)
文件同步Site mirroring (copies associated pictures while copying data, such as copying product data and copying product pictures at the same time).

Working principle diagram:

Schematic diagram of replication in a single computer room:

Schematic diagram of remote computer room replication:

github address: https://github.com/alibaba/otter

maven central warehouse: https://mvnrepository.com/artifact/com.alibaba.otter/canal.client

9. P3C

The P3C plugin presents the Alibaba Java Coding Guide, which integrates the best programming practices of the Alibaba Group technical team over the years. Since we encourage reuse and better understanding of each other's programs, large Java programming teams place stringent requirements on code quality across projects.

Alibaba has seen a lot of programming problems in the past. For example, flawed database table structures and index designs can lead to software architectural flaws and performance risks. Another example is a messy code structure that is difficult to maintain. Also, unauthenticated vulnerable code is vulnerable to hackers. In order to solve these problems, we wrote this document for Alibaba's Java developers.

For more information, please refer to the Alibaba Java Coding Guide:

Chinese version: Alibaba Java Development Manual
English version: Alibaba Java Coding Guidelines

The project consists of 3 parts:

PMD implementation
IntelliJ IDEA plugin
Eclipse plugin

Forty-nine rules are implemented based on PMD. For more detailed information, please refer to the P3C-PMD document. The four rules implemented in the IDE plugins (IDEA and Eclipse) are as follows:

[Mandatory] Prohibit use of deprecated classes or methods. Note: For example, decode(String source, String encode) should be used instead of the deprecated method decode(String encodeStr). Once an interface is deprecated, the interface provider is obliged to provide a new interface. In the meantime, it's the client programmer's duty to check what its new implementation is.
[Mandatory] Overridden methods from interfaces or abstract classes must be marked with the @Override annotation. Counterexample: For getObject() and get0bject(), the first is the letter "O" and the second is the number "0". In order to accurately determine whether the coverage is successful, an @Override annotation is required. At the same time, once the method signature in the abstract class changes, the implementation class will immediately report a compile-time error.
[Mandatory] A static field or method should be referenced directly by its class name rather than its corresponding object name.
[Mandatory] The usage of hashCode and equals should follow:

If equals is overridden, hashCode is overridden.
These two methods must be overridden for Set because they are used to ensure that duplicate objects are not inserted into the Set.
These two methods must be overridden if a custom object is used as the key of the Map. Note: String can be used as key of Map, because these two methods have been overridden.

The effect of using the p3c plug-in:

The latest version of the Alibaba Java Development Manual download address: https://github.com/alibaba/p3c/blob/master/Java Development Manual (Songshan Edition).pdf

github address: https://github.com/alibaba/p3c/tree/master/idea-plugin

10. Spring Cloud Alibaba

Spring Cloud Alibaba provides a one-stop solution for distributed application development. It contains all the components needed to develop distributed applications, allowing you to easily develop applications using Spring Cloud.

Using Spring Cloud Alibaba, you only need to add some annotations and a small amount of configuration to connect Spring Cloud applications to Alibaba's distributed solutions and build distributed application systems through Alibaba middleware.

The main functions are as follows:

流量控制和服务降级: By default, the flow control of HTTP service is supported. You can also use annotations to customize traffic control and service degradation rules. Rules can be changed dynamically.
服务注册和发现: The service can be registered, and the client can use the bean managed by Spring to automatically integrate the Ribbon to discover the instance.
分布式配置: Supports externalized configuration in distributed systems, and automatically refreshes when the configuration changes.
事件驱动: Supports building highly scalable event-driven microservices connected to shared messaging systems.
分布式事务: Supports high-performance, easy-to-use distributed transaction solutions.
阿里云对象存储: Massive, safe, low-cost, highly reliable cloud storage service. Store and access any type of data in any application, anytime, anywhere.
阿里云SchedulerX: Accurate, highly reliable, and highly available scheduled job scheduling service with response time in seconds.
阿里云短信: SMS service covering the world, Ali SMS provides convenient, efficient and intelligent communication capabilities to help companies quickly contact customers.

It mainly includes the following components:

Sentinel: Take the flow as the entry point to protect the stability of the service from multiple dimensions such as flow control, circuit breaker degradation, and system load protection.
Nacos: A dynamic service discovery, configuration management, and service management platform that makes it easier to build cloud-native applications.
RocketMQ: An open source distributed messaging system, based on highly available distributed cluster technology, provides low-latency, highly reliable message publishing and subscription services.
Dubbo: Apache Dubbo™ is a high-performance Java RPC framework.
Seata: Alibaba open source product, an easy-to-use high-performance microservice distributed transaction solution.
Alibaba Cloud OSS: Alibaba Cloud Object Storage Service (OSS for short) is a massive, secure, low-cost, and highly reliable cloud storage service provided by Alibaba Cloud. You can store and access any type of data in any application, anytime, anywhere.
Alibaba Cloud SchedulerX: A distributed task scheduling product developed by the Alibaba middleware team, which provides second-level, accurate, highly reliable, and highly available scheduled (based on Cron expression) task scheduling services.
Alibaba Cloud SMS: SMS service covering the world, friendly, efficient and intelligent interconnected communication capabilities, help enterprises quickly build customer access channels.

github address: https://github.com/alibaba/spring-cloud-alibaba

Configure maven dependencies:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-alibaba-dependencies</artifactId>
            <version>2.2.6.RELEASE</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

One last word (please pay attention, don't prostitute me for nothing)

If this article is helpful or inspiring to you, please scan and send the QR code to pay attention. Your support is the biggest motivation for me to keep writing.

Ask for one-click triple link: like, forward, and watch.

Pay attention to the official account: [Su San said technology], reply in the official account: interviews, code artifacts, development manuals, time management have awesome fan benefits, and reply: join the group, you can communicate and learn from many seniors of BAT manufacturers .