NIO: One thread per request, but the connection request sent by the client will be registered on the multiplexer, and the multiplexer will only start a thread for processing when there is an I/O request connected to the multiplexer.
Features of NIO: event-driven model, single-threaded multitasking, non-blocking I/O, I/O read and write is no longer blocked, block-based transmission is more efficient than stream-based transmission, more advanced IO function zero-copy, IO multiplexing greatly improves the scalability and practicality of Java network applications. Based on the Reactor threading model.
NIO is buffer-oriented, while traditional IO is stream-oriented.
2. The three core parts of
NIO NIO mainly include: Selector and Channel abstraction in the java.nio.channels package, and Buffer abstraction in the java.nio package.
a) Stream (stream) in Channel and IO is almost the same level. It's just that Stream is one-way, such as InputStream, OutputStream. Channel is two-way, which can be used for both read and write operations. Channel is the source or destination of data, used to provide data to the buffer or read buffer data.
b) Buffer is a continuous memory block, which is the transfer place for NIO data reading and writing.
c) Selector, the core class of asynchronous IO, it can detect events on one or more channels (channel) and distribute the events. Using a select thread can listen to events on multiple channels and trigger corresponding responses based on event-driven triggers. There is no need to allocate a thread for each channel.
3. NIO server establishment process
4. Selector.open(): Open a Selector; ServerSocketChannel.open(): Create a server-side Channel; bind(): Bind to a port. And configure non-blocking mode; register(): register Channel and concerned events to Selector; select() polling to get ready events
5. Reactor threading model
a) single-threaded model
b) multi-threaded model
c) master-slave Multithreading model
6. Netty's threading model
Netty receives and processes user requests based on multiplexers through the Reactor model, and internally implements two thread pools, the boss thread pool and the work thread pool. The Boss thread pool is responsible for processing the request accept event, encapsulates the corresponding socket into a NioSocketChannel, and hands it over to the work thread pool for processing; the work thread pool is responsible for the requested read and write events, and the event processing process is handled by a specific handler ( Refer to Reactor's master-slave multithreading model)
7. Features of Netty
a) High concurrency - NIO's Reactor model (based on an event-driven design pattern)
b) Fast transmission - data is read directly from IO to memory. Netty implements zero-copy through ByteBuf
c) Encapsulated well - improves ease of use through the encapsulation of channel, Bytebuf, and code
8. Netty's zero-copy implementation principle
Netty uses DIRECT BUFFERS to receive and send ByteBuffer, and uses off-heap direct memory for Socket reading Write without making a second copy of the byte buffer. If there is one more memory copy in the heap memory, the JVM will copy the heap memory Buffer to the direct memory, and then write it into the Socket.
9. Dubbo 's architecture
Provider: Service provider that exposes services
Consumer: Service consumer that calls
remote
services . Dubbo's communication protocol a) Dubbo - the default, simple long connection, NIO asynchronous communication protocol. Suitable for scenarios with small amount of data and concurrency b) RMI, Hessian, Http, websocket 11. What serialization framework is used by Dubbo by default , Hessian serialization is used by default, and there are also serialization by Duddo, FastJson, and Java. 12. What is the principle for service providers to implement fail-kick? Nodes in zookeeper have a life cycle. The specific life cycle depends on the type of node. Nodes are mainly divided into persistent nodes, persistent sequential nodes, temporary nodes, and temporary sequential nodes. The so-called persistent node means that after the node is created, it will always exist until there is a delete operation to actively clear the node, that is to say, it will not disappear due to the invalidation of the client session that created the node. The life cycle of the temporary node is bound to the client session, that is, if the client session fails, the node will be automatically cleared. In distributed systems, we often need to know whether a machine is available. In ZK, we let all machines register a temporary node. To judge whether a machine is available, we only need to judge whether this node exists in ZK. There is no need to directly connect to the machine that needs to be checked, which reduces the complexity of the system.
13. Zookeeper in dubbo is the registry center. If the registry cluster is down, can the publisher and subscriber still communicate? Yes
, when the consumer starts, the consumer will pull the address of the registered producer from zk Data such as interfaces are cached locally. Each time it is called, it is called according to the locally stored address.
14. The test and production environments share a set of zookeepers, how to ensure that consumption does not conflict
a) Implement IP whitelist function through custom Filter
b) Implement through service grouping function
c) Implement through version control
15. How to do Dubbo cluster fault tolerance?
It is recommended to use Failover to automatically switch to other servers for read operations. By default, it will retry two other servers. It is recommended to use Failfast to fail quickly for write operations. If a call fails, an error will be reported immediately.
16. How does zookeeper ensure the sequential consistency of transactions?
Zookeeper uses incremental transaction ID to identify, all proposals are added with zxid when they are proposed, zxid is actually a 64-bit number, and the upper 32 bits are The epoch is used to identify whether the leader has changed. If a new leader is generated, the epoch will increase automatically, and the lower 32 bits are used to increment the count. When a new proposal is generated, it will first issue a transaction execution request to other servers according to the two-stage process of the database. If more than half of the machines can execute and succeed, then the execution will begin.
17. The leader election
of The leader crashes or the leader loses most of the followers. At this time, zk enters the recovery mode. In the recovery mode, a new leader needs to be re-elected to restore all servers to a correct state. Zk's election algorithm uses the ZAB protocol:
a) The election thread first initiates a query to all servers
b) After receiving the replies from all servers, it calculates the server with the largest zxid, and sets the relevant information of this server as the server to vote for the next time
c) The thread sets the current zxid with the largest zxid The server is set to the leader recommended by the current server. If the winning server gets n/2 + 1 server votes at this time, set the currently recommended leader as the winning server.
18. The client's polling mechanism for zookeeper serverList is
initialized by Zookeeper During the process, the server will be saved in a list, and then randomly scattered, and then one by one from the 0th position
. When will the temporary nodes created by 19. be deleted, is it deleted as soon as the connection is broken? What is the delay?
After the connection is disconnected, ZK will not remove the temporary data immediately. Only when the session expires, will the temporary data established by the session be removed.
20. How do servers in a ZooKeeper cluster communicate?
The Leader server will establish a TCP connection with each Follower/Observer server, and create an entity called LearnerHandler for each F/O. LearnerHandler is mainly responsible for the network communication between Leader and F/O, including data synchronization, request forwarding and Proposal voting. The Leader server saves all F/O LearnerHandlers.
21. What should I do if the call timeout com.alibaba.dubbo.remoting.TimeoutException occurs?
Usually, the business processing is too slow and can be executed on the service provider: jstack PID > jstack.log Analyze which method call the thread is stuck on. Here is the reason for the slowness. If you cannot tune performance, set the timeout to a larger value.
22. What should I do if java.util.concurrent.RejectedExecutionException or Thread pool exhausted occurs?
RejectedExecutionException indicates that the thread pool has reached the maximum value, and there is no idle connection, and some tasks have been refused to execute. The reason may be that the connection pool is not enough.
Thread pool exhausted is usually when min and max are not the same size, indicating that the currently created connections are used up, an expansion is performed, and new threads are created, but it does not affect the operation.
Adjust dubbo.properites:
// Set to the same size to reduce thread pool shrinkage overhead
dubbo.service.min.thread.pool.size=200
dubbo.service.max.thread.pool.size=200
23. IOC in Spring realization principle.
The implementation principle of IoC in Spring is the factory mode plus reflection mechanism. The most basic technology in IOC is "Reflection". Generally speaking, it is to dynamically generate objects according to the given class name. This programming method allows the object to be determined when it is generated. The objects to be produced in Spring are defined in the configuration file, the purpose is to improve flexibility and maintainability.
Spring IOC is responsible for creating objects, managing objects (through dependency injection (DI), assembling objects, configuring objects, and managing the entire lifecycle of those objects.
24. What problem does Spring AOP solve? How?
The main purpose of aspect-oriented programming is to separate the business code from the general code, and separate the general code (transaction, log), thereby reducing the complexity of the application.
Spring aop uses a dynamic proxy mechanism to generate a proxy object for the target object during code execution, and weaves the cross-cutting object into this proxy object, and the system uses this proxy object instead of the real target object.
JDK's dynamic proxy -
Cglib's dynamic proxy through reflection and dynamic compilation - dynamic proxy through integration
25. What are the ways of spring's dependency injection
a) setter injection <property name="userDao" ref=" userDao"/>
b) Constructor injection <constructor-arg ref="userDao"/>
26. The process of SpringMVC processing requests.
a) DispatcherServlet receives the request and performs mapping path matching
b) According to the request to HandlerMapping to get the corresponding processor
c) Process the request and return the ModelAndView model to the dispatcher
d) Dispatcher resolves the logical view name to the real view name with the help of the view resolver Parsing work
e) After getting the real object view, dispatcher uses this view to render
modelAndview 27. In Mybatis, what is the difference between #{} and ${}?
#{} is precompiled processing, ${} is string replacement.
When Mybatis processes #{}, it will replace #{} in sql with the ? sign;
when Mybatis processes ${}, it replaces ${} with the value of the variable.
Using #{} can effectively prevent SQL injection and improve system security.
28. What should I do when the attribute name in the entity class is different from the field name in the table?
a) By defining the alias of the field name in the sql statement of the query, make the alias of the field name consistent with the attribute name of the entity class
b) By <resultMap> to map the one-to-one correspondence between field names and entity class attribute names
29. How does Mybatis perform paging? What is the principle of the paging plugin?
Mybatis uses the RowBounds object for paging, which is memory paging for the ResultSet result set instead of physical paging. You can directly write parameters with physical paging in sql to complete the physical paging function, or you can use the paging plug-in to complete the physical paging . The basic principle of the paging plug-in is to use the plug-in interface provided by Mybatis to implement a custom plug-in, intercept the sql to be executed in the plug-in interception method, then rewrite the sql, and add the corresponding physical paging statement and physical paging parameters according to the dialect dialect.
30. How to pass multiple parameters in mapper?
a) Use placeholder #{0}
b) Use @param annotation
31. Batch insert in MyBatis
a) Set ExecutorType.BATCH, and then use dynamic sql foreach to splicing sql to achieve batch insert.
b) Pit:
i. The content in <![CDATA[ ]]> will be ignored by the parser, so do not use <![CDATA[ ]]> when using dynamic sql
ii. When the amount of data inserted in a single time is too large, commit insert batches in batches within the same transaction scope
iii. Use insert table values () () instead of multiple insert splicing
32. Kafka's architecture
33. Several Kafka Basic concepts
a) Topic: Specifically refers to the different classifications of feeds of messages processed by Kafka.
b) Partition: The physical grouping of topics. A topic can be divided into multiple partitions, and each partition is an ordered queue. Each message in the partition is assigned an ordered id (offset).
c) Message: Message is the basic unit of communication. Each producer can publish some messages to a topic.
d) Producers: message and data producers, the process of publishing messages to a topic in Kafka is called producers.
e) Consumers: message and data consumers, the process of subscribing to topics and processing the messages published by them is called consumers.
f) Broker: Cache broker, one or more servers in the Kafka cluster are collectively referred to as brokers.
34. The process of Kafka message sending The
Producer publishes the message to the partition of the specified topic according to the specified partition method (round-robin, hash, etc.).
After the kafka cluster receives the message sent by the Producer, it persists it to the hard disk, and Keep messages for a specified length of time (configurable), regardless of whether messages are consumed.
The consumer pulls data from the kafka cluster and controls the offset that gets the message
35. Kayka's design points
a) Use the cache of the Linux file system directly to cache data efficiently.
b) Using linux Zero-Copy to improve the sending performance.
c) The cost of accessing data on disk is O(1).
d) Explicit distribution, that is, there will be multiple producers, brokers and consumers, all of which are distributed. There is no load balancing mechanism between producers and brokers. Zookeeper is used for load balancing between brokers and consumers.
36. Can we use Kafka without Zookeeper? It
is not possible to bypass Zookeeper and contact the Kafka broker directly. Once Zookeeper stops working, it cannot serve client requests.
a) Zookeeper is mainly used for communication between different nodes in the cluster
b) In Kafka, it is used to commit offsets, so if a node fails in any case, it can move from the previously committed offset c) In addition to
this, it performs other activities such as: leader detection, distributed synchronization, configuration management, identifying when new nodes leave or connect, clustering, node real-time status, and more.
37. How can you get accurate information from Kafka during data production?
To get accurate Kafka messages, you have to follow two things: avoid duplication during data consumption and avoid duplication during data production.
Here are two ways to get a semantics exactly when the data is generated:
a) use a separate writer per partition, whenever you see a network error, check the last message in that partition to see if your last write was successful
b) include a primary key (UUID) in the message or other), and anti-duplication in the user
38. What is an ISR?
ISR is a set of replica synchronization queues that are fully synchronized with the leaders, that is to say, the ISR contains all submitted messages. The ISR should always contain all replicas until a real failure occurs. If a replica leaves the leader, it will be removed from the ISR.
39. Kafka's replication principle
In order to improve the reliability of messages, each topic partition of Kafka has N replicas (replicas), where N (greater than or equal to 1) is the number of topic replication factors (replica fators). Kafka implements automatic failover through a multi-copy mechanism, and when a broker in the Kafka cluster fails, the service is still available.
40. Kafka data reliability guarantee
When the producer sends data to the leader, the level of data reliability can be set through the request.required.acks parameter:
a) 1 (default): This means that the leader of the producer in the ISR has successfully received the data. The received data is confirmed and the next message is sent. If the leader goes down, data will be lost.
b) 0: This means that the producer does not need to wait for an acknowledgment from the broker to continue sending the next batch of messages. In this case, the data transmission efficiency is the highest, but the data reliability is indeed the lowest.
c) -1: The producer needs to wait for all followers in the ISR to confirm that the data has been received before a transmission is completed, and the reliability is the highest.