Series thirty-one JBoss: JBoss Data Grid (Infinispan) cache mode

2019 Unicorn Enterprises heavily recruiting engineers Python standard >>>

Red Hat JBoss Data Grid is a middleware product ( http://www.redhat.com/products/jbossenterprisemiddleware/data-grid/ ), JBoss Data Grid is based on the open source community products Infinispan ( http://infinispan.org/ ). Enterprise Edition JBoss Data Grid with the community version of Infinispan exactly as it was, so this section were based on Infinispan.

Outline

Infinispan offers two modes:

Local mode - non-clustered mode Infinispan provided, in local mode, Infinispan can be regarded as a single node, the data cached in memory. Compared with the cluster model, because we will not consider scalability, fault tolerance, clustering, etc., which makes Infinispan higher performance, more efficient.
Trunked mode - a plurality of trunked mode Infinispan server (Infinispan instance) to form a cluster, the cluster all points constituting a data grid. Mesh state of a node need to be copied to a subset of all other nodes, or node. This subset of nodes to determine how much the Infinispan data grid fault tolerance and scalability, a subset of nodes too large hamper Infinispan grid scalability. Between the nodes in the cluster model is to replicate state by JGroups, we will discuss in detail later. Depending on the cluster model replication strategy can be divided into a distributed mode, copy mode and invalid mode

Local Mode

Although Infinispan main features embodied in the cluster model, Infinispan local mode provides the need for practical and efficient characteristics, Cache interface inherits the JDK Infinispan of ConcurrentMap, and ordinary compared Map, Infinispan provides the following features:

Cache entry may be preserved by the cache memory to store a third party server (relational database, No-SQL, clouds), and by synchronous, asynchronous mode operation of the cache memory storage server third-party
Evicted cache entries prevent Java virtual machine memory overflow
To ensure long expired unused entries in the cache entry is removed
JTA and XA support enables the cache operation guaranteed transactional
MVCC is based on concurrent read read cache to ensure that processing entity may be told non-blocking

Infinispan server cache container

Equivalent to a cache server Infinispan container, the container may have a plurality of cache entries, the cache can be local, it may also be clustered. Infinispan server protocol (HotRod, Rest, Memcached) for a client application to connect to the server, the container cache operation. Infinispan server profile cache cache-container attribute represents the container, this attribute comprises a plurality of cache configurations, including a local cache and a cache cluster. Follows a simple configuration for the cache container, we excised from JDG / standalone / configuration / standalone.xml in.

<subsystem xmlns="urn:infinispan:server:core:5.3" default-cache-container="local">
	<cache-container name="local" default-cache="default" start="EAGER">
	</cache-container>
</subsystem>

cache-container comprising the following properties:

name identifies the name of the cache container
determining a default-cache cache instance default cache container
determining whether the start-loading buffer container when the server starts Infinispan, the valid values may be, if the container is loaded cache server starts to EAGER Infinispan, LAZY is loaded when the request to EAGER or LAZY

Local configuration mode

In a configuration server Infinispan added to the local cache buffer container, as follows:

<cache-container name="local" default-cache="default" >
    <local-cache name="default" start="EAGER">
        <locking isolation="NONE" acquire-timeout="30000" concurrency-level="1000" striping="false" />
        <transaction mode="NONE" />
    </local-cache>
</cache-container>

As created by locally cached profile is similar to the default cache is created by Infinispan interfaces DefaultCacheManager. Further, the local caching and cache cluster may be located simultaneously in the cache-container, but if the cluster cache buffer vessel, the cache-container must be <transport /> properties. local-cache element includes the following attributes and sub-elements:

name identifies the name of the local cache
determining whether the default start buffer to start buffer vessel instance, the valid values may be EAGER or LAZY, starting cache instance if the buffer container is started to EAGER, LAZY said load only when requested
batching determine whether the batch operation of the local cache
indexing the determined local cache index attribute Valid values are NONE, LOCAL, and ALL

Further in the local cache, if we load the API by Infinispan cache, such as creating a cache transfer profile by DefaultCacheManager Infinispan construction method, the following is an example configuration file Infinispan:

<clustering mode="local" />

Note that, as specified above for the local cache cluster configuration mode, in this case, even if the definition <transport /> properties.

Cluster Mode

According to state replication strategy is different cluster model subdivided into: a distributed mode, copy mode and an inactive mode, and then we introduced in turn. Clustered mode we can further configuration status is synchronous or asynchronous transfer mode.

If the synchronization mode is used, the sender blocking wait for the receiver response, until a response is received the sender threads before continuing to run. Mode to transmit data asynchronously without blocking, without waiting for a response, which ensures the continuity of the cluster. For example, in a Web application cluster, using the sticky session load balancing manner (sticky-session), requests are to the same node, this case we use asynchronous replication HTTP session state. We can configure the asynchronous or synchronous mode attribute, an example configuration in synchronous replication mode:

<replicated-cache name="default" start="EAGER" mode="SYNC" batching="false" >
...
</replicated-cache>

Another acquire cluster grid we can provide the interface through Infinispan physical address:

AdvancedCache.getRpcManager().getT ransport().getPhysicalAddresses()

Three cluster model Infinispan

This discussion Infinispan at three cluster model: key distributed mode, copy mode and an inactive mode, these three models is designed Infinispan lies. Distributed model is in its ability to handle data of linear growth, providing the possibility for large data processing; copy mode is commonly used enterprise applications cluster model adopted; invalid mode provides a solution based on caching technology for enterprise applications in highly concurrent transaction program, then we in turn discuss these patterns.

Distributed mode

Infinispan distributed cache entry mode is stored on a subset of the grid nodes, each grid node rather than a cache entry is saved. For fault tolerance and generally backup data grid, the cache entry is stored in two or more nodes. And clustering pattern compared to the other, since the cache entries stored on a subset of grid nodes, so superior to other modes in scalability.

FIG, 4 is constructed as a server Infinispan data into a grid of four nodes, the cache entry is saved subset 2. We add data to the server 1 K / V, the hashing algorithm decides which of the data stored on a node, the data stored as shown in FIG. 2 on the server. When reading data, obtaining the same data to the corresponding node hash algorithm, we obtain Fig cache entry K / V 3 on the server.

FIG distributed mode 1 Infinispan

Infinispan distributed mode data backup, or data stored in a sub-grid reflects the characteristics Infinispan and scalability on large data sets. The following formula to calculate the capacity data grid Infinispan:

Capacity = (memory * number) / numOwners

Capacity - capacity data grid
memory - the memory size of a single node in the data grid
Number of data nodes in the mesh - number
numOwners - a subset of nodes in the mesh data (a data backup or a data storage nodes)

For example, we have 50 nodes, for each node on the memory size used is Infinispan 4GB, a subset of nodes in the data grid is 2, then the data capacity of the grid (4 * 50) / 2, that is 100GB . Distributed mode uses continuous hashing algorithm so that the read data transparency, then we have to explore the continuity of the hash algorithm.

Continuity Distributed hash algorithm mode

Infinispan distributed mode selected by the continuity of the hash algorithm from the grid node to hold out those cache entries, the total number of nodes is a subset of the data grid node, the hash algorithm configuration file to know the total number of subsets of nodes, while it maintains a cache this entry has been present on the node number. Related to the size of the number of nodes subset Infinispan data grid performance and fault tolerance, the number is too large subset of nodes that cache entry is saved on many nodes can affect the performance, but the number is too small subset of nodes in turn affect fault tolerance.

Process hashing algorithm used to access data cache further data from the grid, the access data is not transmitted to the all-nodes multicast request, the access process is not needed to maintain very variety of metadata, a PUT operation, only generating sub-node number (numOwners) remote reposition collector grid. A GET operation is only one node in a grid produced for remote dispatching, but also produce a grid node number of sub-sets (numOwners) transporting a background in real terms, and these are transported concurrently, the query results to be returned to the dispatching By. The other is, in distributed mode, regardless of the configuration of a synchronous or asynchronous transporting transporting, if not present in the local cache entry, the synchronization request is used to retrieve data from transporting other nodes.

Distributed Configuration Mode

Distributed pattern belongs Infinispan cluster cache arranged in distributed mode we need to add the server Infinispan follows buffer vessel, particularly edit JDG / standalone / configuration / clustered.xml, adding elements to the distributed-cache buffer container (cache-container ).

<cache-container name="local" default-cache="default" >
    <distributed-cache name="default" mode="{SYNC/ASYNC}" segments="${NUMBER}" start="EAGER">
        <locking isolation="NONE" acquire-timeout="30000" concurrency-level="1000" striping="false" />
        <transaction mode="NONE" />
    </distributed-cache>
</cache-container>

distributed-cache elements arranged distributed cache information by:

name defines the name of the cache, the cache is uniquely identified with
cache mode defines the copy mode, and the effective value of SYNC ASYNC
This property is selective segments, which hash until the end of the grid, the total number of its recommended value multiplied by a grid node 10, the default for this property is 80
determining whether the default start buffer to start buffer vessel instance, the valid values may be EAGER or LAZY, starting cache instance if the buffer container is started to EAGER, LAZY said load only when requested

In addition, we Infinispan by the API, the use of distributed mode cache loading CacheManager example, during loading Infinispan we need to specify the configuration file, the following example is to profile:

<clustering mode="dist">
    <sync replTimeout="${TIME}" />
    <stateTransfer chunkSize="${SIZE}" fetchInMemoryState="{true/false}" awaitInitialTransfer="{true/false}" timeout="${TIME}" />
    <transport clusterName="${NAME}" distributedSyncTimeout="${TIME}" strictPeerToPeer="{true/false}" transportClass="${CLASS}" />
</clustering>

mode attribute clustering element defines clusters for distributed cache mode.

replTimeout property sync element specifies the maximum timeout of waiting for fortune transported. If the timeout period has not yet received confirmation of remote dispatching, an exception is thrown.

stateTransfer element defines a cluster state transitions is how complete, the state transition occurs in a cluster node leaves or a new node is added. This element has the following attributes:

chunkSize defines the size of the number of batch processing cache entry, if the attribute value is greater than 0, the number of entries in the cache number is set for the attribute batch, will be less than if the attribute value is equal to 0, the batch is disabled
fetchInMemoryState If the attribute value is true, the cache request the data around the time instances in the cache start time of the impact loading buffer
If the attribute value is awaitInitialTransfer fetchInMemoryState true, awaitInitialTransfer property allows the first CacheManager.getCache () method of transporting blocked, the data cache until the cache is acquired adjacent instance. This property is applied to distributed mode and a copy mode, the default is true
timeout defines the maximum wait time data acquisition timeout adjacent cache instance, milliseconds, if the return is not received within the timeout period neighboring nodes, the cache loader is canceled, the corresponding exception is thrown

transport elements are transport configuration cache data, it has the following properties:

clusterName defines the name of the cluster, the cluster node only define the same name, to be able to join the cluster, building Infinispan data grid
distributedSyncTimeout defines waiting to acquire the distributed lock time, this distributed cache instance locks to ensure that the state exchange, distribution and real-time hash cache entry
strictPeerToPeer If set to true, if the cache is not present example, the copy operation fails, throw an exception; if set to false, if the cache is not present example, the operation fails ignored copy, just do simple logging
transportClass that define the class represents the transmission buffer

Distributed mode GET and PUT

In distributed mode, perform a GET operation before executing the write command. This occurs because the method (e.g., Cache.put ()) K value corresponding to the cache before returning with the key, is determined as follows:

cache.put("key", "value-previous-cache");
String previous = cache.put("key", "value");

Previous string value value-previous-cache. Indeed, Infinispan Cache object is achieved java.util.Map, the value corresponding to the Key Map interface defined previously put () method returns the required buffer. If the value is found, the corresponding return empty, but still GET operation is performed. Infinispan the write operation is PUT, GET PUT operation takes precedence over the operation, and always in sync, regardless of the configuration of Infinispan is synchronous or asynchronous, synchronous always waiting for the return value.

Infinispan GET operation in preference to PUT operation occurs, this operation is very expensive, especially when large mesh node key corresponding to the correlation value is relatively large, we can prevent the GET operation does not wait for each node returns, when receiving any node return, the end of the synchronization wait, add Flag.SKIP_REMOTE_LOOKUP mark in the GET operation can be ignored synchronized wait, this will not affect the operation of the cache, but undermines the definition of java.util.Map, if you need to use previously data exists, this flag is not set.

Copy mode

Infinispan mode of replication is a simple cluster cache, cache instance adjacent cache instance automatic discovery in a network environment, and then to construct a cluster, constructing such a plurality of nodes into a cluster grid. Adding a node to cache entries will be replicated to all other nodes, the same any node in the grid can get all the data in the cluster.

2, 4 Infinispan data server to construct a grid of four nodes, we added to the server data K 1 / V, the data is copied to the other three nodes.

FIG copy mode 2 Infinispan

Copy mode usage scenarios

Mode is used to copy a shared state in the cluster, the cluster number of squares performance considerations copy mode should be less than 10, because any change in the state of a point in the cluster needs to be copied to the other node, if the cluster network too many grid nodes, a large number of messages need to be transferred between nodes, network transmission pressure will affect the performance of the cluster. Copy mode message usually multicast mode of transmission, which may improve the performance to a certain extent. Copy mode, GET operation returns are returns from local without the need for remote dispatching, which makes the query faster, the advantages of this is also where the copy mode.

Configure replication mode

Infinispan copy mode belonging to the cluster buffer, arranged in the copy mode we need to add Infinispan server follows the cache container, particularly edit JDG / standalone / configuration / clustered.xml, was added to the buffer vessel replicated-cache. Also in the cache cluster configuration, JGroups must be reasonable configuration, Infinispan cluster caching using JGroups copy the data.

<cache-container name="local" default-cache="default" >
    <replicated-cache name="default" mode="{SYNC/ASYNC}" start="EAGER">
        <locking isolation="NONE" acquire-timeout="30000" concurrency-level="1000" striping="false" />
        <transaction mode="NONE" />
    </replicated-cache>
</cache-container>

replicated-cache configuration information element by replicated cache:

name defines the name of the cache, the cache is uniquely identified with
It defines the mode of replication cache, valid values SYNC (synchronous) and ASYNC (asynchronous)
determining whether the default start buffer to start buffer vessel instance, the valid values may be EAGER or LAZY, starting cache instance if the buffer container is started to EAGER, LAZY said load only when requested

In addition, we Infinispan by the API, using the copy mode cache loading CacheManager example, during loading Infinispan we need to specify the configuration file, the following example is to profile:

<clustering mode="repl">
    <sync replTimeout="${TIME}" />
    <stateTransfer chunkSize="${SIZE}" fetchInMemoryState="{true/false}" awaitInitialTransfer="{true/false}" timeout="${TIME}" />
    <transport clusterName="${NAME}" distributedSyncTimeout="${TIME}" strictPeerToPeer="{true/false}" transportClass="${CLASS}" />
</clustering>

mode attribute clustering element defines the cluster cache copy mode (repl).

stateTransfer element defines a cluster state transitions is how complete, the state transition occurs in a cluster node leaves or a new node is added. This element has the following attributes:

chunkSize defines the size of the number of batch processing cache entry, if the attribute value is greater than 0, the number of entries in the cache number is set for the attribute batch, will be less than if the attribute value is equal to 0, the batch is disabled
fetchInMemoryState If the attribute value is true, the cache request the data around the time instances in the cache start time of the impact loading buffer
If the attribute value is awaitInitialTransfer fetchInMemoryState true, awaitInitialTransfer property allows the first CacheManager.getCache () method of transporting blocked, the data cache until the cache is acquired adjacent instance. This property is applied to distributed mode and a copy mode, the default is true
timeout defines the maximum wait time data acquisition timeout adjacent cache instance, milliseconds, if the return is not received within the timeout period neighboring nodes, the cache loader is canceled, the corresponding exception is thrown

transport elements are transport configuration cache data, it has the following properties:

clusterName defines the name of the cluster, the cluster node only define the same name, to be able to join the cluster, building Infinispan data grid
distributedSyncTimeout defines waiting to acquire the distributed lock time, this distributed cache instance locks to ensure that the state exchange, distribution and real-time hash cache entry
strictPeerToPeer If set to true, if the cache is not present example, the copy operation fails, throw an exception; if set to false, if the cache is not present example, the operation fails ignored copy, just do simple logging
transportClass that define the class represents the transmission buffer

Copy mode asynchronous and synchronous replication

Data replication between the replication mode mesh nodes can be asynchronous can also be synchronized, we can choose according to their own needs of the application.

Replication transporting blocking thread (PUT operation) until the modified or added data update to all nodes on the node, all the nodes is not received confirmation operation is completed, the requesting thread has been transported in a wait state, so to ensure the integrity of replication.

Asynchronous and synchronous replication replication Instead, the client request thread relative transporting speed, there is no need to wait until the other mesh node sends back an acknowledgment message. Asynchronous replication perform the copy operation is done in the background, replication errors that occur during recorded in the log, asynchronous replication may happen that this client transporting end of the transaction is successful, but on one node transaction is a failure and that All data is not fully replicated to other nodes.

In some examples, such a phenomenon may occur, a cache configuration using asynchronous replication, but the reality is blocked waiting for synchronization return, it is because some of Infinispan operation is synchronous, such as state exchange, these operations are always synchronized, regardless of the configuration is asynchronous or synchronous. We can use the following method to make the exchange without the use of state synchronization:

Prohibit state exchange, using ClusteredCacheLoader load a remote node status when the state needs to obtain
REPL_SYNC used in the state exchange, using asynchronous API (e.g. cache.putAsync (k, v))), which can asynchronous switching state
Used in the state exchange REPL_SYNC, all remote dispatching is synchronous, but the client can not be blocked, which requires the use of a replication queue. We recommend this approach is

Replication Queue

Infinispan copy modes use a replication queue, the cache entry to copy the change to the other nodes. Copy queue using the following properties:

Previously set time interval
Size of the queue elements copied over
Combined with the previously set time interval exceeds the size of the queue elements replication

Replication queue will be ensured replication, replication queue in the cache entry by way of bulk copy, not a single one of the copy, thus reducing the copy number, increases the performance. Copying and asynchronous replication queue is typically used together. Use a replication queue, the client request is not blocked waiting for other nodes to return. When a copy queue, the main drawback is that replication is carried out periodically, and the size of this particular size of the queue according to the previously set interval. This may cause some of the backlog of messages in the queue time more often, of course, do not use the message queue will make the timely transmission of copy.

As follows using a replication queue configuration example:

<replicated-cache name="asyncCache"  start="EAGER"  mode="ASYNC" batching="false" indexing="NONE" queue-size="1000" queue-flush-interval="500">
	...
</replicated-cache>

Replication is used as the asynchronous replication queue, the queue size is 1000, the set time interval is 500 milliseconds.

Failure Mode

Failure mode is also a clustered cache, but in fact a cluster does not share any data between the various nodes, but simply run from the node removal may be outdated data. This cache mode only when other permanent storage, such as exists in the database is meaningful, that the mode is typically used to optimize system frequent read operations, the cache based on the Infinispan as a database, thus preventing the need to read each state access to the database. If the cache is configured for invalidation rather than replication, each time when the data is changed on one node, the other nodes in the cluster receive a message informing them that the data is now stale and should be evicted from the cache.

FIG 3 Infinispan Failure Mode

As shown, four Infinispan server 3 constructs a data grid into four nodes, we add data K / V_new server 1 to other nodes (servers 2, server 3, the server 4) will receive the message, the key data corresponding to the value of K is obsolete, which in the server 2 K / V_old expelled from the cache. Failure mode, when all nodes using a shared cache loading will cause remote cache retrieve modified data. The benefit of this is twofold: compared with the cache data copy, copy only the invalidation message to the network pressure is very small, network traffic is minimized; the other nodes in the cluster to load data loaded in a lazy way to load modification through data acquisition only when needed. Only after the cache invalidation message send the modified data, cache data can be modified in a batch process or transaction, the transaction limit is no transmission failure message and batch processing. After usually modify cache data submitted successfully failure message. This is usually more efficient as invalidation transaction information as a whole, rather than on a per modification.

Failure Mode Configuration

Belonging to the cluster buffer Infinispan failure mode, the failure mode configuration server Infinispan we need to add the following configuration cache container, particularly edit JDG / standalone / configuration / clustered.xml, was added to the buffer vessel invalidated-cache. Also in the cache cluster configuration, JGroups must be reasonable configuration, failure mode uses JGroups copy invalidation messages.

<cache-container name="local" default-cache="default" >
    <invalidated-cache name="default" mode="{SYNC/ASYNC}" start="EAGER">
        <locking isolation="NONE" acquire-timeout="30000" concurrency-level="1000" striping="false" />
        <transaction mode="NONE" />
    </invalidated-cache>
</cache-container>

invalidated-cache configuration element failure mode by the following information:

name defines the name of the cache, the cache is uniquely identified with
failure mode defines the message reproduction mode, the effective value of SYNC (synchronous) and ASYNC (asynchronous)
determining whether the default start buffer to start buffer vessel instance, the valid values may be EAGER or LAZY, starting cache instance if the buffer container is started to EAGER, LAZY said load only when requested

In addition, we Infinispan by the API, using the failure mode cache loading CacheManager example, during loading Infinispan we need to specify the configuration file, the following example is to profile:

<clustering mode="inv">
    <sync replTimeout="${TIME}" />
    <stateTransfer chunkSize="${SIZE}" fetchInMemoryState="{true/false}" awaitInitialTransfer="{true/false}" timeout="${TIME}" />
    <transport clusterName="${NAME}" distributedSyncTimeout="${TIME}" strictPeerToPeer="{true/false}" transportClass="${CLASS}" />
</clustering>

mode attribute clustering element defines the cluster cache failure mode (inv). replTimeout property sync element specifies the maximum timeout of waiting for fortune transported. If the timeout period has not yet received confirmation of remote dispatching, an exception is thrown. stateTransfer element defines a cluster state transitions is how complete, the state transition occurs in a cluster node leaves or a new node is added. This element has the following attributes:

chunkSize defines the size of the number of batch processing cache entry, if the attribute value is greater than 0, the number of entries in the cache number is set for the attribute batch, will be less than if the attribute value is equal to 0, the batch is disabled
fetchInMemoryState If the attribute value is true, the cache request the data around the time instances in the cache start time of the impact loading buffer
If the attribute value is awaitInitialTransfer fetchInMemoryState true, awaitInitialTransfer property allows the first CacheManager.getCache () method of transporting blocked, the data cache until the cache is acquired adjacent instance. This property is applied to distributed mode and a copy mode, the default is true
timeout defines the maximum wait time data acquisition timeout adjacent cache instance, milliseconds, if the return is not received within the timeout period neighboring nodes, the cache loader is canceled, the corresponding exception is thrown

transport elements are transport configuration cache data, it has the following properties:

clusterName defines the name of the cluster, the cluster node only define the same name, to be able to join the cluster, building Infinispan data grid
distributedSyncTimeout defines waiting to acquire the distributed lock time, this distributed cache instance locks to ensure that the state exchange, distribution and real-time hash cache entry
strictPeerToPeer If set to true, if the cache is not present example, the copy operation fails, throw an exception; if set to false, if the cache is not present example, the operation fails ignored copy, just do simple logging
transportClass that define the class represents the transmission buffer

Asynchronous and synchronous

Failure Failure mode operation can be synchronous, or asynchronous. In analogy to the copy mode, the synchronization failure blocking transmission failure message thread, until all nodes of the cluster receives all return failure information, failure data of all nodes expelled; asynchronous invalidation different and which broadcast the failure message, but does not block and awaits a response.

L1 cache

Repeatedly GET would create a duplicate of remote calls occur, in order to prevent repeated long-distance calls, we need to enable L1 cache. The value of the L1 cache of the remote return transported, stored locally in a very short period of time (configurable), so repeated GET operation does not cause a remote call repeated. Figure 4, if you enable L1 cache, then the same Server3 of GET KEY operation will not result in any long-distance calls.

4 L1 cache FIG.

l1 element includes the following attributes:

enabled L1 cache is enabled
lifespan remote transported saved in local time

L1 cache is not always contribute to performance, the L1 cache is turned consumption performance, any node on the cache entry is updated, a corresponding invalid message needs to be sent to other nodes in a multicast manner, the other nodes receive invalid message to the L1 cache in the cache entry is invalid, thus increasing the network load. If a large cache entry exists in the L1 cache it will consume a lot of memory, L1 cache and are suitable for reading some of the entries in the distributed cache mode in duplicate, to open a specific need to go through the L1 cache performance testing to ensure that help enhance performance for your scenario.

So far we have discussed the cache cluster (distributed mode, copy mode, failure mode) and a local cache, JBoss series thirty-two will discuss the contents of the above, through experimental tests verify the above theory.

Reproduced in: https: //my.oschina.net/iwuyang/blog/197214