The in-memory caching system is designed to increase application performance by holding frequently-requested data in memory, reducing the need for database queries to get that data.
The caching system is optimized for use in a clustered installation, where you set up and configure a separate external cache server. In a single-machine installation, the application will use a local cache in the application's server's process, rather than a cache server.
Parts of the In-Memory Caching System
In a clustered installation, caching system components interoperate with the clustering system to provide fast response to client requests while also ensuring that cached data is available to all nodes in the cluster.
Application server. The application manages the relationship between user requests, the near cache, the cache server, and the database.
Near cache. Each application server has its own near cache for the data most recently requested from that cluster node. The near cache is the first place the application looks, followed by the cache server, then the database.
Cache server. The cache server is installed on a machine separate from application server nodes in the cluster. It's available to all nodes in the cluster (in fact, you can't create a cluster without declaring the address of a cache server).
Local cache. The local cache exists mainly for single-machine installations, where a cache server might not be present. Like the near cache, it lives with the application server. The local cache should only be used for single-machine installations or for data that should not be available to other nodes in a cluster. An application server's local cache does not participate in synchronization across the cluster.
Clustering system. The clustering system reports near cache changes across the application server nodes. As a result, although data is not fully replicated across nodes, all nodes are aware when the content of their near caches must be updated from the cache server or the database.
How In-Memory Caching Works
For typical content retrievals, data is returned from the near cache (if the data has been requested recently from the current application server node), from the cache server (if the data has been recently requested from another node in the cluster), or from the database (if the data is not in a cache).
Data retrieved from the database is placed into a cache so that subsequent retrievals will be faster.
Here's an example of how changes are handled:
- Client makes a change, such as an update to a user profile. Their change is made through node A of the cluster, probably via a load balancer.
- The node A application server writes the change to the application database.
- The node A app server puts the newly changed data into its near cache for fast retrieval later.
- The node A app server puts the newly changed data to the cache server, where it will be found by other nodes in the cluster.
- Node A tells the clustering system that the contents of its near cache have changed, passing along a list of the changed cache items. The clustering system collects change reports and regularly sends them in a batch to other nodes in the cluster. Near caches on the other nodes drop any entries corresponding to those in the change list.
- When the node B app server receives a request for the data that was changed, and which it has removed from its near cache, it looks to the cache server.
- Node B caches the fresh data in its own near cache.
Cache Server Deployment Design
In a clustered configuration, the cache server should be installed on a machine separate from the clustered application server nodes. That way, the application server process is not contending for CPU cycles with the cache server process. It is possible to have the application server run with less memory than in a single-machine deployment design. Also note that it is best if the cache servers and the application servers are located on the same network switch. This will help reduce latency between the application servers and the cache servers.
Choosing the Number of Cache Server Machines
A single dedicated cache server with four cores can easily handle the cache requests from up to six application server nodes running under full load. All cache server processes are monitored by a daemon process which will automatically restart the cache server if the JVM fails completely.
In a cluster, the application will continue to run even if all cache servers fail. However, performance will degrade significantly because requests previously handled via the cache will be transferred to the database, increasing its load significantly.
Adjusting Cache-Related Memory
Adjusting Near Cache Memory
The near cache, which runs on each application server node, starts evicting cached items to free up memory once the heap reaches 75 percent of the maximum allowed size. When you factor in application overhead and free space requirements to allow for efficient garbage collection, a 2GB heap means that the typical amount of memory used for caching will be no greater than about 1GB.
For increased performance (since items cached in the near cache are significantly faster to retrieve than items stored remotely on the cache server) larger sites should increase the amount of memory allocated to the application server process. To see if this is the case, you can watch the GC logs (or use a tool such as JConsole or VisualVM after enabling JMX), noting if the amount of memory being used never goes below about 70 percent even after garbage collection occurs.
Adjusting Cache Server Memory
The cache server process acts similarly to the near cache. However, it starts eviction once the heap reaches 80 percent of the maximum amount. On installations with large amounts of content, the default 1GB allocated to the cache server process may not be enough and should be increased.
To adjust the amount of memory the cache server process will use, edit the /etc/jive/conf/cache.conf file, uncomment the following line, and set it to a new value:
#JVM_HEAP='1024'
Make sure to set the min and the max to the same value -- otherwise, evictions may occur prematurely. If you need additional cache server memory, recommended values are 2048 (2GB) or 4096 (4GB). You'll need to restart the cache server for this change to take effect. See Managing Cache Servers for more information.