Apache Ignite study notes (6): Ignite use the Entry Processor

Previous article, we have actually used two different ways to access data in Ignite. One way is the first article mentioned by JDBC client using SQL to access data, in this article we will see not using JDBC, how Ignite API using SQL to access data. There is a way to use what I call the cache API, to access data that is used get / put. Ignite achieved JCache (JSR 107) standard, so in addition to the basic cache operation, we will introduce the use of atomic operations and EntryProcessor number of cache.

Cache API

---

Map of Ignite provides a similar API for manipulating data on the cache, but the realization of the Ignite the data on the Map distributed across multiple nodes, and to ensure that these operations are multi-thread / process safety. We can simply use on multiple nodes get / put to Ignite cache read and write data, and the data synchronization, concurrency control and other complex issues left to be resolved Ignite. In addition to get / put operation, the Ignite also provides other atomic operations and asynchronous operation, such getAndPutIfAbsent, getAndPutAsync, putIfAbsent, putIfAbsentAsync, getAndReplace, getAndReplaceAsync the like, a complete list of API can be seen here .

Ignite also supports entry processor JCache defined in the standard. I did not carefully read the JCache definition of entry processor, but according to Ignite documentation and experience, compared to basic caching get / put operations, entry processor are the following Features / Benefits:

1. Compared to the basic operation of the get / put, etc., in the entry processor, we can achieve more complex cache update logic, for example, we can read a value in the cache, and then after doing some custom calculations, and then update the cache value.
2. And get / put / putIfAbsent operations such as in the entry processor in all the operations are atomic, which is to ensure the operation entry processor as defined in either succeed or fail. If no entry processor, in order to achieve the same purpose, we need to need to be updated lock data cache, the cache update data, and finally release the lock. And with entry proce, we can focus more on logical cache update, regardless of how the locking and unlocking.
3. Entry processor allows to operate directly on the data node. Distributed cache, if the updated cached data has been calculated according to the data in the cache, the cache data often need to be transmitted between the nodes. The entry processor is to be sent to the node where the cache data of the sequence of operations, compared to the sequence of the data buffer, to be more efficient.

Entry Processor code sample

Here we look at previous examples of reconstruction, and to see how a call entry processor in the Ignite. In this example, the value of the cache key is still the name of the city, but the value of value is no longer the name of the province where a simple city, but a City instance of the class. Here is the definition of class City:

public class City {
    private String cityName;
    private String provinceName;
    private long population;

    public City(String cityName, String provinceName, long population) {
        this.cityName = cityName;
        this.provinceName = provinceName;
        this.population = population;
    }

   ...
}

In the City class, we put a member variable of a population that represents the population of the city. In the main program, we create multiple threads, the entry processor constantly revised the number of people in different cities. Each entry processor to do is very simple: read the current population plus 1, and then updated to the new value of the cache. The following is the main program code

public class IgniteEntryProcessorExample {
    public static void main(String[] args) {
        // start an ignite cluster
        Ignite ignite = startCluster(args);

        CacheConfiguration<String, City> cacheCfg = new CacheConfiguration<>();
        cacheCfg.setName("CITY");
        cacheCfg.setCacheMode(CacheMode.PARTITIONED);
        cacheCfg.setBackups(1);
        IgniteCache<String, City> cityProvinceCache = ignite.getOrCreateCache(cacheCfg);

        // let's create a city and put it in the cache
        City markham = new City("Markham", "Ontario", 0);
        cityProvinceCache.put(markham.getCityName(), markham);
        System.out.println("Insert " + markham.toString());

        // submit two tasks to increase population
        ExecutorService service = Executors.newFixedThreadPool(2);
        IncreaseCityPopulationTask task1 = new IncreaseCityPopulationTask(cityProvinceCache, markham.getCityName(), 10000);
        IncreaseCityPopulationTask task2 = new IncreaseCityPopulationTask(cityProvinceCache, markham.getCityName(), 20000);
        Future<?> result1 = service.submit(task1);
        Future<?> result2 = service.submit(task2);
        System.out.println("Submit two tasks to increase the population");

        service.shutdown();
        try {
            service.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        // get the population and check whether it is 30000
        City city = cityProvinceCache.get(markham.getCityName());
        if (city.getPopulation() != 30000) {
            System.out.println("Oops, the population is " + city.getPopulation() + " instead of 30000");
        } else {
            System.out.println("Yeah, the population is " + city.getPopulation());
        }
    }

    public static class IncreaseCityPopulationTask implements Runnable {
        private IgniteCache<String, City> cityProvinceCache;
        private String cityName;
        private long population;

        public IncreaseCityPopulationTask(IgniteCache<String, City> cityProvinceCache,
                                          String cityName, long population) {
            this.cityProvinceCache = cityProvinceCache;
            this.cityName = cityName;
            this.population = population;
        }

        @Override
        public void run() {
            long p = 0;
            while(p++ < population) {
                cityProvinceCache.invoke(cityName, new EntryProcessor<String, City, Object>() {

                    @Override
                    public Object process(MutableEntry<String, City> mutableEntry, Object... objects)
                            throws EntryProcessorException {
                        City city = mutableEntry.getValue();
                        if (city != null) {
                            city.setPopulation(city.getPopulation() + 1);
                            mutableEntry.setValue(city);
                        }
                        return null;
                    }
                });
            }
        }
    }

    private static Ignite startCluster(String[] args) {
      ...
    }
}

• 4 to 10 lines, and the previous example, we start a Ignite node, and creates a cache named "CITY" of, cache the key is the city's name (String), cache the value is a City object instance.
• 13 to 15 lines, we have created a name for "Markham" The City instance, its initial value is 0 population.
• 18 to 30 lines, we create two threads, each thread starts after IncreaseCityPopulationTask will call the Run () function, except that when we specify a different threads created to increase the number of population, a increase of 10,000, a 20,000 increase times.
• In lines 33 to 38, we retrieved from cache instance named "Markham" and check it is not the final population 30,000. If the operation between the two threads (read cache, increase in population, write cache) is an atomic operation, then the end result should be 30,000.
• 57 to 68 is a specific usage Entry Processor by cityProvinceCache.invoke function can be called entry processor (), a first parameter invoke () function is a function key to entry processor data. The second parameter is an instance of entry processor, the instance must implement the interface class EntryProcessor process () function. After the second parameter, can also pass a plurality of parameters, these parameters will be passed when calling the process () function.
• In process () function, the first parameter contains mutableEntry process () function operates the key data and value, can be obtained (if the key value does not exist in the cache by MutableEntry.getKey () and MutableEntry.getValue () in, getValue () returns null). After the second parameter of objects, calling invoke () In addition to the key and EntryProcessor time function, passing the parameters.
• Entry processor may be implemented in a number of complex logic, then call MutableEntry.setValue () value to modify the values. If you need to remove the value, call MutableEntry.remove ().
• EntryProcessor () is called, cache corresponding key values ​​are locked, so that different entry processor for the same key are mutually exclusive, to ensure that all operations are entry processor an atomic operation.
• In addition, one thing should be noted that, stateless operation requires entry processor is because there is a single entry processor may execute multiple times on the primary and backup nodes, so make sure the only operating entry processor and cache the current value of the correlation, and also if the parameters and the state of the current node relevant, can cause write run on different nodes entry processor cache inconsistent values. For details, see invoke () function of the document.

to sum up

---

In this article we introduced the basic put Ignite Cache / get () other operations outside operations, such as asynchronous operation and entry processor ** this article used the example of the complete code and maven project can be here found.

The next article, we will continue to look at how to use Ignite the SQL API to query and modify cache.