day96-performance stress test-optimization-optimization of three-level classification data acquisition (reduce the number of interactions with the database)

Sometimes, through reasonable business code design to avoid frequent database interactions, throughput can also be greatly improved

Here is the table of our previous throughput test, you can see a batch of slow throughput obtained by three-level classification data

This is the previous code. If you look closely, you can see that every time you get the sub-category, you will get it from the database and continue to interact with the database. The larger the amount of data in the parent category, the more interactions you have. 

    @Override
    public Map<String, List<Catelog2Vo>> getCatalogJson() {

        List<CategoryEntity> category1EntityList = findCatelog1s();

        Map<String, List<Catelog2Vo>> catelog2Vos = category1EntityList.stream().collect(Collectors.toMap(k -> k.getCatId().toString(), v -> {
            //1.查到一级分类的所有二级分类
            List<CategoryEntity> category2EntityList = this.list(new QueryWrapper<CategoryEntity>().eq("parent_cid", v.getCatId()));

            //2.获取二级分类vo集合
            List<Catelog2Vo> catelog2VoList = category2EntityList.stream().map(catelog2 -> {

                List<CategoryEntity> category3EntityList = this.list(new QueryWrapper<CategoryEntity>().eq("parent_cid", catelog2.getCatId()));

                //3.获取三级分类vo集合
                List<Catelog2Vo.Catelog3Vo> catelog3Vos = null;
                if(category3EntityList !=null && category3EntityList.size() > 0){
                    catelog3Vos = category3EntityList.stream().map(catelog3 -> {
                        //获取三级分类vo
                        Catelog2Vo.Catelog3Vo catelog3Vo = new Catelog2Vo.Catelog3Vo(catelog3.getParentCid().toString(), catelog3.getCatId().toString(), catelog3.getName());
                        return catelog3Vo;
                    }).collect(Collectors.toList());
                }
                //2.获取二级分类vo
                Catelog2Vo catelog2Vo = new Catelog2Vo(v.getCatId().toString(), catelog3Vos, catelog2.getCatId().toString(), catelog2.getName());

                return catelog2Vo;
            }).collect(Collectors.toList());
            return catelog2VoList;
        }));

        return catelog2Vos;
    }

Hands-on

Idea: First query all the classified data at once, and then get the corresponding sub-categories from it each time. The code after transformation is as follows

    @Override
    public Map<String, List<Catelog2Vo>> getCatalogJson() {

        List<CategoryEntity> categoryEntities = this.list(null);

        List<CategoryEntity> category1EntityList = getParent_cid(categoryEntities,0l);

        Map<String, List<Catelog2Vo>> catelog2Vos = category1EntityList.stream().collect(Collectors.toMap(k -> k.getCatId().toString(), v -> {
            //1.查到一级分类的所有二级分类
            List<CategoryEntity> category2EntityList = getParent_cid(categoryEntities,v.getCatId());

            //2.获取二级分类vo集合
            List<Catelog2Vo> catelog2VoList = category2EntityList.stream().map(catelog2 -> {

                List<CategoryEntity> category3EntityList = getParent_cid(categoryEntities,catelog2.getCatId());

                //3.获取三级分类vo集合
                List<Catelog2Vo.Catelog3Vo> catelog3Vos = null;
                if(category3EntityList !=null && category3EntityList.size() > 0){
                    catelog3Vos = category3EntityList.stream().map(catelog3 -> {
                        //获取三级分类vo
                        Catelog2Vo.Catelog3Vo catelog3Vo = new Catelog2Vo.Catelog3Vo(catelog3.getParentCid().toString(), catelog3.getCatId().toString(), catelog3.getName());
                        return catelog3Vo;
                    }).collect(Collectors.toList());
                }
                //2.获取二级分类vo
                Catelog2Vo catelog2Vo = new Catelog2Vo(v.getCatId().toString(), catelog3Vos, catelog2.getCatId().toString(), catelog2.getName());

                return catelog2Vo;
            }).collect(Collectors.toList());
            return catelog2VoList;
        }));

        return catelog2Vos;
    }

    private List<CategoryEntity> getParent_cid( List<CategoryEntity> categoryEntities,Long parent_cid) {

        List<CategoryEntity> collect = categoryEntities.stream().filter(categoryEntity -> categoryEntity.getParentCid() == parent_cid).collect(Collectors.toList());
        return collect;
    }

 

Pressure test again

 

Check the aggregation report and find that the throughput is dozens of times higher than before, although it is still not too high 

Fill out the form

 If you still need to optimize, you need to go to the next article, use the cache to continue to optimize