Extend MapStruct based on AbstractProcessor to automatically generate entity mapping tool class

Author: JD Logistics Wang Beiyong Yao Zaiyi

1 background

In the daily development process, especially in the DDD process, the mutual conversion of domain models such as VO/MODEL/PO is often encountered. At this point, we will perform set|get settings field by field. Either use tools to perform violent attribute copying. In the process of violent attribute copying, a good tool can improve the running efficiency of the program, otherwise it will cause extreme situations such as low performance and hidden details setting OOM.

2 Existing technology

  1. Direct set|get method: It’s okay when there are few fields, but when the fields are very large, the workload is huge, and repeated operations are time-consuming and labor-intensive.
  1. Realize value mapping through reflection + introspection: For example, many open source apache-common, spring, and hutool tool classes provide such implementation tools. The disadvantage of this method is low performance and black-box property copying. The processing of different tool classes is different: the property copy of spring will ignore the type conversion but will not report an error, hutool will automatically perform type conversion, and some tool settings will throw exceptions, etc. There are production problems and positioning is difficult.
  1. mapstruct: Before using it, you need to manually define the converter interface, and automatically generate the implementation class according to the interface class annotation and method annotation. The attribute conversion logic is clear, but different domain object conversions need to write a separate layer of conversion interface or add a conversion method.

3 Extended design

3.1 mapstruct introduction

This extension component is extended based on mapstruct, and briefly introduces the principle of mapstruct implementation.

mapstruct is implemented based on JSR 269, which is a specification introduced by JDK. With it, it is possible to process annotations at compile time, and read, modify and add content in the abstract syntax tree. JSR 269 uses Annotation Processor to process annotations during compilation. Annotation Processor is equivalent to a plug-in of the compiler, so it is also called plug-in annotation processing.

 

We know that java's class loading mechanism needs to pass the compile time and run time. As shown below

mapstruct is in the process of compiling the source code during the above compilation period, and secondarily generates bytecode by modifying the syntax tree, as shown in the figure below

The above steps can be summarized as follows:

1. Generate an abstract syntax tree. The Java compiler compiles the Java source code and generates an abstract syntax tree (Abstract Syntax Tree, AST).

2. Call the program that implements the JSR 269 API. As long as the program implements the JSR 269 API, the implemented annotation processor will be invoked during compilation.

3. Modify the abstract syntax tree. In the program that implements JSR 269 API, you can modify the abstract syntax tree and insert your own implementation logic.

4. Generate bytecode. After the abstract syntax tree is modified, the Java compiler will generate a bytecode file corresponding to the modified abstract syntax tree.

From the perspective of the implementation principle of mapstruct, we found that mapstruct attribute conversion logic is clear and has good scalability. The problem is that it is necessary to write a separate conversion interface or add a conversion method. Can the conversion interface or method be automatically extended?

3.2 Improvement plan

The mapstruct scheme mentioned above has a drawback. That is, if there is a new domain model conversion, we have to manually write a layer of conversion interface. If there is a conversion between A/B models, it is generally necessary to define four methods: A->B, B->A, List<A >->List<B>, List<B>->List<A>

In view of this, this solution defines the original mapstruct in the annotation of the conversion interface class and the annotation of the conversion method, and forms a new packaging annotation through mapping. Define this annotation directly on the class or field of the model, and then generate the conversion interface directly at compile time according to the custom annotation on the model, and then mapstruct generates the specific conversion implementation class again according to the automatically generated interface.

Note: The annotations of the classes and methods in the automatically generated interface are the annotations of the original mapstruct, so the original functions of the mapstruct are not lost. The detailed adjustment is as follows:

4 realization

4.1 Technology dependency

  1. Compile-time annotation processor AbstractProcessor: Annotation Processor is equivalent to a plug-in of the compiler, so it is also called plug-in annotation processing. To achieve JSR 269, there are mainly the following steps .

1) Inherit the AbstractProcessor class, rewrite the process method, and implement your own annotation processing logic in the process method.

2) Create a javax.annotation.processing.Processor file in the META-INF/services directory to register your own implementation

2. Google AutoService: AutoService is an open source library created by Google to facilitate the generation of an open source library that complies with the ServiceLoader specification. It is very easy to use. Only need to add annotations to automatically generate specification constraint files.

Knowledge point: The advantage of using AutoService is that it helps us not to manually maintain the META-INF file directory and file content required by Annotation Processor. It will automatically produce for us, and the method of use is very simple, just add the following annotations to the custom Annotation Processor class @AutoService(Processor.class)

  1. mapstruct: Help implement the conversion interface automatically generated by the custom plug-in, and inject it into the spring container (described in the existing solution).
  2. javapoet: JavaPoet is an open source library for dynamically generating code. Help us generate java class files easily and quickly. The main features are as follows:

1) JavaPoet is a third-party dependency that can automatically generate Java files.

2) Simple and easy-to-understand API, easy to use.

3) Automatically generate complex and repetitive Java files to improve work efficiency and simplify the process.

4.2 Implementation steps

  • Step 1: Automatically generate the enumeration required to convert the interface class, respectively annotate the class AlpacaMap and the field AlpacaMapField.

1) AlpacaMap: defined on the class, the attribute target specifies the target model to be converted; the attribute uses specifies the external objects on which the Alpaca conversion process depends.

2) AlpacaMapField: Do an alias wrapping for all annotations supported by the original mapstruct, and use the AliasFor annotation provided by spring.

Knowledge points: @AliasFor is an annotation of the Spring framework, which is used to declare the alias of the annotation attribute. It has two different application scenarios:

Aliases in annotations

Metadata Alias

The main difference between the two is whether they are in the same annotation.

  • Step 2: AlpacaMapMapperDescriptor implementation. The main function of this class is to load all the model classes that use the first step to define the enumeration, and then save the class information and class Field information for direct use later. The fragment logic is as follows:
AutoMapFieldDescriptor descriptor = new AutoMapFieldDescriptor();
            descriptor.target = fillString(alpacaMapField.target());
            descriptor.dateFormat = fillString(alpacaMapField.dateFormat());
            descriptor.numberFormat = fillString(alpacaMapField.numberFormat());
            descriptor.constant = fillString(alpacaMapField.constant());
            descriptor.expression = fillString(alpacaMapField.expression());
            descriptor.defaultExpression = fillString(alpacaMapField.defaultExpression());
            descriptor.ignore = alpacaMapField.ignore();
             ..........

 

  • Step 3: The AlpacaMapMapperGenerator class mainly generates corresponding class information, class annotations, class methods, and annotation information on methods through JavaPoet
Generated class information: TypeSpec createTypeSpec(AlpacaMapMapperDescriptor descriptor) 
Generated class annotation information AnnotationSpec buildGeneratedMapperConfigAnnotationSpec(AlpacaMapMapperDescriptor descriptor) { 
Generated class method information: MethodSpec buildMappingMethods(AlpacaMapMapperDescriptor descriptor) 
Generated method annotation information: List<AnnotationSpec> buildMethodMapperMapperAnnotation(AlpacaMapMapperDescriptor descriptor)

In the process of realizing the generated class information, it is necessary to specify the interface class AlpacaBaseAutoAssembler of the generated class. This class mainly defines four methods as follows:

public interface AlpacaBaseAutoAssembler<S,T>{
    T copy(S source);

    default List<T> copyL(List<S> sources){
        return sources.stream().map(c->copy(c)).collect(Collectors.toList());
    }

    @InheritInverseConfiguration(name = "copy")
    S reverseCopy(T source);

    default List<S> reverseCopyL(List<T> sources){
        return sources.stream().map(c->reverseCopy(c)).collect(Collectors.toList());
    }
}
  • Step 4: Because the generated class converter is injected into the spring container. Therefore, it is necessary to top an annotation that specifically generates mapstruct injection into the spring container. This annotation is automatically generated through the class AlpacaMapSpringConfigGenerator. The core code is as follows
private AnnotationSpec buildGeneratedMapperConfigAnnotationSpec() {
        return AnnotationSpec.builder(ClassName.get("org.mapstruct", "MapperConfig"))
                .addMember("componentModel", "$S", "spring")
                .build();
    }
  • Step 5: Through the above steps, we have defined related classes, related class methods, related class annotations, and related class method annotations. At this time, string them together to generate class file output through Annotation Processor. The core method is as follows
private void writeAutoMapperClassFile(AlpacaMapMapperDescriptor descriptor){
        System.out.println("开始生成接口:"+descriptor.sourcePackageName() + "."+ descriptor.mapperName());
        try (final Writer outputWriter =
                     processingEnv
                             .getFiler()
                             .createSourceFile(  descriptor.sourcePackageName() + "."+ descriptor.mapperName())
                             .openWriter()) {
            alpacaMapMapperGenerator.write(descriptor, outputWriter);
        } catch (IOException e) {
            processingEnv
                    .getMessager()
                    .printMessage( ERROR,   "Error while opening "+ descriptor.mapperName()  + " output file: " + e.getMessage());
        } 
    }

Knowledge points: In javapoet, the core class first has about a few classes, which can be referred to as follows:

JavaFile is used to construct and output a Java file containing a top-level class, which is an abstract definition of a .java file

TypeSpec TypeSpec is the abstract type of a class/interface/enumeration

MethodSpec MethodSpec is an abstract definition of a method/constructor

FieldSpec FieldSpec is an abstract definition of a member variable/field

ParameterSpec ParameterSpec is used to create method parameters

AnnotationSpec AnnotationSpec is used to create marker annotations

 

5 practice

The following is an example to illustrate how to use it. Here we define a model Person and a model Student, which involve ordinary strings, enumerations, time formatting and complex type replacement of field conversions. The specific steps are as follows.

5.1 Introducing dependencies

The code has been uploaded to the code base. If you need specific requirements, you can re-pull the branch and package it for use

<dependency>
            <groupId>com.jdl</groupId>
            <artifactId>alpaca-mapstruct-processor</artifactId>
            <version>1.1-SNAPSHOT</version>
        </dependency>

5.2 Object Definition

The uses method must be a bean in a normal spring container. This bean provides a @Named annotation method for class field annotations. The qualifiedByName attribute in AlpacaMapField can be specified as a string, as shown in the figure below

@Data
@AlpacaMap(targetType = Student.class,uses = {Person.class})
@Service
public class Person {
    private String make;
    private SexType type;

    @AlpacaMapField(target = "age")
    private Integer sax;

    @AlpacaMapField(target="dateStr" ,dateFormat = "yyyy-MM-dd")
    private Date date;

    @AlpacaMapField(target = "brandTypeName",qualifiedByName ="convertBrandTypeName")
    private Integer brandType;

    @Named("convertBrandTypeName")
    public  String convertBrandTypeName(Integer brandType){
        return BrandTypeEnum.getDescByValue(brandType);
    }

    @Named("convertBrandTypeName")
    public  Integer convertBrandType(String brandTypeName){
        return BrandTypeEnum.getValueByDesc(brandTypeName);
    }
}

5.3 Generating results

Use maven to package or compile and observe. At this time, two files PersonToStudentAssembler and PersonToStudentAssemblerImpl are generated in the target/generated-source/annotatins directory

The class file PersonToStudentAssembler is automatically generated by the custom annotator, the content is as follows

@Mapper(
    config = AutoMapSpringConfig.class,
    uses = {Person.class}
)
public interface PersonToStudentAssembler extends AlpacaBaseAutoAssembler<Person, Student> {
  @Override
  @Mapping(
      target = "age",
      source = "sax",
      ignore = false
  )
  @Mapping(
      target = "dateStr",
      dateFormat = "yyyy-MM-dd",
      source = "date",
      ignore = false
  )
  @Mapping(
      target = "brandTypeName",
      source = "brandType",
      ignore = false,
      qualifiedByName = "convertBrandTypeName"
  )
  Student copy(final Person source);
}

PersonToStudentAssemblerImpl is automatically generated by mapstruct according to the PersonToStudentAssembler interface annotator, the content is as follows

@Component
public class PersonToStudentAssemblerImpl implements PersonToStudentAssembler {

    @Autowired
    private Person person;

    @Override
    public Person reverseCopy(Student arg0) {
        if ( arg0 == null ) {
            return null;
        }
        Person person = new Person();
        person.setSax( arg0.getAge() );
        try {
            if ( arg0.getDateStr() != null ) {
                person.setDate( new SimpleDateFormat( "yyyy-MM-dd" ).parse( arg0.getDateStr() ) );
            }
        } catch ( ParseException e ) {
            throw new RuntimeException( e );
        }
        person.setBrandType( person.convertBrandType( arg0.getBrandTypeName() ) );
        person.setMake( arg0.getMake() );
        person.setType( arg0.getType() );
        return person;
    }

    @Override
    public Student copy(Person source) {
        if ( source == null ) {
            return null;
        }
        Student student = new Student();
        student.setAge( source.getSax() );
        if ( source.getDate() != null ) {
            student.setDateStr( new SimpleDateFormat( "yyyy-MM-dd" ).format( source.getDate() ) );
        }
        student.setBrandTypeName( person.convertBrandTypeName( source.getBrandType() ) );
        student.setMake( source.getMake() );
        student.setType( source.getType() );
        return student;
    }
}

5.4 Spring container reference

At this time, in our spring container, @Autowired can directly introduce the interface PersonToStudentAssembler instance to perform four kinds of maintenance data conversion

AnnotationConfigApplicationContext applicationContext = new  AnnotationConfigApplicationContext();
        applicationContext.scan("com.jdl.alpaca.mapstruct");
        applicationContext.refresh();
        PersonToStudentAssembler personToStudentAssembler = applicationContext.getBean(PersonToStudentAssembler.class);
        Person person = new Person();
        person.setMake("make");
        person.setType(SexType.BOY);
        person.setSax(100);
        person.setDate(new Date());
        person.setBrandType(1);
        Student student = personToStudentAssembler.copy(person);
        System.out.println(student);
        System.out.println(personToStudentAssembler.reverseCopy(student));
        List<Person> personList = Lists.newArrayList();
        personList.add(person);
        System.out.println(personToStudentAssembler.copyL(personList));
        System.out.println(personToStudentAssembler.reverseCopyL(personToStudentAssembler.copyL(personList)));

Console prints:

personToStudentStudent(make=make, type=BOY, age=100, dateStr=2022-11-09, brandTypeName=集团KA)
studentToPersonPerson(make=make, type=BOY, sax=100, date=Wed Nov 09 00:00:00 CST 2022, brandType=1)
personListToStudentList[Student(make=make, type=BOY, age=100, dateStr=2022-11-09, brandTypeName=集团KA)]
studentListToPersonList[Person(make=make, type=BOY, sax=100, date=Wed Nov 09 00:00:00 CST 2022, brandType=1)]

Notice:

  • The qualifiedByName annotation attribute is not friendly to use. If this attribute is used, an inversion type conversion function needs to be defined. Because the abstract interface AlpacaBaseAutoAssembler we defined earlier has an annotation as shown in the figure below, the reverse mapping from the target object to the source object, because of the overloading of java, the same name and different references are not the same method, so when S is transferred to T, it will be retrieved This method is not available. Therefore, you need to define your own conversion function
@InheritInverseConfiguration(name = "copy")

For example, when converting from S to T, the first method will be used. When converting from T to S, a method with the same Named annotation must be defined.

@Named("convertBrandTypeName")
    public  String convertBrandTypeName(Integer brandType){
        return BrandTypeEnum.getDescByValue(brandType);
    }

    @Named("convertBrandTypeName")
    public  Integer convertBrandType(String brandTypeName){
        return BrandTypeEnum.getValueByDesc(brandTypeName);
    }
  • When using the qualifiedByName annotation, the specified Named annotation method must be defined as an object that can be managed by the spring container, and this object Class needs to be introduced through the model class annotation attribute used

Knowledge points:

InheritInverseConfiguration is very powerful and can be reverse mapped. From the above PersonToStudentAssemblerImpl, we can see that the above attribute sax can be mapped to sex, and the reverse mapping can be automatically mapped from sex to sax. But the @Mapping#expression, #defaultExpression, #defaultValue and #constant that are being mapped will be ignored by the reverse mapping. In addition, the inverse mapping of a field can be overridden by ignore, expression or constant

6 Conclusion

Reference documents:

https://github.com/google/auto/tree/master/service

https://mapstruct.org/

https://github.com/square/javapoet

{{o.name}}
{{m.name}}

Guess you like

Origin my.oschina.net/u/4090830/blog/6814571