SqlSource and dynamic sql parsing
mybaits can be interpolated by #{} or spliced by ${}. Another most powerful feature is that it can dynamically parse sql statements, regardless of dynamic or static analysis, sql is passed Different SqlSource implementations are packaged. This article will explain in depth the dynamic sql parsing.
1. Analysis of sql parsing and loading
1. The main parsing method XMLScriptBuilder's parseScriptNode.
a) Parse the Node node of the <select> statement tag
public SqlSource parseScriptNode() {
// 核心： 解析动态sql , 并且动态sql的存储形式是以SqlNode节点列表的方式
List<SqlNode> contents = parseDynamicTags(context);
// SqlNode的节点
MixedSqlNode rootSqlNode = new MixedSqlNode(contents);
SqlSource sqlSource = null;
if (isDynamic) { // 创建动态SqlSource
sqlSource = new DynamicSqlSource(configuration, rootSqlNode);
} else { // 创建静态SqlSource (RawSqlSource是StaticSqlSource简单包装)
sqlSource = new RawSqlSource(configuration, rootSqlNode, parameterType);
}
return sqlSource;
}b) Parse dynamic sqlparseDynamicTags
private List<SqlNode> parseDynamicTags(XNode node) {
List<SqlNode> contents = new ArrayList<SqlNode>();
NodeList children = node.getNode().getChildNodes(); // <select>的所有子节点
for (int i = 0; i < children.getLength(); i++) {
XNode child = node.newXNode(children.item(i));
// 如果是文本节点
if (child.getNode().getNodeType() == Node.CDATA_SECTION_NODE || child.getNode().getNodeType() == Node.TEXT_NODE) {
String data = child.getStringBody("");
// 包装成TextSqlNode,内部判断是否有${}，确定是否动态.
TextSqlNode textSqlNode = new TextSqlNode(data);
if (textSqlNode.isDynamic()) {
contents.add(textSqlNode);
isDynamic = true;
} else {
// 否则，另外包装成StaticTextSqlNode节点
contents.add(new StaticTextSqlNode(data));
}
// 如果是元素节点
} else if (child.getNode().getNodeType() == Node.ELEMENT_NODE) { // issue #628
String nodeName = child.getNode().getNodeName();
NodeHandler handler = nodeHandlers.get(nodeName);
if (handler == null) {
throw new BuilderException("Unknown element <" + nodeName + "> in SQL statement.");
}
// 处理此if等动态节点，内部会递归调用parseDynamicTags，将解析的List<SqlNode>包装成MixedSqlNode。
handler.handleNode(child, contents);
isDynamic = true; // 只要存在一个动态形式，那么isDynamic就是动态的。
}
}
return contents;
} 2. Process summary
From the main analysis process, we know that the sql analysis of mybaits is to finally parse the node node in the statement label into SqlNode, and SqlNode contains text nodes, if nodes, etc., if we only have static SqlNode, then finally these SqlNode will be packaged as StaticSqlSource, otherwise it will be packaged as DynamicSqlSource. Let's first introduce the main class structure of SqlNode and SqlSource to facilitate subsequent detailed analysis.
The following figure demonstrates the combination of sql nodes after parsing.
3. Introduction to SqlNode
public interface SqlNode {
// 备注： apply是sql运行时期用的，因为这个时候才有sql参数，才可以解析OGNL，返回false，不进行后续节点的解析。
boolean apply(DynamicContext context);
}This is the interface method of SqlNode. The apply method is mainly to process the SQL fragments of SqlNode to form the final use of SQL prepared statements. The analysis of dynamic SQL mainly relies on OGNL analysis. We first imagine DynamicContext as a map to store external operating parameters, which is the purpose of splicing sql nodes.
The main implementation:
IfSqlNode : if node packaging, the apply logic is to determine whether the expression is ture parsed with the operating parameter OGNL, and to decide whether to splice this node sql in the future.
TextSqlNode : Text node, containing ${} placeholders, apply logic is to run parameter OGNL to parse and replace placeholders .
StaticTextSqlNode : Pure static text node, only stores sql fragments.
MixedSqlNode : A list of multiple SqlNode nodes. The apply logic is to call each SqlNode to apply.
...
SqlNode's final analysis apply method at runtime is to use OGNL to process the ${} placeholder parameters for text nodes, and add static text nodes directly. If the dynamic node is an if node, then use OGNL to parse the test value with operating parameters and return true. Just add the sql of this node. The apply logic of other dynamic nodes is similar. In addition, all sql is spliced, and #{} placeholders will be processed. This is a direct replacement? to parse paramterMapping. It will be explained when the SqlSource class is introduced.
4 Introduction to SqlSource
Top-level interface:
public interface SqlSource {
BoundSql getBoundSql(Object parameterObject);
}Main realization:
DynamicSqlSource : The dynamic sqlSource contains a combined MixedSqlNode. Call apply, and the spliced sql will also be processed #{}
StaticSqlSource : The static sqlSource directly contains the final prepared statement.
RawSqlSource : Packaging static sqlSource, mainly processing #{}, parsed into? From this, and construct ParameterMapping.
Dependent class:
BoundSql : package final sql, paramterMapping, parameter value.
public class DynamicSqlSource implements SqlSource {
private Configuration configuration;
private SqlNode rootSqlNode; // 顶级SqlNode
public DynamicSqlSource(Configuration configuration, SqlNode rootSqlNode) {
this.configuration = configuration;
this.rootSqlNode = rootSqlNode;
}
public BoundSql getBoundSql(Object parameterObject) {
DynamicContext context = new DynamicContext(configuration, parameterObject); // 实际参数添加到DynamicContext
// 所有SqlNode的apply操作，动态node采用CGNL解析，最后拼接sql到context的sql属性
rootSqlNode.apply(context);
SqlSourceBuilder sqlSourceParser = new SqlSourceBuilder(configuration);
Class<?> parameterType = parameterObject == null ? Object.class : parameterObject.getClass();
// 因为上面处理的sql没有处理#{}的占位，并且#{}可以解析出ParameterMapping，替换成?
SqlSource sqlSource = sqlSourceParser.parse(context.getSql(), parameterType, context.getBindings());
BoundSql boundSql = sqlSource.getBoundSql(parameterObject);
// 实际参数放到_parameter为key的AdditionalParameter的Map，添加到boundSql
for (Map.Entry<String, Object> entry : context.getBindings().entrySet()) {
boundSql.setAdditionalParameter(entry.getKey(), entry.getValue());
}
return boundSql;
}
}
Second, the use stage of SqlSource
After SqlSource is parsed, where is it stored? Where is it used? We enter the interpretation of the SqlSource use phase.
1. Where is it stored?
Regardless of whether SqlSource is dynamic sql or non-dynamic sql, its storage location must correspond to a statement object, and the statement object is MappedStatement.
public final class MappedStatement {
private SqlSource sqlSource;2. Where is it used?
We review the main execution process of mybaits, create Sqlsession, when executing a query, sqlsession will find the specific MappedStatement according to the namespace + method id, take out the SqlSource from the MappedStatement and combine the externally passed parameters, and process the sqlSource to parse it into a prepared statement + parameter mapping package. BoundSql is the specific use process.
Sqlsession's selectList method
public <E> List<E> selectList(String statement, Object parameter, RowBounds rowBounds) {
MappedStatement ms = configuration.getMappedStatement(statement); // 取出MappedStatement
List<E> result = executor.query(ms, wrapCollection(parameter), rowBounds, Executor.NO_RESULT_HANDLER);
return result;
}Executor's query method
public <E> List<E> query(MappedStatement ms, Object parameter, RowBounds rowBounds, ResultHandler resultHandler) throws SQLException {
BoundSql boundSql = ms.getBoundSql(parameter); // 解析最终预处理语句+ParameterMapping
CacheKey key = createCacheKey(ms, parameter, rowBounds, boundSql);
return query(ms, parameter, rowBounds, resultHandler, key, boundSql);
}3. Dynamic sqlNode analysis
From the getBoundSql of SqlSource, you can see that the top sqlnode will execute apply, and each element node is composed of MixedNode. This will call the apply method of each sqlNode from top to bottom. The parameter DynamicContext of the apply method is responsible for packaging the runtime sql Parameters, and is responsible for splicing SQL statements, and provides some extensions of ONGL. Let's look at the apply method of the next part of sqlNode.
DynamicContext 拼接sql的方法
public void appendSql(String sql) {
sqlBuilder.append(sql);
sqlBuilder.append(" ");
}
public String getSql() {
return sqlBuilder.toString().trim();
}
组合节点，内部调用每个组合节点apply
public class MixedSqlNode implements SqlNode {
private List<SqlNode> contents;
public MixedSqlNode(List<SqlNode> contents) {
this.contents = contents;
}
public boolean apply(DynamicContext context) {
for (SqlNode sqlNode : contents) {
sqlNode.apply(context);
}
return true;
}
}文本节点(含${})
public class TextSqlNode implements SqlNode {
private String text;
public TextSqlNode(String text) {
this.text = text;
}
public boolean apply(DynamicContext context) {
// 处理${}
GenericTokenParser parser = createParser(new BindingTokenParser(context));
// 拼接sql
context.appendSql(parser.parse(text));
return true;
}
// 处理${}
private GenericTokenParser createParser(TokenHandler handler) {
return new GenericTokenParser("${", "}", handler);
}
// handler方法实现
private static class BindingTokenParser implements TokenHandler {
private DynamicContext context;
public BindingTokenParser(DynamicContext context) {
this.context = context;
}
public String handleToken(String content) {
// 从map的_parameter属性取出外部参数
Object parameter = context.getBindings().get("_parameter");
if (parameter == null) {
context.getBindings().put("value", null);
} else if (SimpleTypeRegistry.isSimpleType(parameter.getClass())) {
// 设置简单类型的参数到value
context.getBindings().put("value", parameter);
}
// 用ognl取值
Object value = OgnlCache.getValue(content, context.getBindings());
return (value == null ? "" : String.valueOf(value)); // issue #274 return "" instead of "null"
}
}if节点
public class IfSqlNode implements SqlNode {
private ExpressionEvaluator evaluator;
private String test;
private SqlNode contents;
public IfSqlNode(SqlNode contents, String test) {
this.test = test;
this.contents = contents;
this.evaluator = new ExpressionEvaluator();
}
public boolean apply(DynamicContext context) {
// ognl取test的表达式计算结果
if (evaluator.evaluateBoolean(test, context.getBindings())) {
// 此if节点的内部MixedNode全部applye
contents.apply(context);
return true;
}
return false;
}
}
纯静态节点StaticTextSqlNode
public class StaticTextSqlNode implements SqlNode {
private String text;
public StaticTextSqlNode(String text) {
this.text = text;
}
public boolean apply(DynamicContext context) {
// 直接拼接sql
context.appendSql(text);
return true;
}
}(A basic test case will be added after the use of OGNL)
4. The concrete realization of parsing parameters
The getBoundSql method of dynamic sqlsource has been introduced above. We are concerned about how parameters are parsed into ParameterMapping at runtime.
// getBoundSql method fragment
SqlSource sqlSource = sqlSourceParser.parse(context.getSql(), parameterType, context.getBindings());// parse#{}
public SqlSource parse(String originalSql, Class<?> parameterType, Map<String, Object> additionalParameters) {
ParameterMappingTokenHandler handler = new ParameterMappingTokenHandler(configuration, parameterType, additionalParameters);
GenericTokenParser parser = new GenericTokenParser("#{", "}", handler);
String sql = parser.parse(originalSql);
return new StaticSqlSource(configuration, sql, handler.getParameterMappings());
}// GenericTokenParser has been introduced before, it is to call the appropriate processor processing according to the placeholder, we directly look at the processor implementation.
public String handleToken(String content) {
parameterMappings.add(buildParameterMapping(content)); // 每找到一个#{}创建一个parameterMapping
return "?";
}
private ParameterMapping buildParameterMapping(String content) {
Map<String, String> propertiesMap = parseParameterMapping(content); // 创建一个map
String property = propertiesMap.get("property");
Class<?> propertyType;
if (metaParameters.hasGetter(property)) { // issue #448 get type from additional params
propertyType = metaParameters.getGetterType(property);
} else if (typeHandlerRegistry.hasTypeHandler(parameterType)) {
propertyType = parameterType; // 存在注册类型处理器的类型直接赋值给propertyType
} else if (JdbcType.CURSOR.name().equals(propertiesMap.get("jdbcType"))) {
propertyType = java.sql.ResultSet.class;
} else if (property != null) { // 不存在的类型，但是值不为空
MetaClass metaClass = MetaClass.forClass(parameterType); // 反射class
if (metaClass.hasGetter(property)) { // 存在get方法
propertyType = metaClass.getGetterType(property); // 设置类型为get方法的类型
} else {
propertyType = Object.class; // 其他情况就是Object的类型
}
} else {
propertyType = Object.class;
}
拿到参数值和参数类型，下面就是构建ParameterMapping的过程。
ParameterMapping.Builder builder = new ParameterMapping.Builder(configuration, property, propertyType);
// .....
return builder.build();
}
to sum up
The dynamic SQL implementation of mybaits is divided into when it is created, the SQL fragments are packaged into individual SqlNode nodes. If this SqlNode node is parsed by the element node, then it is a combined SqlNode, otherwise it is a single SqlNode, and the SqlNode will be packaged into DynamicSqlSource If it is run time, the apply method of the top node will be called, that is, all nodes will execute the apply method, and finally use Ognl to parse the actual parameters of these dynamic points, and there is also a process of splicing sql in this process. After splicing all After the sql node, the process of #{} is finally performed, ParameterMapping is parsed, and #{} is replaced with ?, and the prepared statement and ParameterMapping are finally packaged into BoundSql for subsequent use. '
Supplement: Basic usage test of Ognl.
/**
* Ognl表达式测试
*
* 概述.Ognl表达式语言
*
* 从语言角度来说：它是一个功能强大的表达式语言，用来获取和设置 java 对象的属性 ，
* 它旨在提供一个更高抽象度语法来对 java 对象图进行导航。另外，java 中很多可以
* 做的事情，也可以使用 OGNL 来完成，例如：列表映射和选择。对于开发者来说，使用
* OGNL，可以用简洁的语法来完成对 java 对象的导航。通常来说：通过一个“路径”来完
* 成对象信息的导航，这个“路径”可以是到 java bean 的某个属性，或者集合中的某个索
* 引的对象，等等，而不是直接使用 get 或者 set 方法来完成。
*
* 首先来介绍下OGNL的三要素：
*
* 1、表达式：表达式（Expression）是整个OGNL的核心内容，所有的OGNL操作都是针对表达式解析
* 后进行的。通过表达式来告诉OGNL操作到底要干些什么。因此，表达式其实是一个带有语法含义
* 的字符串，整个字符串将规定操作的类型和内容。OGNL表达式支持大量的表达式，如“链式访问对象”、表达式计算s
*
* 2、Root对象：
* OGNL的Root对象可以理解为OGNL的操作对象。当我们指定了一个表达式的时候，我们需要指定这个表达式针对的是
* 哪个具体的对象。而这个具体的对象就是Root对象，这就意味着，如果有一个OGNL表达式，那么我们需要针对Root
* 对象来进行OGNL表达式的计算并且返回结果。
*
* 3、上下文环境
* 有个Root对象和表达式，我们就可以使用OGNL进行简单的操作了，如对Root对象的赋值与取值操作。但是，实际上在
* OGNL的内部，所有的操作都会在一个特定的数据环境中运行。这个数据环境就是上下文环境（Context）。OGNL的上下
* 文环境是一个Map结构，称之为OgnlContext。Root对象也会被添加到上下文环境当中去。
*
*/
public class LearnOgnl {
/**
* 1、访问root对象, 使用Api是Ognl.getValue(express, root)
*/
@Test
public void test1() {
Person person = new Person("id", "name");
MetaObject meta = MetaObject.forObject(person, new DefaultObjectFactory(), new DefaultObjectWrapperFactory());
meta.setValue("child.name", "haha"); // 简化构建对象操作，用MetaObject创建子对象属性。
try {
System.out.println(Ognl.getValue("name", person)); // name
System.out.println(Ognl.getValue("child", person)); // Person [id=null, name=haha, children=[], child=null]
System.out.println(Ognl.getValue("child.name", person)); // haha
} catch (OgnlException e) {
e.printStackTrace();
}
}
/**
* 2、上下文对象访问，上下文对象访问需要用#获取，使用Api是Ognl.getValue(express, context, root)
*/
@Test
public void test2() throws OgnlException {
Person person = new Person("id", "name");
MetaObject meta = MetaObject.forObject(person, new DefaultObjectFactory(), new DefaultObjectWrapperFactory());
meta.setValue("child.name", "haha");
Map<String, Object> context = new HashMap<String, Object>();
context.put("person", person);
System.out.println(Ognl.getValue("#person.id", context, person)); // id
System.out.println(Ognl.getValue("#person.name", context, person)); // name
System.out.println(Ognl.getValue("name", context, person)); // name
}
/**
* 3、方法调用, ognl可以调用方法，并且传入上下文参数。
*/
@Test
public void test3() throws OgnlException {
Person person = new Person();
Map<String, Object> context = new HashMap<String, Object>();
context.put("name", "张三");
Ognl.getValue("setName(#name)", context, person);
System.out.println(Ognl.getValue("getName()", context, person)); // 张三
}
/**
* 4、数组集合的访问 ，从此案例可以看出对一些运算符能够支持，
* 运算符常见支持如下：
* 减法、乘法、除法、取余 (2+5、6/2、6%2)
* 字符串相加 ( "str1" + "str2")
* 逻辑判断 (i == j)
*/
@Test
public void test4() {
Person person = new Person();
Map<String, Object> context = new HashMap<String, Object>();
String[] strings = { "aa", "bb" };
ArrayList<String> list = new ArrayList<String>();
list.add("aa");
list.add("bb");
Map<String, String> map = new HashMap<String, String>();
map.put("key1", "value1");
map.put("key2", "value2");
context.put("list", list);
context.put("strings", strings);
context.put("map", map);
try {
System.out.println(Ognl.getValue("#strings[0]", context, person));
System.out.println(Ognl.getValue("#list[0]", context, person));
System.out.println(Ognl.getValue("#list[0 + 1]", context, person));
System.out.println(Ognl.getValue("#map['key1']", context, person));
System.out.println(Ognl.getValue("#map['key' + '2']", context, person));
} catch (OgnlException e) {
e.printStackTrace();
}
}
/**
* 5、演示mybaits解析test表达的表达式
*/
@Test
public void test() throws OgnlException {
String express = "begin!=null and end!=null";
Map<String, String> map = new HashMap<String, String>();
map.put("begin", null);
map.put("end", null);
Object ok = Ognl.getValue(express, map, map);
System.out.println(ok); // false
map.put("begin", "");
map.put("end", "");
ok = Ognl.getValue(express, map, map);
System.out.println(ok); // true
}
}
class Person{
private String id;
private String name;
private List<Person> children = new ArrayList<Person>();
private Person child;
public Person(String id, String name) {
this.id = id;
this.name = name;
}
public Person() {
super();
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public List<Person> getChildren() {
return children;
}
public void setChildren(List<Person> children) {
this.children = children;
}
public Person getChild() {
return child;
}
public void setChild(Person child) {
this.child = child;
}
@Override
public String toString() {
return "Person [id=" + id + ", name=" + name + ", children=" + children + ", child=" + child + "]";
}
}
end ！