nodejs how simple and elegant access to mysql database

I. Background problem

Since the birth of nodejs appeared a large number of web frameworks such as express koa2 egg and so on, the front end can no longer rely on the back-end can control their own server-side logic. The original position of the rear end of the front end of the development of the students now also write fast, roll up its sleeves and is doing a project a few weeks front-end, back-end all get their own, and what a efficiency.

Although various frameworks provide some of their own interfaces to simplify CRUD operations, but still does not solve the complex conditions of a query, the server paging and so on, resulting in the development process, many developers or directly spliced ​​SQL to access the database. So we wanted to how to access the database easier to use.

Second, the library design

Database operations can be seen as doing one interaction with the database, interactive data transfer is + command, we want to try to use simple code to describe each access to the database. It is possible to encapsulate mysqljs database operations, redesigned API to access a database. Provide powerful and smooth access to nodejs api mysql database tools library, goal is to access the database logic can be used to complete a line of code, make it easier to access the database and elegant. Open Source Address: https://github.com/liuhuisheng/ali-mysql-client

1. Initial Configuration

Initialized as follows

const db = new DbClient({
  host     : '127.0.0.1',
  user     : 'root',
  password : 'secret',
  database : 'my_db'
});

2. construct a query

• 2.1 Query single value

// 查询单个值，比如下面例子返回的是数字51，满足条件的数据条数
var result = await db
  .select("count(1)")
  .from("page")
  .where("name", "测试", "like")
  .queryValue();

• 2.2 single query data

// 查询单条数据，返回的是 result = {id:12, name: '测试页面', ....}
const result = await db
  .select("*")
  .from("page")
  .where("id", 12) // id = 12
  .queryRow();

• 2.3 query multiple data

// 查询多条数据 返回的是 ressult = [{...}, {...}];
const result = await db
  .select("*")
  .from("page")
  .where("name", "测试页面", 'like') // name like '%测试页面%'
  .queryList();

• 2.4 server-side paging query

// 查询多条数据（服务端分页） 返回的是 ressult = {total: 100, rows:[{...}, {...}]};
const result = await db
  .select("*")
  .from("page")
  .where("id", 100, "lt") // id < 100
  .queryListWithPaging(3, 20); //每页 20 条，取第 3 页

• More than 250 table associated with the query

// 多表关联查询
var result = await db
  .select("a.page_id, a.saga_key")
  .from("page_edit_content as a")
  .join("left join page as b on b.id = a.page_id")
  .where("b.id", 172)
  .queryList();

• 2.6 In addition to supporting the various multi-table queries outer join, of course, also supports groupby orderby having other complex query

const result = await db
  .select("a1 as a, b1 as b, count(c) as count")
  .from("table")
  .where("date", db.literals.now, "lt") // date < now()
  .where("creator", "huisheng.lhs")  // creator = 'huisheng.lhs"
  .groupby("a1, b1")
  .having("count(category) > 10")
  .orderby("id desc")
  .queryListWithPaging(2); //默认每页20条，取第2页

3. insert structure

const task = {
  action: "testA",
  description: "desc1",
  state: "123",
  result: "result1"
};

// 插入一条数据
const result = await db
  .insert("task", task)
  .execute();

// 也支持直接写字段，支持增加字段
const result = await db
  .insert("task")
  .column("action", "test")
  .column("create_time", db.literals.now)
  .execute();

// 插入多条数据
const tasks = [ task1, taks2, task3 ];
const result = await db
  .insert("task", tasks)
  .execute();

// 支持增加或覆盖字段
const result = await db
  .insert("task", tasks)
  .column('create_time', db.literals.now)  // 循环赋值给每一行数据
  .column('create_user', 'huisheng.lhs')
  .execute();

4. Update configuration

const task = {
  action: "testA",
  description: "desc1",
  state: "123",
  result: "updateResult"
};

//更新数据
const result = await db
  .update("task", task)
  .where("id", 1)
  .execute();

//更新数据，支持增加字段
const result = await db
  .update("task")
  .column("action", "test-id22")
  .column("create_time", db.literals.now)
  .where('id', 2)
  .execute();

5. Delete construction

//删除id为1的数据
const result = await db
  .delete("task")
  .where("id", 1)
  .execute();

6. Transaction Control

const trans = await db.useTransaction();

try {
  // 数据库操作
  // await trans.insert(...)
  // await trans.update(...)
  await trans.commit();
} catch (e) {
  await trans.rollback();
}

7. complicated conditions of the query design

7.1 query all parameter descriptions

// 查询条件所有参数
const result = await db
  .where(field, value, operator, ignore, join) // 支持的所有参数
  .where({field, value, operator, ignore, join}) //支持对象参数
  .queryList();
  
// 复杂查询条件
const result = await db
  .select("*")
  .from("page")
  .where("id", 100, "gt") // id > 100
  .where("tags", "test", "like") //name like '%test%'
  .where("tech", tech, "eq", "ifHave") // tech='tech_value' 当 tech 为空时，不做为查询条件
  .where("tags", tags, "findinset", "ifHave", "or")
  .queryList();

• field field name
• value passed in value
• operator operator, default equal4
• Add to whether or not ignore the condition, the condition is ignored when it returns false
• join connection symbol (and or), and the default is

7.2 operator operating logic defined

This parameter is well understood, the default value is equal, or supports by the incoming string function, passing the string can be matched to the logic defined,

const result = await db
  .select("*")
  .from("page");
  .where("id", 100, "lt")  // id < 100
  .where("group_code", "dacu") // group_code = "dacu"
  .queryList();

We can understand the operator for the query logic package stitching used to expand the capacity of complex conditions can be customized by the operator to complete. Function has the following form:

const customOperator =  ({ field, value }) => {
  if (condition) {
    return {
      sql: '?? = ?',
      arg: [ field, value ],
    };
  } else {
    return {
      sql: '?? > ?',
      arg: [ field, value ],
    };
   }
};

// 可直接使用也可注册到全局
const config = db.config();
config.registerOperator("customOperator", customOperator);

7.3 Add to ignore whether the conditions

This need to explain the query when the condition is satisfied xx ignore, ignore originally designed to simplify the code, for example, the following code is common, there is an input value, the query interface, query conditions do not enter a value :

const query = db
  .select("*")
  .from("page");
  .where("id", 100, "lt");

if (name){
    query.where("name", name, 'like');
}

if (isNumber(source_id)){
    query.where('source_id', source_id)
}

const result = await query.queryList();

The above code can be simplified to ignore the use of:

const result = await db
  .select("*")
  .from("page")
  .where("id", 100, "lt")
  .where("name", name, "like", "ifHave") //使用内置 ifHave，如果name为非空值时才加为条件
  .where("source_id", tech, "eq", "ifNumber") //使用内置 ifNumber
  .queryList();

Pass the incoming string or support function, passing the string can be matched to the logic defined in the form of the following functions:

const customIgnore = ({field, value}) => {
    if (...){
        return false;
    }
    
    return true;
};

//也可以注册到全局使用
const config = db.config();
config.registerIgnore("customIgnore", customIgnore);

7.4 Priority support query

// where a = 1 and (b = 1 or c < 1) and d = 1
const result = await db.select('*')
  .from('table')
  .where('a', 1)
  .where([
    {field: 'b', value: '1', operator:'eq'},
    {field: 'c', value: '1', operator:'lt', join: 'or'},
  ])
  .where('d', 1)
  .queryList();

7.5 Examples of complex queries in real scene

// 复杂查询，真实场景示例，项目中拓展了keyword、setinset等operator及ignore
const result = await app.db
  .select('a.*, b.id as fav_id, c.name as biz_name, d.group_name')
  .from('rocms_page as a')
  .join(`left join favorite as b on b.object_id = a.id and b.object_type = "rocms_page" and b.create_user = "${this.ctx.user.userid}"`)
  .join('left join rocms_biz as c on c.biz = a.biz')
  .join('left join rocms_biz_group as d on d.biz = a.biz and d.group_code = a.biz_group')
  // 关键字模糊查询
  .where('a.name,a.biz,a.biz_group,a.support_clients,a.owner,a.status', query.keywords, 'keywords', 'ifHasValueNotNumber') // 关键字在这些字段中模糊查询
  .where('a.id', query.keywords, 'eq', 'ifNumber') // 关键字中输入了数字时当作id查询
  // 精确查询
  .where('a.id', query.id, 'eq', 'ifHave')
  .where('a.name', query.name, 'like', 'ifHave')
  .where('a.biz', query.biz, 'eq', 'ifHave')
  .where('a.biz_group', query.biz_group, 'eq', 'ifHave')
  .where('a.support_clients', query.support_clients, 'setinset', 'ifHave')
  .where('a.status', query.status, 'insetfind', 'ifHave')
  .where('a.owner', query.owner, 'eq', 'ifHave')
  .where('a.offline_time', query.owner, 'eq', 'ifHave')
  // TAB类型 我的页面own、我的收藏fav、所有页面all
  .where('a.owner', this.ctx.user.userid, 'eq', () => query.queryType === 'own')
  .where('b.id', 0, 'isnotnull', () => query.queryType === 'fav')
  // 分页查询
  .orderby('a.update_time desc, a.id desc')
  .queryListWithPaging(query.pageIndex, query.pageSize);

4. custom configuration

const config = db.config();

// 自定义operator
config.registerOperator('ne', ({ field, value }) => {
  return { sql: '?? <> ?', arg: [ field, value ] };
});

// 自定义ignore
config.registerIgnore('ifNumber', ({ value }) => {
  return !isNaN(Number(value));
});

// 监听事件 执行前
config.onBeforeExecute(function({ sql }) {
  console.log(sql);
});

// 监听事件 执行后
config.onAfterExecute(function({ sql, result }) {
  console.log(result);
});

// 监听事件 执行出错
config.onExecuteError(function({ sql, error }) {
  console.log(error);
});

The built-in operator and ignore

• Built-in default operator
  • eq (equal)
  • ne (not equal)
  • in (in)
  • gt (greater than)
  • ge (greater than or equal)
  • lt (less than)
  • le (less than or equal）
  • isnull (is null)
  • isnotnull (is not null)
  • like (like)
  • startwith (start with)
  • endwith (end with)
  • between (between)
  • findinset (find_in_set(value, field))
  • insetfind (find_in_set(field, value))
  • sql (custom sql)
  • keywords (keywords query)
• Built-in default ignore
  • ifHave (if the value is added to the condition)
  • ifNumber (if the value is added to the condition)

Third, using the example

• demo-egg
• demo-koa
• demo-express

And it can be used together in the egg-mysql eggjs, the support directly into the egg-mysql or ali-rds object is initialized, to avoid duplication create a connection pool.

// this.app.mysql 为egg-mysql对象
const db = new DbClient(this.app.mysql)

Fourth, the open source project

The project was initially released only the network, including within their own projects, in addition to the recent consolidation within the network to rely on open source to GitHub, Ali-MySQL-Client: https://github.com/liuhuisheng/ali-mysql-client

The library is designed to provide powerful and smooth access to nodejs mysql database api, the goal is to access the database logic can be used to complete a line of code, make it easier to access the database and elegant. Welcome any comments or suggestions you can give me feedback at any time.