1. Installation and deployment
1. The version 3.0.7.1 I am using here is because I see that version 3.x has been released for a year and has added many new features, and 3.x is officially recommended. For version 2.x, the official website has recommended upgrading to 3.x, so considering the future development of the project, we decided to use version 3.x
2. Download the installation package. This is a Linux installation. Although the 3.x version can already support window installation, in order to consider subsequent performance testing and later deployment, Linux is still used here. Here are the installation steps, as officially stated. It’s very clear, so I won’t be verbose anymore.
3. After the installation is completed, the corresponding service will be installed into the system service. We can handle it directly through systemctl start xxx, official website document
4. After the startup is successful, we check the service status. After it is indeed successful,
5. In the bin directory of our current user, there are many tdengine executable files.
2. Remote link
1. After linking, there are many ways to link.
2. TDengine provides two ways for connectors to establish connections:
- Establish a connection with taosd through the REST API provided by the taosAdapter component. This connection method is hereinafter referred to as "REST connection"
- Establish a connection directly with the server program taosd through the client driver taosc. This connection method is hereinafter referred to as "native connection".
3. One thing that needs to be explained here is that if we use REST 连接, then the taosAdapter must be started on the server side. If you do not start it with docker, you have to start the service manually. Otherwise you won't be able to useREST 连接
As far as my current testing is concerned, only docker will help you start taosAdapter after it is created. Other windows and linux do not have it. We need to start it manually.
4. Start/stop taosAdapter
On Linux systems the taosAdapter service is managed by systemd by default. Use the command systemctl start taosadapter to start the taosAdapter service. Use the command systemctl stop taosadapter to stop the taosAdapter service.
2.1. Use DBeaver link
1. Download and install the latest DBeaver
2. Click on the upper left corner to create a link, and then select TDengine.
3. Fill in the username and password
Note that here you can see that when using the tool link, the URL is TAOS-RS, which means that our link method is "REST connection", so the server-side taosAdapter must be started.
4. The driver link version here is 3.2.1
5. Link successful
The mp_test and bfh databases were created by me
2.2. Use the official website client TDengineGUI link
1. tdengine also has a client for display, but the experience is not very good.
2. Click the download link https://github.com/arielyang/TDengineGUI/tags to download the zip, unzip and install it
3. Link, please refer to dbeaver for the method.
Note that the URL that can be used here is TAOS-RS, which means that our link method is "REST connection", so the server-side taosAdapter must be started.
3. springBoot use
1. The links in the following projects are all native connections
Establish a connection directly with the server program taosd through the client driver taosc. This connection method is hereinafter referred to as "native connection".
2. You can also use the RS method, but that method is 30% less efficient than the native one.
3. Just change the parameter url: jdbc:TAOS://192.168.172.129:6030/mp_test?charset=UTF-8&locale=en_US.UTF-8&timezone=UTC-8 to "jdbc:TAOS-RS://192.168.172.129:6030/mp_test?charset=UTF-8&locale=en_US.UTF-8&timezone=UTC-8";
3.1. How to use JDBC (prepareStatement)
1. Instead of completing the switching of multiple data sources throughdynamic-datasource-spring-boot-starter, you only need to simply configure the data source and database connection pool. There are two reasons why tdengine is not directly integrated into mybatisPlus here.
- When integrating into mybatisPlus, multiple data sources are required. Whether you use the dynamic-datasource solution or write it yourself, you need to process and make logical code. You may need to consider this maintenance when version maintenance is needed later.
- The second is that no matter how this tdengine develops, it will definitely support the jdbc solution and provide the latest demo, but other examples may be very slow, and may not even support other versions of mybatis and springBoot.
- Efficiency issues, the official JDBC website mentioned in the section on improving efficiency that we need to insert in batches, and gave examples of related JDBC, but if we use mybatisPlus, we need to take a look at the source code to see if mybatisPlus does any extra operations. , increasing our business workload.
Of course, this is just my personal opinion, so if you are unwilling to use this solution, you can jump directly to Section 3.2 and use
dynamic-datasource-spring-boot-starterto manage multiple data sources through mybatiPlus
3.1.1. Project configuration and its code
The project is very simple, you will understand after reading it
1. The entire project structure
2. pom dependency
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.14</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.28</version>
</dependency>
<!-- mybatis plus 依赖 -->
<dependency>
<groupId>com.baomidou</groupId>
<artifactId>mybatis-plus-boot-starter</artifactId>
<version>3.5.1</version>
</dependency>
<!-- lombok插件 -->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<!-- alibaba的fastjson -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.83</version>
</dependency>
<!-- hutool工具 -->
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.8.5</version>
</dependency>
<!-- TDengine 依赖 -->
<dependency>
<groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId>
<version>3.2.4</version>
</dependency>
<!--Druid 方便控制台查看-->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid-spring-boot-starter</artifactId>
<version>1.2.16</version>
</dependency>
</dependencies>
3. Configuration file
# 项目端口
server:
port: 7280
servlet:
# 项目路径
context-path: /tadt
shutdown: graceful #开启优雅停机
spring:
application:
name: thermal-api-demonstration-tdengine
lifecycle:
timeout-per-shutdown-phase: 30s #设置缓冲时间 默认也是30s
# Mysql配置
datasource:
druid:
# 接下来（one,two）其实就都是自定义配置了，springBoot是识别不了的（当然你也可以另起其它行，到其它位置），我们需要将这些配置映射到对应的类上，springBoot不会帮我们做
one:
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/test01?useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=Asia/Shanghai&useSSL=true&characterEncoding=UTF-8
username: root
password: 123456
type: com.alibaba.druid.pool.DruidDataSource
name: mysqlDataSource # 在druid 内数据源的名称
# springboot2.0整合了hikari ,据说这是目前性能最好的java数据库连接池，但是 druid 有控制面板方便查看
# 手动配置数据源
validation-query: SELECT 1 FROM DUAL # 连接是否有效的查询语句
validation-query-timeout: 60000 # 连接是否有效的查询超时时间
# 建议 连接数 = ((核心数 * 2) + 有效磁盘数)
initial-size: 40 #初始化时建立物理连接的个数，初始化发生在显示调用 init 方法，或者第一次 getConnection 时
min-idle: 40 # 最小连接池数量
max-active: 100 #最大连接池数量
test-on-borrow: false #申请连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
test-on-return: false #归还连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
time-between-eviction-runs-millis: 60000 # 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位是毫秒
min-evictable-idle-time-millis: 300000 # 配置一个连接在池中最小生存的时间，单位是毫秒
two:
driver-class-name: com.taosdata.jdbc.TSDBDriver
# 这里指定了具体的数据库 需要注意，
# 如果换成不指定具体数据库名称 jdbc:TAOS://192.168.172.129:6030?charset=UTF-8&locale=en_US.UTF-8&timezone=UTC-8 则在sql中使用必须要指定数据库的名称 dba.table_b
url: jdbc:TAOS://192.168.172.129:6030/mp_test?charset=UTF-8&locale=en_US.UTF-8&timezone=UTC-8
username: root
password: taosdata
type: com.alibaba.druid.pool.DruidDataSource
name: tdengineDataSource # 在druid 内数据源的名称
# springboot2.0整合了hikari ,据说这是目前性能最好的java数据库连接池，但是 druid 有控制面板方便查看
# 手动配置数据源
validation-query: select server_status() # 连接是否有效的查询语句
validation-query-timeout: 60000 # 连接是否有效的查询超时时间
# 建议 连接数 = ((核心数 * 2) + 有效磁盘数)
initial-size: 10 #初始化时建立物理连接的个数，初始化发生在显示调用 init 方法，或者第一次 getConnection 时
min-idle: 10 # 最小连接池数量
max-active: 20 #最大连接池数量
test-on-borrow: false #申请连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
test-on-return: false #归还连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
time-between-eviction-runs-millis: 60000 # 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位是毫秒
min-evictable-idle-time-millis: 300000 # 配置一个连接在池中最小生存的时间，单位是毫秒
# 热部署
devtools:
restart:
enabled: true
# jackson 配置
jackson:
time-zone: GMT+8
date-format: yyyy-MM-dd HH:mm:ss
# 上传文件
servlet:
multipart:
max-file-size: 50MB
max-request-size: 50MB
logstash:
host: 192.168.172.228 # logstash部署的服务器IP
env: dev # 将日志加载到elk中项目的环境后缀名称-时间
# mybatis配置
mybatis-plus:
# xml文件路径
mapper-locations: classpath:mapper/*/*.xml
# 实体类路径
type-aliases-package: com.asurplus.*.entity
configuration:
# 驼峰转换
map-underscore-to-camel-case: true
# 是否开启缓存
cache-enabled: false
# 打印sql
log-impl: org.apache.ibatis.logging.stdout.StdOutImpl
# 全局配置
global-config:
# 数据库字段驼峰下划线转换
db-column-underline: true
# id自增类型(数据库id自增)
id-type: 0
4、DataSourceConfig
package cn.jt.thermalapidemonstrationtdengine.config;
import com.alibaba.druid.pool.DruidDataSource;
import com.alibaba.druid.support.http.StatViewServlet;
import com.alibaba.druid.support.http.WebStatFilter;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.web.servlet.FilterRegistrationBean;
import org.springframework.boot.web.servlet.ServletRegistrationBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Primary;
import org.springframework.stereotype.Component;
import javax.servlet.Filter;
import javax.sql.DataSource;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年08月01日
*/
@Component
@Slf4j
public class DataSourceConfig {
public static final String MYSQL_DATA_SOURCE = "mysqlDataSource";
public static final String TDENGINE_DATA_SOURCE = "tdengineDataSource";
/**
* http://127.0.0.1:7280/tadt/druid
* 配置Druid的监控视图
*
* @return
*/
@Bean
public ServletRegistrationBean<StatViewServlet> druidStatViewServlet() {
ServletRegistrationBean<StatViewServlet> registrationBean =
new ServletRegistrationBean<>(new StatViewServlet(), "/druid/*");
// 配置Druid监控页面的登录用户名和密码
registrationBean.addInitParameter("loginUsername", "admin");
registrationBean.addInitParameter("loginPassword", "123456");
// 设置 IP 白名单，允许访问的 IP，多个 IP 用逗号分隔
registrationBean.addInitParameter("allow", "127.0.0.1");
// 设置 IP 黑名单，拒绝访问的 IP，多个 IP 用逗号分隔（当 IP 在黑名单中同时又在白名单中时，优先于白名单）
// servletRegistrationBean.addInitParameter("deny", "192.168.1.100");
// 是否能够重置数据
registrationBean.addInitParameter("resetEnable", "false");
return registrationBean;
}
/**
* 配置Druid的WebStatFilter
*
* @return
*/
@Bean
public FilterRegistrationBean<Filter> druidWebStatFilter() {
FilterRegistrationBean<Filter> registrationBean = new FilterRegistrationBean<>();
registrationBean.setFilter(new WebStatFilter());
// 添加过滤规则
registrationBean.addUrlPatterns("/*");
// 配置不拦截的路径
registrationBean.addInitParameter("exclusions", "*.js,*.gif,*.jpg,*.png,*.css,*.ico,/druid/*");
return registrationBean;
}
/**
* 自动配置的数据源，使用Druid连接池
* druid 会管理这数据源
*
* @return
*/
@Bean(name = MYSQL_DATA_SOURCE)
@Primary
@ConfigurationProperties(prefix = "spring.datasource.druid.one")
public DataSource dataSource() {
return new DruidDataSource();
}
/**
* 手动配置的数据源，也使用Druid连接池
* druid 会管理这数据源
* <p>
* 这里不直接集成tdengine 到 mybatisPlus 的原因有两个
* 1、集成到 mybatisPlus 时需要多数据源，无论是使用dynamic-datasource的方案，还是自己去写，都需要去处理做逻辑代码，可能后期需要版本的维护就又要考虑这个维护
* 2、就是无论这个tdengine 如何发展，它一定会支持 jdbc 的方案，但是可能不会支持 mybatis和 springBoot，
* 3、效率问题
*
* @return
*/
@Bean(name = TDENGINE_DATA_SOURCE)
@ConfigurationProperties(prefix = "spring.datasource.druid.two")
public DataSource customDataSource() {
return new DruidDataSource();
}
}
5、TdEngineController
package cn.jt.thermalapidemonstrationtdengine.controller;
import cn.jt.thermalapidemonstrationtdengine.service.TdEngineService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.sql.Timestamp;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月31日
*/
@Slf4j
@RestController
@RequestMapping("/td")
public class TdEngineController {
@Autowired
private TdEngineService tdEngineService;
@GetMapping("/mockOne")
public String mockOne() {
tdEngineService.insertOneByTime(new Timestamp(System.currentTimeMillis()));
return "ok";
}
@GetMapping("/mockMany")
public String mockMany() {
tdEngineService.mockMany();
return "ok";
}
}
6、TestController
package cn.jt.thermalapidemonstrationtdengine.controller;
import cn.hutool.core.util.RandomUtil;
import cn.jt.thermalapidemonstrationtdengine.entity.A;
import cn.jt.thermalapidemonstrationtdengine.service.AService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@Slf4j
@RestController
@RequestMapping("/test")
public class TestController {
@Autowired
private AService aService;
@GetMapping("/mockMysql")
public String mockMysql() {
A a = new A();
a.setName(RandomUtil.randomString(5));
aService.save(a);
return "ok";
}
@Transactional(rollbackFor = Exception.class)
@GetMapping("/mockMysql2")
public String mockMysql2() {
A a = new A();
a.setName(RandomUtil.randomString(5));
aService.save(a);
int i = 1/0;
A a2 = new A();
a2.setName(RandomUtil.randomString(5));
aService.save(a2);
return "ok";
}
}
7、A
package cn.jt.thermalapidemonstrationtdengine.entity;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@TableName("a")
@AllArgsConstructor
@NoArgsConstructor
@Data
public class A {
@TableId(type = IdType.AUTO)
private Integer id;
private String name;
}
8、AMapper
package cn.jt.thermalapidemonstrationtdengine.mapper;
import cn.jt.thermalapidemonstrationtdengine.entity.A;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.apache.ibatis.annotations.Mapper;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@Mapper
public interface AMapper extends BaseMapper<A> {
}
9、TdEngineService
package cn.jt.thermalapidemonstrationtdengine.service;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月31日
*/
public interface TdEngineService {
/**
* 插入一条数据
*
* @param ts 时间
*/
void insertOneByTime(java.sql.Timestamp ts);
/**
* 模拟大量数据
*/
void mockMany();
}
10、TdEngineServiceImpl
package cn.gxm.stdmdd.service.impl;
import cn.gxm.stdmdd.config.DataSourceConfig;
import cn.gxm.stdmdd.service.TdEngineService;
import cn.hutool.core.date.DateUtil;
import cn.hutool.core.util.RandomUtil;
import com.google.common.util.concurrent.ThreadFactoryBuilder;
import com.taosdata.jdbc.TSDBPreparedStatement;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Service;
import javax.sql.DataSource;
import java.math.BigDecimal;
import java.sql.Connection;
import java.sql.SQLException;
import java.time.LocalDateTime;
import java.time.ZoneOffset;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.*;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月31日
*/
@Slf4j
@Service
public class TdEngineServiceImpl implements TdEngineService {
public static final ExecutorService MESSAGE_LISTENER_HANDLER_EXECUTOR =
new ThreadPoolExecutor(20, 20,
Integer.MAX_VALUE, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>(),
new ThreadFactoryBuilder().setNameFormat("message-listener-handler-executor-" + "%d")
.setUncaughtExceptionHandler((thread, throwable) -> log.error("ThreadPool {} got exception", thread, throwable))
.build(), new ThreadPoolExecutor.AbortPolicy());
@Autowired
@Qualifier(DataSourceConfig.TDENGINE_DATA_SOURCE)
private DataSource dataSource;
@Override
public void insertOneByTime(java.sql.Timestamp ts) {
BigDecimal min = BigDecimal.valueOf(0.00);
BigDecimal max = BigDecimal.valueOf(100.00);
try (Connection conn = dataSource.getConnection()) {
String psql = "INSERT INTO bfh.dn_bfh202203450556 VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)";
try (TSDBPreparedStatement pstmt = conn.prepareStatement(psql).unwrap(TSDBPreparedStatement.class)) {
// 如果是使用 parameterIndex 参数的方法，参数下标从1开始...
pstmt.setTimestamp(1, ts);
// 一共20个属性，20个通道
for (int i = 2; i <= 21; i++) {
float v = RandomUtil.randomBigDecimal(min, max).floatValue();
pstmt.setFloat(i, v);
}
int i = pstmt.executeUpdate();
log.info("插入成功：" + i);
}
} catch (SQLException e) {
log.error("插入error ", e);
}
}
@Override
public void mockMany() {
// 三亿条数据
int total = 300000000;
// int total = 8 * 10000;
// 前电脑CPU是8核的（给10个线程数量容易凑成整数）
int threadCount = 10;
// 每个线程需要处理的数量
int perThread = total / threadCount;
// 所有的任务列表
List<CompletableFuture<Void>> allTaskList = new ArrayList<>();
// 起始时间 2022-xx-01 15:30:50.123 ~
ArrayList<Long> startTimeList = new ArrayList<>(threadCount);
// 因为数据库设置了过期时间，是3650 即10年，所以插入的数据不能超过这个范围
for (int i = 0; i < threadCount; i++) {
LocalDateTime dateTime1 = LocalDateTime.of(2022, i + 1, 1, 15, 30, 50, 123_000_000);
// 转换为毫秒
startTimeList.add(dateTime1.toInstant(ZoneOffset.UTC).toEpochMilli());
}
String[] device_numbers = {
"dn_bfh202203450556", "dn_bfh202307291045"};
log.info("开始：{}", DateUtil.now());
for (int i = 0; i < threadCount; i++) {
int finalI = i;
CompletableFuture<Void> voidCompletableFuture = CompletableFuture.runAsync(() -> {
// 每个线程每次1000条 数据
int perSize = 1000;
int perCount = perThread / perSize;
int timeCount = 0;
try (Connection conn = dataSource.getConnection()) {
BigDecimal min = BigDecimal.valueOf(0.00);
BigDecimal max = BigDecimal.valueOf(100.00);
// INSERT INTO ? VALUES("2018-10-03 14:38:05.000",10.3, 10.3, 10.3, 10.3, 10.3, 10.3, 10.3,10.3,10.3, 10.3, 10.3, 10.3,10.3, 10.3, 10.3, 10.3, 10.3, 10.3, 10.3,10.3)
// 上述这样一个插入语句大约162B
// tdengine 目前一条sql限制最多1M 1M = 1024kb = 1024 * 1024 B
// 所以 约等于 1024 * 1024 / 162 = 6,472.6 所以目前 一次插入为 1000 条没有问题
String psql = "INSERT INTO ? VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)";
try (TSDBPreparedStatement pstmt = conn.prepareStatement(psql).unwrap(TSDBPreparedStatement.class)) {
for (int j = 1; j <= perCount; j++) {
Long curThreadStartTime = startTimeList.get(finalI) + (++timeCount);
pstmt.setTableName("bfh." + device_numbers[0]);
ArrayList<Long> tsList = new ArrayList<>();
for (int k = 0; k < perSize; k++, timeCount++) {
tsList.add(curThreadStartTime + k);
}
pstmt.setTimestamp(0, tsList);
// 一共20个属性，20个通道
for (int p = 1; p <= 20; p++) {
ArrayList<Float> fieldList = new ArrayList<>();
for (int p1 = 0; p1 < perSize; p1++) {
fieldList.add(RandomUtil.randomBigDecimal(min, max).floatValue());
}
pstmt.setFloat(p, fieldList);
}
// add column
pstmt.columnDataAddBatch();
// execute column
pstmt.columnDataExecuteBatch();
}
}
} catch (SQLException e) {
throw new RuntimeException(e);
}
});
allTaskList.add(voidCompletableFuture);
}
// 等待所有任务完成
CompletableFuture<Void> allTasks = CompletableFuture.allOf(allTaskList.toArray(new CompletableFuture[allTaskList.size()]));
// 阻塞等待所有任务完成
allTasks.join();
log.info("结束：{}", DateUtil.now());
}
}
11、AService
package cn.jt.thermalapidemonstrationtdengine.service;
import cn.jt.thermalapidemonstrationtdengine.entity.A;
import com.baomidou.mybatisplus.extension.service.IService;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
public interface AService extends IService<A> {
}
12、AServiceImpl
package cn.jt.thermalapidemonstrationtdengine.service.impl;
import cn.jt.thermalapidemonstrationtdengine.entity.A;
import cn.jt.thermalapidemonstrationtdengine.mapper.AMapper;
import cn.jt.thermalapidemonstrationtdengine.service.AService;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.springframework.stereotype.Service;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@Service
public class AServiceImpl extends ServiceImpl<AMapper, A> implements AService {
}
13、ThermalApiDemonstrationTdengineApplication
package cn.jt.thermalapidemonstrationtdengine;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
@MapperScan({
"cn.jt.thermalapidemonstrationtdengine.mapper"})
public class ThermalApiDemonstrationTdengineApplication {
public static void main(String[] args) {
SpringApplication.run(ThermalApiDemonstrationTdengineApplication.class, args);
}
}
14、A.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="cn.jt.thermalapidemonstrationtdengine.mapper.AMapper">
</mapper>
15. Druid data source display panelhttp://127.0.0.1:7280/tadt/druid/index.html, there are two data sources we named, with different configurations. If you do not display it, remember to use the two data separately first. Operate the source, for example, if Fengbie requests the MockOne interface, two data sources are used, and the following will begin to display.
3.1.2. Initial experience of the efficiency of TDengine (as a rough reference, detailed parameters have not been adjusted)
1. The official website also provides a taosBenchmark test tool, you can try it out
2. What I want to say here is that we write the code ourselves to experience it. Here we simulate the temperature data of a device. This device has 10 temperature records per second. Here we simulate its annual volume, 300 million records.
3. Of course, this is a preliminary experience. The server equipment (TDengine linux centos7.9) information is as follows
6. The window computer that generates data
7. Request the above interfacemockMany, 10 threads will be started, each thread has 30 million pieces of data, and 1,000 pieces of data will be inserted in batches each time. Here we use the suggestions from the official website
- Use bulk insert
- Use parameter binding
8. The test results here are: Start: 2023-08-02 09:02:10 ~ End: 2023-08-02 09:37:01. I also randomly operated MySQL, which took a total of 35 minutes.
- Note that the inserted time ts field cannot be repeated, otherwise data overwriting will be performed.
- Secondly, if the data is overwritten by a large amount of data with the same timestamp, resulting in a lot of fragmentation, just export the data and then import it (during the test phase, you can directly delete the library and table and re-establish it). The Enterprise Edition supports online reorganization. Otherwise the insertion will be very slow
- Although TDengine will handle the issue of out-of-order data insertion and put the out-of-order data into order, this will affect performance.
9. Supplement the structure of the database, super table, and sub-table here, as follows
-- 创建一个数据库 保存最近十年的数据
-- KEEP 表示数据文件保存的天数，缺省值为 3650，取值范围 [1, 365000]，且必须大于或等于 DURATION 参数值。
-- 数据库会自动删除保存时间超过 KEEP 值的数据。KEEP 可以使用加单位的表示形式，如 KEEP 100h、KEEP 10d 等，支持 m（分钟）、h（小时）和 d（天）三个单位。也可以不写单位，如 KEEP 50，此时默认单位为天
create database bfh KEEP 3650
use bfh
-- 10万个 设备就是 10万个 子表（一个设备一个子表），并且这个超级表下的都是同一种类型的采集量（比如温度）
-- 标签后续可以增加和修改,标签可以用于后续聚合，比如你增加一个 localtion的标签，后续你可以根据地理位置来聚合数据
-- 这里采用多列模型，因为这些通道的温度一定是同时采集的，如果有不同采集频率的可以使用单列模型，就是再建立一个超级表单独处理
-- dn 是 device_number 缩写
CREATE STABLE if not exists bfh_temperature
(
ts timestamp,
temperature1 float,
temperature2 float,
temperature3 float,
temperature4 float,
temperature5 float,
temperature6 float,
temperature7 float,
temperature8 float,
temperature9 float,
temperature10 float,
temperature11 float,
temperature12 float,
temperature13 float,
temperature14 float,
temperature15 float,
temperature16 float,
temperature17 float,
temperature18 float,
temperature19 float,
temperature20 float
) tags (
dn nchar(64)
)
-- 创建子表的时候，建议将设备唯一标识作为表名称，如设备序列号（就是出厂的时候，硬件那边肯定会有的）
-- dn 是 device_number 缩写,bfh202307291045是这台设备的型号
create table dn_bfh202307291045 using bfh_temperature tags('bfh202307291045')
create table dn_bfh202203450556 using bfh_temperature tags('bfh202203450556')
-- 时间顺序可以乱序，tdengine 会在插入的时候，帮你排好顺序，所以我们可以放心的使用多线程插入
insert into bfh.dn_bfh202307291045(ts,temperature1) VALUES ('2023-07-29 17:02:45.023',512.20)
--DELETE from bfh.dn_bfh202307291045
select count(*) from bfh.dn_bfh202307291045
--DROP DATABASE bfh;
select count(*) from bfh_temperature;
select count(*) from dn_bfh202203450556;
select * from dn_bfh202203450556 limit 1000000,10;
select * from dn_bfh202203450556 where ts>'2022-01-01 23:47:31.124' limit 1000000,10;
10. Query operations on database data
- select count(*) from dn_bfh202203450556; takes 91ms
- select count(*) from bfh_temperature; takes 75ms
- select * from dn_bfh202203450556 limit 1000000,10; takes 373ms
- select * from dn_bfh202203450556 where ts>‘2022-01-01 23:47:31.124’ limit 1000000,10; 耗时360ms
11. Save disk storage size, about 23G
12. CPU and memory usage
3.1.3. Off-topic explanation
1. Because we added the @Primary annotation to the Mysql data source, mybatisPlus manages this by default.
2. If you need to configure multiple other data sources later, you can configure three, four, and other data sources in the current way, and then configure jdbcTemplate, and then configure transactions for jdbcTemplate. We can use jdbcTemplate to operate the database later.
- In Spring Boot, JdbcTemplate is a simple JDBC tool provided by Spring for performing SQL operations. If you use multiple data sources in your application and want to configure the JdbcTemplate for each data source separately, you can do this as follows:
- Create multiple JdbcTemplate Beans in the configuration class and associate them with their respective data sources.
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.JdbcTemplate;
import javax.sql.DataSource;
@Configuration
public class JdbcTemplateConfig {
@Bean
public JdbcTemplate jdbcTemplateOne(@Qualifier("dataSourceOne") DataSource dataSourceOne) {
return new JdbcTemplate(dataSourceOne);
}
@Bean
public JdbcTemplate jdbcTemplateTwo(@Qualifier("dataSourceTwo") DataSource dataSourceTwo) {
return new JdbcTemplate(dataSourceTwo);
}
// 添加其他数据源的 JdbcTemplate Bean，根据需要继续添加
}
- In the Service or Repository class that needs to use JdbcTemplate, use the @Autowired annotation to inject the corresponding JdbcTemplate
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Repository;
@Repository
public class MyRepository {
private final JdbcTemplate jdbcTemplateOne;
private final JdbcTemplate jdbcTemplateTwo;
@Autowired
public MyRepository(@Qualifier("jdbcTemplateOne") JdbcTemplate jdbcTemplateOne,
@Qualifier("jdbcTemplateTwo") JdbcTemplate jdbcTemplateTwo) {
this.jdbcTemplateOne = jdbcTemplateOne;
this.jdbcTemplateTwo = jdbcTemplateTwo;
}
// 在这里可以使用 jdbcTemplateOne 和 jdbcTemplateTwo 分别对不同数据源进行数据库操作
// ...
}
- In Spring Boot, JdbcTemplate does not support transactions by default. If you need to enable transaction support in a method using JdbcTemplate, you can use Spring's transaction management mechanism, namely declarative transactions. Declarative transactions allow you to use annotations to mark methods that require transaction management, and Spring will automatically add transaction processing to these methods. During method execution, if an exception occurs, the transaction will be rolled back; if the method execution is successful, the transaction will be committed.
- Following are the steps to enable transaction support in JdbcTemplate methods: Configure transaction manager: First, you need to configure the transaction manager in the configuration class to enable transaction support in the application.
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.jdbc.DataSourceTransactionManagerAutoConfiguration;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import javax.sql.DataSource;
@Configuration
@EnableTransactionManagement
public class TransactionConfig {
@Bean
public PlatformTransactionManager transactionManager(@Qualifier("dynamicDataSource") DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
}
- In the above code, we create a transaction manager bean called transactionManager and pass the dynamicDataSource data source to it. dynamicDataSource is the dynamic data source we configured earlier, which contains multiple data sources. By adding the @EnableTransactionManagement annotation on the configuration class, we enable Spring's transaction management functionality.
- In the Service or Repository class that requires transaction management, use the @Transactional annotation to mark the methods that need to enable transactions.
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
@Service
public class MyService {
private final JdbcTemplate jdbcTemplate;
public MyService(JdbcTemplate jdbcTemplate) {
this.jdbcTemplate = jdbcTemplate;
}
@Transactional
public void someTransactionalMethod() {
// 在这个方法中执行数据库操作，该方法会启用事务
// 如果方法执行过程中出现异常，事务会回滚
// 如果方法执行成功，事务会提交
}
}
-
By adding the @Transactional annotation on the someTransactionalMethod() method, we mark the method as requiring transaction management. In this way, when calling this method, Spring will automatically add transaction support to the method to ensure the atomicity and consistency of database operations.
-
Note that in order for the @Transactional annotation to take effect, you must ensure that the annotation is used on a Spring-managed bean method. If you use this annotation on a non-public method in the same class, Spring will not be able to apply the transaction proxy.
-
With the above configuration, you can enable transaction support in methods using JdbcTemplate and ensure transaction safety for database operations.
3.2. Use mybatisPlus and dynamic-datasource-spring-boot-starter to manage
1. The project structure is as follows. The project is also very simple. You can understand it by looking at the code.
3.2.1. Project configuration and its code
1、pom.xml
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.14</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.26</version>
</dependency>
<!-- mybatis plus 依赖 -->
<dependency>
<groupId>com.baomidou</groupId>
<artifactId>mybatis-plus-boot-starter</artifactId>
<version>3.5.1</version>
</dependency>
<!-- mybatis plus 动态数据源 多数据源 -->
<dependency>
<groupId>com.baomidou</groupId>
<artifactId>dynamic-datasource-spring-boot-starter</artifactId>
<version>3.5.1</version>
</dependency>
<!--Druid 方便控制台查看-->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid-spring-boot-starter</artifactId>
<version>1.2.16</version>
</dependency>
<!-- lombok插件 -->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<!-- alibaba的fastjson -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.79</version>
</dependency>
<!-- hutool工具 -->
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.8.5</version>
</dependency>
<!-- TDengine 依赖 -->
<dependency>
<groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId>
<version>3.2.4</version>
</dependency>
</dependencies>
2、application.yml
server:
port: 7012
shutdown: graceful
spring:
lifecycle:
timeout-per-shutdown-phase: 30s
autoconfigure:
# DruidDataSourceAutoConfigure会注入一个DataSourceWrapper，其会在原生的spring.datasource下找url,
# username,password等。动态数据源URL等配置是在dynamic下，因此需要排除，否则会报错。
# 或者 在启动类上排除 @SpringBootApplication(exclude = DruidDataSourceAutoConfigure.class)
exclude: com.alibaba.druid.spring.boot.autoconfigure.DruidDataSourceAutoConfigure
# 配置数据源信息
datasource:
druid: # 参数说明 https://developer.aliyun.com/article/1157595
filters: stat,wall #监控统计(包含慢日志)用的filter:stat 日志用的filter:log4j 防御sql注入的filter:wall (这个必须是全局配置，所有数据源的方式是一样的，你放到具体的数据源下面配置，会发现启动报错)
web-stat-filter: # 不统计这些请求(这个必须是全局配置，所有数据源的方式是一样的，你放到具体的数据源下面配置，会发现没有这个参数)
exclusions: '*.js,*.gif,*.jpg,*.png,*.css,*.ico,/druid/*'
#设置访问druid监控页面的拦截路径及账号和密码
stat-view-servlet:
allow: 127.0.0.1 # 允许那些ip 访问
login-username: admin
login-password: 123456
enabled: true # 开启网页访问 http://127.0.0.1:7012/druid/sql.html
dynamic:
# 设置默认的数据源或者数据源组,默认值即为master
primary: test01
# 严格匹配数据源,默认false.true未匹配到指定数据源时抛异常,false使用默认数据源
strict: false
datasource:
test01:
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/test01?useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=Asia/Shanghai&useSSL=true&characterEncoding=UTF-8
username: root
password: 123456
druid: # 单独配置
validation-query: SELECT 1 FROM DUAL # 连接是否有效的查询语句
validation-query-timeout: 60000
# 建议 连接数 = ((核心数 * 2) + 有效磁盘数)
initial-size: 50 #初始化时建立物理连接的个数，初始化发生在显示调用 init 方法，或者第一次 getConnection 时
min-idle: 50 # 最小连接池数量
max-active: 100 #最大连接池数量
test-on-borrow: false #申请连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
test-on-return: false #归还连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
time-between-eviction-runs-millis: 60000 # 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位是毫秒
min-evictable-idle-time-millis: 300000 # 配置一个连接在池中最小生存的时间，单位是毫秒
test02:
url: jdbc:mysql://127.0.0.1:3306/test02?useUnicode=true&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=Asia/Shanghai&useSSL=true&characterEncoding=UTF-8
driver-class-name: com.mysql.cj.jdbc.Driver
username: root
password: 123456
druid: # 单独配置
validation-query: SELECT 1 FROM DUAL # 连接是否有效的查询语句
validation-query-timeout: 60000 # 连接是否有效的查询超时时间
# 建议 连接数 = ((核心数 * 2) + 有效磁盘数)
initial-size: 40 #初始化时建立物理连接的个数，初始化发生在显示调用 init 方法，或者第一次 getConnection 时
min-idle: 40 # 最小连接池数量
max-active: 100 #最大连接池数量
test-on-borrow: false #申请连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
test-on-return: false #归还连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
time-between-eviction-runs-millis: 60000 # 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位是毫秒
min-evictable-idle-time-millis: 300000 # 配置一个连接在池中最小生存的时间，单位是毫秒
thermalDemonstration: # 配置 tdengine 的数据源
driver-class-name: com.taosdata.jdbc.TSDBDriver
# 这里指定了具体的数据库 需要注意，
# 如果换成不指定具体数据库名称 jdbc:TAOS://192.168.172.129:6030?charset=UTF-8&locale=en_US.UTF-8&timezone=UTC-8 则在sql中使用必须要指定数据库的名称 dba.table_b
url: jdbc:TAOS://192.168.172.129:6030/mp_test?charset=UTF-8&locale=en_US.UTF-8&timezone=UTC-8
username: root
password: taosdata
druid: # 单独配置
validation-query: select server_status() # 连接是否有效的查询语句
validation-query-timeout: 60000 # 连接是否有效的查询超时时间
# 建议 连接数 = ((核心数 * 2) + 有效磁盘数)
initial-size: 10 #初始化时建立物理连接的个数，初始化发生在显示调用 init 方法，或者第一次 getConnection 时
min-idle: 10 # 最小连接池数量
max-active: 10 #最大连接池数量
test-on-borrow: false #申请连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
test-on-return: false #归还连接时会执行validationQuery检测连接是否有效,开启会降低性能,默认为true
time-between-eviction-runs-millis: 60000 # 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位是毫秒
min-evictable-idle-time-millis: 300000 # 配置一个连接在池中最小生存的时间，单位是毫秒
# mybatis配置
mybatis-plus:
# xml文件路径
mapper-locations: classpath:mapper/*/*.xml
# 实体类路径
type-aliases-package: cn.gxm.springboottdengine.entity.*
configuration:
# 驼峰转换
map-underscore-to-camel-case: true
# 是否开启缓存
cache-enabled: false
# 打印sql
log-impl: org.apache.ibatis.logging.stdout.StdOutImpl
# 全局配置
global-config:
# 数据库字段驼峰下划线转换
db-column-underline: true
# id自增类型(数据库id自增)
id-type: 0
3、TDengineTemperatureController
package cn.gxm.springboottdengine.controller;
import cn.gxm.springboottdengine.entity.thermaldemonstration.Temperature;
import cn.gxm.springboottdengine.mapper.thermaldemonstration.TemperatureMapper;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.sql.Timestamp;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Random;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@RestController
@RequestMapping("/td/temperature")
public class TDengineTemperatureController {
private static Random random = new Random(System.currentTimeMillis());
private static String[] locations = {
"北京", "上海", "深圳", "广州", "杭州"};
@Autowired
private TemperatureMapper mapper;
@GetMapping("before")
public String before() {
mapper.dropSuperTable();
// create table temperature
mapper.createSuperTable();
// create table t_X using temperature
for (int i = 0; i < 10; i++) {
mapper.createTable("t" + i, locations[random.nextInt(locations.length)], i);
}
// insert into table
int affectRows = 0;
// insert 10 tables
for (int i = 0; i < 10; i++) {
// each table insert 5 rows
for (int j = 0; j < 5; j++) {
Temperature one = new Temperature();
one.setTs(new Timestamp(1605024000000L));
one.setTemperature(random.nextFloat() * 50);
one.setLocation("望京");
one.setTbIndex(i);
affectRows += mapper.insertOne(one);
}
}
// Assert.assertEquals(50, affectRows);
return "影响数量 ： " + affectRows;
}
@GetMapping("after")
public String after() {
mapper.dropSuperTable();
return "删除超级表";
}
/***
* test SelectList
* **/
@GetMapping("testSelectList")
public List<Temperature> testSelectList() {
List<Temperature> temperatureList = mapper.selectList(null);
// temperatureList.forEach(System.out::println);
return temperatureList;
}
/***
* test InsertOne which is a custom metheod
* ***/
@GetMapping("testInsert")
public String testInsert() {
Temperature one = new Temperature();
one.setTs(new Timestamp(1605025000000L));
one.setTemperature(random.nextFloat() * 50);
one.setLocation("望京");
int affectRows = mapper.insertOne(one);
// Assert.assertEquals(1, affectRows);
return "插入行数：" + affectRows;
}
/***
* test select By map
* ***/
@GetMapping("testSelectByMap")
public List<Temperature> testSelectByMap() {
Map<String, Object> map = new HashMap<>();
map.put("location", "北京");
List<Temperature> temperatures = mapper.selectByMap(map);
// Assert.assertTrue(temperatures.size() > 1);
return temperatures;
}
/***
* test selectObjs
* **/
@GetMapping("testSelectObjs")
public List<Object> testSelectObjs() {
List<Object> ts = mapper.selectObjs(null);
// System.out.println(ts);
return ts;
}
/**
* test selectC ount
**/
@GetMapping("testSelectCount")
public long testSelectCount() {
long count = mapper.selectCount(null);
// Assert.assertEquals(10, count);
return count;
}
/****
* 分页
*/
@GetMapping("testSelectPage")
public IPage<Temperature> testSelectPage() {
IPage page = new Page(1, 2);
IPage<Temperature> temperatureIPage = mapper.selectPage(page, null);
System.out.println("total : " + temperatureIPage.getTotal());
System.out.println("pages : " + temperatureIPage.getPages());
for (Temperature temperature : temperatureIPage.getRecords()) {
System.out.println(temperature);
}
return temperatureIPage;
}
}
4、TDengineWeatherController
package cn.gxm.springboottdengine.controller;
import cn.gxm.springboottdengine.entity.thermaldemonstration.Weather;
import cn.gxm.springboottdengine.mapper.thermaldemonstration.WeatherMapper;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.sql.Timestamp;
import java.util.List;
import java.util.Random;
@RestController
@RequestMapping("/td/weather")
public class TDengineWeatherController {
private static Random random = new Random(System.currentTimeMillis());
@Autowired
private WeatherMapper mapper;
@GetMapping("createTable")
public void createTable() {
mapper.dropTable();
mapper.createTable();
Weather one = new Weather();
one.setTs(new Timestamp(1605024000000L));
one.setTemperature(12.22f);
one.setLocation("望京");
one.setHumidity(100);
mapper.insertOne(one);
}
@GetMapping("testSelectList")
public List<Weather> testSelectList() {
List<Weather> weathers = mapper.selectList(null);
// weathers.forEach(System.out::println);
return weathers;
}
@GetMapping("testInsert")
public int testInsert() {
Weather one = new Weather();
one.setTs(new Timestamp(1605024000000L));
one.setTemperature(random.nextFloat() * 50);
one.setHumidity(random.nextInt(100));
one.setLocation("望京");
int affectRows = mapper.insert(one);
// Assert.assertEquals(1, affectRows);
return affectRows;
}
// @Test
@GetMapping("testSelectOne")
public Weather testSelectOne() {
QueryWrapper<Weather> wrapper = new QueryWrapper<>();
wrapper.eq("location", "望京");
Weather one = mapper.selectOne(wrapper);
// System.out.println(one);
// Assert.assertEquals(12.22f, one.getTemperature(), 0.00f);
// Assert.assertEquals("望京", one.getLocation());
return one;
}
// @Test
// public void testSelectByMap() {
// Map<String, Object> map = new HashMap<>();
// map.put("location", "beijing");
// List<Weather> weathers = mapper.selectByMap(map);
// Assert.assertEquals(1, weathers.size());
// }
@GetMapping("testSelectObjs")
public List<Object> testSelectObjs() {
List<Object> ts = mapper.selectObjs(null);
// System.out.println(ts);
return ts;
}
@GetMapping("testSelectCount")
public long testSelectCount() {
long count = mapper.selectCount(null);
// Assert.assertEquals(5, count);
// System.out.println(count);
return count;
}
@GetMapping("testSelectPage")
public IPage<Weather> testSelectPage() {
IPage page = new Page(1, 2);
IPage<Weather> weatherIPage = mapper.selectPage(page, null);
// System.out.println("total : " + weatherIPage.getTotal());
// System.out.println("pages : " + weatherIPage.getPages());
// for (Weather weather : weatherIPage.getRecords()) {
// System.out.println(weather);
// }
return weatherIPage;
}
}
5、TestController
package cn.gxm.springboottdengine.controller;
import cn.gxm.springboottdengine.entity.test01.A;
import cn.gxm.springboottdengine.entity.test02.B;
import cn.gxm.springboottdengine.service.test01.AService;
import cn.gxm.springboottdengine.service.test02.BService;
import cn.hutool.core.util.RandomUtil;
import com.baomidou.dynamic.datasource.annotation.DSTransactional;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@RestController
@RequestMapping("/test")
public class TestController {
@Autowired
private AService aService;
@Autowired
private BService bService;
@GetMapping("/mockMysql")
// @Transactional(rollbackFor = Exception.class) 得用 @DSTransactional
public String mockMysql() {
A a = new A();
a.setName(RandomUtil.randomString(5));
aService.save(a);
B b = new B();
b.setAge(RandomUtil.randomInt(1, 200));
bService.save(b);
return "ok";
}
@GetMapping("/aTrans")
@DSTransactional
public String aTrans() {
A a = new A();
a.setName(RandomUtil.randomString(5));
aService.save(a);
int i = 1 / 0;
A a2 = new A();
a2.setName(RandomUtil.randomString(5));
aService.save(a2);
return "ok";
}
@GetMapping("/bTrans")
@DSTransactional
public String bTrans() {
B b = new B();
b.setAge(RandomUtil.randomInt(1, 200));
bService.save(b);
int i = 1 / 0;
B b2 = new B();
b2.setAge(RandomUtil.randomInt(1, 200));
bService.save(b2);
return "ok";
}
}
6、A
package cn.gxm.springboottdengine.entity.test01;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@TableName("a")
@AllArgsConstructor
@NoArgsConstructor
@Data
public class A {
@TableId(type = IdType.AUTO)
private Integer id;
private String name;
}
7、B
package cn.gxm.springboottdengine.entity.test02;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@TableName("b")
@AllArgsConstructor
@NoArgsConstructor
@Data
public class B {
@TableId(type = IdType.AUTO)
private Integer id;
private Integer age;
}
8、Temperature
package cn.gxm.springboottdengine.entity.thermaldemonstration;
import com.baomidou.mybatisplus.annotation.TableField;
import lombok.Data;
import java.sql.Timestamp;
@Data
public class Temperature {
private Timestamp ts;
private float temperature;
private String location;
@TableField("tbindex")
private int tbIndex;
}
9、Weather
package cn.gxm.springboottdengine.entity.thermaldemonstration;
import lombok.Data;
import java.sql.Timestamp;
@Data
public class Weather {
private Timestamp ts;
private float temperature;
private int humidity;
private String location;
}
10、AMapper
package cn.gxm.springboottdengine.mapper.test01;
import cn.gxm.springboottdengine.entity.test01.A;
import com.baomidou.dynamic.datasource.annotation.DS;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.apache.ibatis.annotations.Mapper;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@Mapper
@DS("test01")
public interface AMapper extends BaseMapper<A> {
}
11、BMapper
package cn.gxm.springboottdengine.mapper.test02;
import cn.gxm.springboottdengine.entity.test02.B;
import com.baomidou.dynamic.datasource.annotation.DS;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.apache.ibatis.annotations.Mapper;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@Mapper
@DS("test02")
public interface BMapper extends BaseMapper<B> {
}
12、TemperatureMapper
package cn.gxm.springboottdengine.mapper.thermaldemonstration;
import cn.gxm.springboottdengine.entity.thermaldemonstration.Temperature;
import com.baomidou.dynamic.datasource.annotation.DS;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.apache.ibatis.annotations.Insert;
import org.apache.ibatis.annotations.Mapper;
import org.apache.ibatis.annotations.Param;
import org.apache.ibatis.annotations.Update;
@Mapper
@DS("thermalDemonstration")
public interface TemperatureMapper extends BaseMapper<Temperature> {
// 演示放到 xml 中调用
// @Update("CREATE TABLE if not exists temperature(ts timestamp, temperature float) tags(location nchar(64), tbIndex int)")
int createSuperTable();
@Update("create table #{tbName} using temperature tags( #{location}, #{tbindex})")
int createTable(@Param("tbName") String tbName, @Param("location") String location, @Param("tbindex") int tbindex);
@Update("drop table if exists temperature")
void dropSuperTable();
@Insert("insert into t${tbIndex}(ts, temperature) values(#{ts}, #{temperature})")
int insertOne(Temperature one);
}
13、WeatherMapper
package cn.gxm.springboottdengine.mapper.thermaldemonstration;
import cn.gxm.springboottdengine.entity.thermaldemonstration.Weather;
import com.baomidou.dynamic.datasource.annotation.DS;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.apache.ibatis.annotations.Insert;
import org.apache.ibatis.annotations.Mapper;
import org.apache.ibatis.annotations.Update;
@Mapper
@DS("thermalDemonstration")
public interface WeatherMapper extends BaseMapper<Weather> {
// 演示放到 xml 中调用
// @Update("CREATE TABLE if not exists weather(ts timestamp, temperature float, humidity int, location nchar(100))")
int createTable();
@Insert("insert into weather (ts, temperature, humidity, location) values(#{ts}, #{temperature}, #{humidity}, #{location})")
int insertOne(Weather one);
@Update("drop table if exists weather")
void dropTable();
}
14、AServiceImpl
package cn.gxm.springboottdengine.service.test01.impl;
import cn.gxm.springboottdengine.entity.test01.A;
import cn.gxm.springboottdengine.mapper.test01.AMapper;
import cn.gxm.springboottdengine.service.test01.AService;
import com.baomidou.dynamic.datasource.annotation.DS;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.springframework.stereotype.Service;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@Service
public class AServiceImpl extends ServiceImpl<AMapper, A> implements AService {
}
15、AService
package cn.gxm.springboottdengine.service.test01;
import cn.gxm.springboottdengine.entity.test01.A;
import com.baomidou.mybatisplus.extension.service.IService;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
public interface AService extends IService<A> {
}
16、BServiceImpl
package cn.gxm.springboottdengine.service.test02.impl;
import cn.gxm.springboottdengine.entity.test02.B;
import cn.gxm.springboottdengine.mapper.test02.BMapper;
import cn.gxm.springboottdengine.service.test02.BService;
import com.baomidou.dynamic.datasource.annotation.DS;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.springframework.stereotype.Service;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
@Service
public class BServiceImpl extends ServiceImpl<BMapper, B> implements BService {
}
17、
package cn.gxm.springboottdengine.service.test02;
import cn.gxm.springboottdengine.entity.test02.B;
import com.baomidou.mybatisplus.extension.service.IService;
/**
* @author GXM
* @version 1.0.0
* @Description TODO
* @createTime 2023年07月28日
*/
public interface BService extends IService<B> {
}
18、SpringbootTdengineDynamicDataSourceApplication
package cn.gxm.springboottdengine;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
@SpringBootApplication(exclude = {
DataSourceAutoConfiguration.class})
@MapperScan({
"cn.gxm.springboottdengine.mapper"})
public class SpringbootTdengineDynamicDataSourceApplication {
public static void main(String[] args) {
SpringApplication.run(SpringbootTdengineDynamicDataSourceApplication.class, args);
}
}
19、A.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="cn.gxm.springboottdengine.mapper.test01.AMapper">
</mapper>
20、B.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="cn.gxm.springboottdengine.mapper.test02.BMapper">
</mapper>
21、TemperatureMapper.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="cn.gxm.springboottdengine.mapper.thermaldemonstration.TemperatureMapper">
<update id="createSuperTable">
CREATE TABLE if not exists temperature
(
ts
timestamp,
temperature
float
) tags
(
location nchar
(
64
), tbIndex int)
</update>
</mapper>
22、WeatherMapper.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="cn.gxm.springboottdengine.mapper.thermaldemonstration.WeatherMapper">
<update id="createTable">
CREATE TABLE if not exists weather
(
ts
timestamp,
temperature
float,
humidity
int,
location
nchar
(
100
))
</update>
</mapper>