Project Description
I found a JavaWeb project on the MOOC website. The content is about high-concurrency spikes. I thought it was very interesting, so I went in and learned it.
Document what you learned in this project..
The gitHub address corresponding to the source code of the project (written by the person watching its video, not the video source code): github.com/codingXiaxw…
I put together what I've learned from the project while combining its materials and watching the video...
Project Dao layer
Logging tool:
<!--1.日志 java日志有:slf4j,log4j,logback,common-logging
slf4j:是规范/接口
日志实现:log4j,logback,common-logging
使用:slf4j+logback
-->
Configuration properties that Mybatis didn't notice before:
Use jdbc's getGeneratekeys to get the self-incrementing primary key value , this property is quite useful.
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration
PUBLIC "-//mybatis.org//DTD Config 3.0//EN"
"http://mybatis.org/dtd/mybatis-3-config.dtd">
<configuration>
<!--配置全局属性-->
<settings>
<!--使用jdbc的getGeneratekeys获取自增主键值-->
<setting name="useGeneratedKeys" value="true"/>
<!--使用列别名替换列名  默认值为true
select name as title(实体中的属性名是title) form table;
开启后mybatis会自动帮我们把表中name的值赋到对应实体的title属性中
-->
<setting name="useColumnLabel" value="true"/>
<!--开启驼峰命名转换Table:create_time到 Entity(createTime)-->
<setting name="mapUnderscoreToCamelCase" value="true"/>
</settings>
</configuration>
If the object returned by Mybatis has associated fields, in addition to using resultMap, there are also the following methods (although I still think resultMap will be more convenient)
<select id="queryByIdWithSeckill" resultType="SuccessKilled">
<!--根据seckillId查询SuccessKilled对象，并携带Seckill对象-->
<!--如何告诉mybatis把结果映射到SuccessKill属性同时映射到Seckill属性-->
<!--可以自由控制SQL语句-->
SELECT
sk.seckill_id,
sk.user_phone,
sk.create_time,
sk.state,
s.seckill_id "seckill.seckill_id",
s.name "seckill.name",
s.number "seckill",
s.start_time "seckill.start_time",
s.end_time "seckill.end_time",
s.create_time "seckill.create_time"
FROM success_killed sk
INNER JOIN seckill s ON sk.seckill_id=s.seckill_id
WHERE sk.seckill_id=#{seckillId}
AND sk.user_phone=#{userPhone}
</select>
Properties that may be used by the database connection pool:
<!--c3p0私有属性-->
<property name="maxPoolSize" value="30"/>
<property name="minPoolSize" value="10"/>
<!--关闭连接后不自动commit-->
<property name="autoCommitOnClose" value="false"/>
<!--获取连接超时时间-->
<property name="checkoutTimeout" value="1000"/>
<!--当获取连接失败重试次数-->
<property name="acquireRetryAttempts" value="2"/>
Spring integrates with Junit:
/**
* Created by codingBoy on 16/11/27.
* 配置spring和junit整合，这样junit在启动时就会加载spring容器
*/
@RunWith(SpringJUnit4ClassRunner.class)
//告诉junit spring的配置文件
@ContextConfiguration({"classpath:spring/spring-dao.xml"})
public class SeckillDaoTest {
//注入Dao实现类依赖
@Resource
private SeckillDao seckillDao;
@Test
public void queryById() throws Exception {
long seckillId=1000;
Seckill seckill=seckillDao.queryById(seckillId);
System.out.println(seckill.getName());
System.out.println(seckill);
}
}
When Mybatis parameter is more than one
When I was learning MyBatis before, if there were more than one parameter, it was loaded using the Map collection!
Discovered in this tutorial, you can do without the Map collection (if they are all basic data types)!
Example: just use @Param!
int reduceNumber(@Param("seckillId") long seckillId, @Param("killTime") Date killTime);
The parameterType can be ignored directly in the XML file!
Avoid throwing exceptions when inserting data repeatedly
If the primary key is repeatedly inserted into data, Mybatis will normally throw an exception, and we don't want it to throw an exception, then we can do this:
Write ignore..
Service layer
everything
A dto package is used as the transport layer. The difference between dto and entity is that entity is used to encapsulate business data, while dto is used to complete data transfer between web and service layers.
For the concept of dto, I have been exposed to it once before, but I have not put it into practice. This time I saw its usage.
My understanding is: Service and Web layer data transfer data by wrapping an object . Because many times the data returned by the Service layer is POJO, many properties of POJO are redundant, and some desired data cannot be included. At this point, dto can once again abstract the data to be transmitted and encapsulate the data to be acquired .
Define multiple exception objects
The previous exception objects were for the entire business, but in fact, multiple exception classes can be subdivided . For example, "repeated seckill" and "seckill shutdown" are all seckill businesses.
The advantage of this is that you can see what is wrong by seeing the exception thrown.
For many exceptions caught in the service layer in the video, I think it can be thrown directly in the service layer, and can also be thrown in the controller. It is more convenient to use the unified exception handler class to manage it directly!
Promote the use of annotations to use transactions
I think the code is more clear, if you use annotations.
There are students below the video who said that if the transaction method is called in the Service, there will be some pitfalls, and I have not encountered it yet. Save it first:
Concurrency does not go up because when multiple threads access a row of data at the same time, a transaction is generated, so a write lock is generated. Whenever a thread that acquires a transaction releases the lock, another queuing thread can get the write lock. QPS and The transaction execution time is closely related. The shorter the transaction execution time, the higher the concurrency, which is also the reason for moving the time-consuming I/O operations out of the transaction.
There is a pit when calling transaction methods in the same class. Students need to pay attention that AOP cannot call transaction methods. The transaction will not take effect. There are several solutions. You can search and find a solution that suits you. The essential problem is that AOP will not use the proxy to call the internal method when the class is called internally.
"There is a pit when calling transaction methods in the same class" Solution 1. If it is an interface-based dynamic proxy, there is no problem, just use the interface to call directly. 2. If it is a class-based dynamic proxy, you can use AopContext.currentProxy() to solve it, Note that the stripping method must be public modified! !
MD5 exposed interface
In fact, I am also wondering if the url exposed by MD5 is really useful, and I have seen people asking questions.
Answered by:
It can't be said that it doesn't work. If you don't encrypt it, the user intercepts your access address, and he sees that the current seckill ID is 1000. He can completely infer other seckill addresses, or he can create a batch of addresses; the seckill in the video is in The seckill time is determined in the database. Naturally, he can't seckill at other times, but it also has a certain impact on the database. If he uses a timer or cyclic seckill software, the endurance of your system is a problem; on the other hand, for some people who have not started The second kill, after he simulates the address, he can use the timer to access it all the time. After encryption, since he can't get the obfuscated code, he can only click on the link to kill...
Simple understanding: After MD5 encryption, the user cannot simulate the real address before the spike, which still has a certain effect.
enum class
In return new SeckillExecution(seckillId, 1, "Seckill success", successKilled); in the code **, the state and stateInfo parameter information we return should be output to the front end, but we don't want to hard code these two in our return code parameters, so we should consider encapsulating these constants with enumeration**,
public enum SeckillStatEnum {
SUCCESS(1,"秒杀成功"),
END(0,"秒杀结束"),
REPEAT_KILL(-1,"重复秒杀"),
INNER_ERROR(-2,"系统异常"),
DATE_REWRITE(-3,"数据篡改");
private int state;
private String info;
SeckillStatEnum(int state, String info) {
this.state = state;
this.info = info;
}
public int getState() {
return state;
}
public String getInfo() {
return info;
}
public static SeckillStatEnum stateOf(int index)
{
for (SeckillStatEnum state : values())
{
if (state.getState()==index)
{
return state;
}
}
return null;
}
}
Web Tier Development Skills
Restful interface design learning
I have been exposed to the idea of RESTful before, but it was not used in the first project. Because I am still not used to it, I am afraid to write a nondescript RESTful interface, so I plan to apply RESTful in the second project.
Reference blog post: kb.cnblogs.com/page/512047…
Details not known before SpringMVC
The @DateTimeFormat annotation formats the time! (I haven't tried this yet)
<!--配置spring mvc-->
<!--1,开启springmvc注解模式
a.自动注册DefaultAnnotationHandlerMapping,AnnotationMethodHandlerAdapter
b.默认提供一系列的功能:数据绑定，数字和日期的format@NumberFormat,@DateTimeFormat
c:xml,json的默认读写支持-->
<mvc:annotation-driven/>
<!--2.静态资源默认servlet配置-->
<!--
1).加入对静态资源处理：js,gif,png
2).允许使用 "/" 做整体映射
-->
<mvc:default-servlet-handler/>
Returns uniformly formatted JSON
Previously, dto was encapsulated in the Web layer and Service to transmit the data of these two layers, and we generally return JSON to the front end for parsing in the Controller.
The best thing to do is to unify the JSON format as well. Doing this will be a good way to form a specification!
//将所有的ajax请求返回类型，全部封装成json数据
public class SeckillResult<T> {
private boolean success;
private T data;
private String error;
public SeckillResult(boolean success, T data) {
this.success = success;
this.data = data;
}
public SeckillResult(boolean success, String error) {
this.success = success;
this.error = error;
}
public boolean isSuccess() {
return success;
}
public void setSuccess(boolean success) {
this.success = success;
}
public T getData() {
return data;
}
public void setData(T data) {
this.data = data;
}
public String getError() {
return error;
}
public void setError(String error) {
this.error = error;
}
}
How to get JSON data
Previously, JSON was obtained object.propertiesin the same way, and this time I saw another way:
JavaScript modularity
Before writing JS code in the project, what function did you want, and where did you write it. After watching this video, I found that JS can be modularized! ! !
The readability of JS is better than before when it is modularized. This is something I have not touched before, so I should pay attention to writing JS code in the future!
Paste a piece of code below to get a feel for it:
/**
* 模块化javaScript
* Created by jianrongsun on 17-5-25.
*/
var seckill = {
// 封装秒杀相关的ajax的url
URL: {
now: function () {
return "/seckill/time/now";
},
exposer: function (seckillId) {
return "/seckill/" + seckillId + "/exposer";
},
execution: function (seckillId, md5) {
return "/seckill/" + seckillId + "/" + md5 + "/execution";
}
},
// 验证手机号码
validatePhone: function (phone) {
return !!(phone && phone.length === 11 && !isNaN(phone));
},
// 详情页秒杀业务逻辑
detail: {
// 详情页开始初始化
init: function (params) {
console.log("获取手机号码");
// 手机号验证登录，计时交互
var userPhone = $.cookie('userPhone');
// 验证手机号
if (!seckill.validatePhone(userPhone)) {
console.log("未填写手机号码");
// 验证手机控制输出
var killPhoneModal = $("#killPhoneModal");
killPhoneModal.modal({
show: true, // 显示弹出层
backdrop: 'static', // 静止位置关闭
keyboard: false // 关闭键盘事件
});
$("#killPhoneBtn").click(function () {
console.log("提交手机号码按钮被点击");
var inputPhone = $("#killPhoneKey").val();
console.log("inputPhone" + inputPhone);
if (seckill.validatePhone(inputPhone)) {
// 把电话写入cookie
$.cookie('userPhone', inputPhone, {expires: 7, path: '/seckill'});
// 验证通过 刷新页面
window.location.reload();
} else {
// todo 错误文案信息写到前端
$("#killPhoneMessage").hide().html("<label class='label label-danger'>手机号码错误</label>").show(300);
}
});
} else {
console.log("在cookie中找到了电话号码，开启计时");
// 已经登录了就开始计时交互
var startTime = params['startTime'];
var endTime = params['endTime'];
var seckillId = params['seckillId'];
console.log("开始秒杀时间=======" + startTime);
console.log("结束秒杀时间========" + endTime);
$.get(seckill.URL.now(), {}, function (result) {
if (result && result['success']) {
var nowTime = seckill.convertTime(result['data']);
console.log("服务器当前的时间==========" + nowTime);
// 进行秒杀商品的时间判断，然后计时交互
seckill.countDown(seckillId, nowTime, startTime, endTime);
} else {
console.log('结果:' + result);
console.log('result' + result);
}
});
}
}
},
handlerSeckill: function (seckillId, mode) {
// 获取秒杀地址
mode.hide().html('<button class="btn btn-primary btn-lg" id="killBtn">开始秒杀</button>');
console.debug("开始进行秒杀地址获取");
$.get(seckill.URL.exposer(seckillId), {}, function (result) {
if (result && result['success']) {
var exposer = result['data'];
if (exposer['exposed']) {
console.log("有秒杀地址接口");
// 开启秒杀，获取秒杀地址
var md5 = exposer['md5'];
var killUrl = seckill.URL.execution(seckillId, md5);
console.log("秒杀的地址为:" + killUrl);
// 绑定一次点击事件
$("#killBtn").one('click', function () {
console.log("开始进行秒杀,按钮被禁用");
// 执行秒杀请求，先禁用按钮
$(this).addClass("disabled");
// 发送秒杀请求
$.post(killUrl, {}, function (result) {
var killResult = result['data'];
var state = killResult['state'];
var stateInfo = killResult['stateInfo'];
console.log("秒杀状态" + stateInfo);
// 显示秒杀结果
mode.html('<span class="label label-success">' + stateInfo + '</span>');
});
});
mode.show();
} else {
console.warn("还没有暴露秒杀地址接口,无法进行秒杀");
// 未开启秒杀
var now = seckill.convertTime(exposer['now']);
var start = seckill.convertTime(exposer['start']);
var end = seckill.convertTime(exposer['end']);
console.log("当前时间" + now);
console.log("开始时间" + start);
console.log("结束时间" + end);
console.log("开始倒计时");
console.debug("开始进行倒计时");
seckill.countDown(seckillId, now, start, end);
}
} else {
console.error("服务器端查询秒杀商品详情失败");
console.log('result' + result.valueOf());
}
});
},
countDown: function (seckillId, nowTime, startTime, endTime) {
console.log("秒杀的商品ID:" + seckillId + ",服务器当前时间：" + nowTime + ",开始秒杀的时间:" + startTime + ",结束秒杀的时间" + endTime);
// 获取显示倒计时的文本域
var seckillBox = $("#seckill-box");
// 获取时间戳进行时间的比较
nowTime = new Date(nowTime).valueOf();
startTime = new Date(startTime).valueOf();
endTime = new Date(endTime).valueOf();
console.log("转换后的Date类型当前时间戳" + nowTime);
console.log("转换后的Date类型开始时间戳" + startTime);
console.log("转换后的Date类型结束时间戳" + endTime);
if (nowTime < endTime && nowTime > startTime) {
// 秒杀开始
console.log("秒杀可以开始,两个条件符合");
seckill.handlerSeckill(seckillId, seckillBox);
}
else if (nowTime > endTime) {
alert(nowTime > endTime);
// console.log(nowTime + ">" + startTime);
console.log(nowTime + ">" + endTime);
// 秒杀结束
console.warn("秒杀已经结束了,当前时间为:" + nowTime + ",秒杀结束时间为" + endTime);
seckillBox.html("秒杀结束");
} else {
console.log("秒杀还没开始");
alert(nowTime < startTime);
// 秒杀未开启
var killTime = new Date(startTime + 1000);
console.log(killTime);
console.log("开始计时效果");
seckillBox.countdown(killTime, function (event) {
// 事件格式
var format = event.strftime("秒杀倒计时: %D天 %H时 %M分 %S秒");
console.log(format);
seckillBox.html(format);
}).on('finish.countdown', function () {
// 事件完成后回调事件，获取秒杀地址，控制业务逻辑
console.log("准备执行回调,获取秒杀地址,执行秒杀");
console.log("倒计时结束");
seckill.handlerSeckill(seckillId, seckillBox);
});
}
},
cloneZero: function (time) {
var cloneZero = ":00";
if (time.length < 6) {
console.warn("需要拼接时间");
time = time + cloneZero;
return time;
} else {
console.log("时间是完整的");
return time;
}
},
convertTime: function (localDateTime) {
var year = localDateTime.year;
var monthValue = localDateTime.monthValue;
var dayOfMonth = localDateTime.dayOfMonth;
var hour = localDateTime.hour;
var minute = localDateTime.minute;
var second = localDateTime.second;
return year + "-" + monthValue + "-" + dayOfMonth + " " + hour + ":" + minute + ":" + second;
}
};
High concurrency performance optimization
The first three articles have already completed this system, but as a seckill system, the amount of concurrency it can support is very low. So now we have to consider how to tune it.
analyze
The address interface of seckill can be optimized with the help of redis, and there is no need to access the database multiple times.
The seckill operation is related to the transaction of the database and cannot be replaced by a cache. The solution given below requires modifying the source code, and the difficulty is relatively difficult.
Let's analyze where the bottleneck is:
- Mysql executes a single SQL statement is actually very fast.
- Mainly the wait for row-level lock transactions, network delays and GC recycling!
Solutions:
Solve the seckill interface
For the seckill interface, you need to use Redis to cache the data. Then the user does not need to access the database when accessing, we just cache the data for Redis.
This time use Jedis to operate Redis.
There is also something worth noting: we can use ProtostuffIOUtil instead of JDK serialization, because this serialization function is much better than JDK's!
package com.suny.dao.cache;
import com.dyuproject.protostuff.LinkedBuffer;
import com.dyuproject.protostuff.ProtostuffIOUtil;
import com.dyuproject.protostuff.runtime.RuntimeSchema;
import com.suny.entity.Seckill;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
/**
* 操作Redis的dao类
* Created by 孙建荣 on 17-5-27.下午4:44
*/
public class RedisDao {
private final Logger logger = LoggerFactory.getLogger(this.getClass());
private final JedisPool jedisPool;
private RuntimeSchema<Seckill> schema = RuntimeSchema.createFrom(Seckill.class);
public RedisDao(String ip, int port) {
jedisPool = new JedisPool(ip, port);
}
public Seckill getSeckill(long seckillId) {
// redis操作业务逻辑
try (Jedis jedis = jedisPool.getResource()) {
String key = "seckill:" + seckillId;
// 并没有实现内部序列化操作
//get->byte[]字节数组->反序列化>Object(Seckill)
// 采用自定义的方式序列化
// 缓存获取到
byte[] bytes = jedis.get(key.getBytes());
if (bytes != null) {
// 空对象
Seckill seckill = schema.newMessage();
ProtostuffIOUtil.mergeFrom(bytes, seckill, schema);
// seckill被反序列化
return seckill;
}
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
return null;
}
public String putSeckill(Seckill seckill) {
// set Object(Seckill) -> 序列化 -> byte[]
try (Jedis jedis = jedisPool.getResource()) {
String key = "seckill:" + seckill.getSeckillId();
byte[] bytes = ProtostuffIOUtil.toByteArray(seckill, schema,
LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE));
// 超时缓存
int timeout=60*60;
return jedis.setex(key.getBytes(), timeout, bytes);
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
return null;
}
}
<!--导入连接redis的JAR包-->
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>2.9.0</version>
</dependency>
<!--添加序列化依赖-->
<dependency>
<groupId>com.dyuproject.protostuff</groupId>
<artifactId>protostuff-core</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>com.dyuproject.protostuff</groupId>
<artifactId>protostuff-runtime</artifactId>
<version>1.1.1</version>
</dependency>
RedisDao is not affected by the proxy of Mybatis, so we need to create it manually.
Ultimately, our service logic will look like this:
Seckill operation optimization
Going back to our seckill operation again, what actually needs to be optimized is the waiting time of our GC and row-level locks.
Our previous logic is as follows: first perform the inventory reduction operation, and then insert the record of successful purchase
In fact, we can insert the record of successful purchase first, and then perform the operation of reducing inventory!
- So what's the difference between the two? ? ? The operation of reducing inventory will cause row-level locks to wait, and if we insert first, we will not be disturbed by row-level locks. Moreover, our two operations are in the same thing, and there will be no "oversold" situation!
Regarding the difference between executing insert first and executing update first, when two transactions insert at the same time, there is no lock competition, and the execution speed will be faster. When two transactions update the same row of data first, one transaction will obtain the row lock, which is locked in the transaction. It will not be released until it is committed, so keeping the lock for the shortest time can improve efficiency
So the logic of our service layer can be changed to this:
This is not the final solution. For performance optimization, we can also run SQL in Mysql without Spring's transaction management. Using stored procedures in Mysql to submit performance
-- 秒杀执行储存过程
DELIMITER $$ -- console ; 转换为
$$
-- 定义储存过程
-- 参数： in 参数 out输出参数
-- row_count() 返回上一条修改类型sql(delete,insert,update)的影响行数
-- row_count:0:未修改数据 ; >0:表示修改的行数； <0:sql错误
CREATE PROCEDURE `seckill`.`execute_seckill`
(IN v_seckill_id BIGINT, IN v_phone BIGINT,
IN v_kill_time TIMESTAMP, OUT r_result INT)
BEGIN
DECLARE insert_count INT DEFAULT 0;
START TRANSACTION;
INSERT IGNORE INTO success_killed
(seckill_id, user_phone, create_time)
VALUES (v_seckill_id, v_phone, v_kill_time);
SELECT row_count()
INTO insert_count;
IF (insert_count = 0)
THEN
ROLLBACK;
SET r_result = -1;
ELSEIF (insert_count < 0)
THEN
ROLLBACK;
SET r_result = -2;
ELSE
UPDATE seckill
SET number = number - 1
WHERE seckill_id = v_seckill_id
AND end_time > v_kill_time
AND start_time < v_kill_time
AND number > 0;
SELECT row_count()
INTO insert_count;
IF (insert_count = 0)
THEN
ROLLBACK;
SET r_result = 0;
ELSEIF (insert_count < 0)
THEN
ROLLBACK;
SET r_result = -2;
ELSE
COMMIT;
SET r_result = 1;
END IF;
END IF;
END;
$$
-- 储存过程定义结束
DELIMITER ;
SET @r_result = -3;
-- 执行储存过程
CALL execute_seckill(1003, 13502178891, now(), @r_result);
-- 获取结果
SELECT @r_result;
Mybatis calling stored procedures is actually the same as JDBC:
When using the stored procedure, we need 4 parameters, in fact, the result is assigned in the stored procedure. We can get the corresponding value through MapUtils. This is something I haven't touched before.
Finally, the system architecture for deployment should look like this:
Summarize
After spending some time watching this video tutorial, I feel that I have learned a lot. I haven't been exposed to optimization-related issues before, but now it has opened up my mind and learned a lot of development specifications, which is also very good. If you are a beginner, you can learn it.
If there are any mistakes in the article, please correct me, and we can communicate with each other. Students who are accustomed to reading technical articles on WeChat and want to get more Java resources can follow WeChat public account: Java3y