page element replacement
The first thing to do is of course to replace the site name and content with your own style.
1. Website configuration
After tracking the front-end code, I found that the configuration comes from the back-end interface. I thought that since it has been stored, there should be a corresponding management page. Sure enough, I found it, but... the demo account is not allowed to operate! Then the next thing to do is obvious, get this user!
2. Account configuration
Switch to idea and find that there is a line of error reporting on the workbench, and jump to the corresponding code according to the error reporting, and find that this authentication method has never been seen before! Quickly program for Baidu.
Sa-token document address: https://sa-token.dev33.cn/
After getting a general understanding of this framework, it is simply a gospel for lazy people x. Then I found that there is user management + password change on the page, then things become easier.
3. File upload
Because the file server configured as expected is minio, the author only attaches two methods of local and Qiniu, so the transformation begins.
Add minio label option
Global search 图片上传方式, find the corresponding binding field, add minio . PS: Ali oss did not exist originally, but after tracking the back-end code, it was found that the dictionary value 2 corresponds to Ali oss, so it was added first.
backend code
From the trace /file/uploadinterface, it can be found that the backend determines which upload strategy to invoke based on the fileUploadWay configuration field.
private void getFileUploadWay() {
strategy = FileUploadModelEnum.getStrategy(systemConfigService.getCustomizeOne().getFileUploadWay());
}
Tracking FileUploadModelEnum found that it is an enumeration class, then first add the enumeration of minio.
First introduce minio in pom.xml .
<!-- Minio -->
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>8.2.2</version>
</dependency>
The original author's configuration method is to add fields to the database. This method is not very used, so the configuration of minio here is added to the configuration file, and then use **@Value** injection.
#============================Minio配置信息===================================
minio:
url: http://ip:9000
accessKey: minio账号
secretKey: minio密码
bucketName: 桶名称
preurl: http://预览地址
Then imitate aliUploadStrategyImpl to create a service corresponding to minio.
package com.shiyi.strategy.imp;
import com.shiyi.strategy.FileUploadStrategy;
import io.minio.MinioClient;
import io.minio.PutObjectArgs;
import io.minio.RemoveObjectsArgs;
import io.minio.Result;
import io.minio.messages.DeleteError;
import io.minio.messages.DeleteObject;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.time.DateFormatUtils;
import org.apache.velocity.shaded.commons.io.FilenameUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
import javax.annotation.PostConstruct;
import java.util.Arrays;
import java.util.Date;
import java.util.UUID;
import java.util.stream.Collectors;
@Service("minioUploadStrategyImpl")
public class MinioUploadStrategyImpl implements FileUploadStrategy {
private final Logger logger = LoggerFactory.getLogger(MinioUploadStrategyImpl.class);
/**
* 服务地址
*/
@Value("${minio.url}")
private String url;
/**
* 预览路径前缀
*/
@Value("${minio.preurl}")
private String preurl;
/**
* 用户名
*/
@Value("${minio.accessKey}")
private String accessKey;
/**
* 密码
*/
@Value("${minio.secretKey}")
private String secretKey;
/**
* 存储桶名称
*/
@Value("${minio.bucketName}")
private String bucketName;
private static MinioClient client = null;
@PostConstruct
private void init(){
client = MinioClient.builder().endpoint(url).credentials(accessKey, secretKey).build();
}
@Override
public String fileUpload(MultipartFile file,String suffix) {
String fileName = null;
try {
String extension = FilenameUtils.getExtension(file.getOriginalFilename());
fileName = DateFormatUtils.format(new Date(), "yyyy/MM/dd") + "/" + UUID.randomUUID() + "." + extension;
PutObjectArgs args = PutObjectArgs.builder()
.bucket(bucketName)
.object(fileName)
.stream(file.getInputStream(), file.getSize(), -1)
.contentType(file.getContentType())
.build();
client.putObject(args);
} catch (Exception e) {
e.printStackTrace();
}
return preurl + "/" + bucketName + "/" + fileName;
}
/**
* 删除文件 -- minio
*
* @param key 文件url
* @return ResponseResult
*/
@Override
public Boolean deleteFile(String ...key) {
if (key.length > 0) {
//批量删除
Iterable<DeleteObject> deleteObjects = Arrays.stream(key).map(s -> new DeleteObject(s)).collect(Collectors.toList());
Iterable<Result<DeleteError>> results = client.removeObjects(
RemoveObjectsArgs.builder()
.bucket(bucketName)
.objects(deleteObjects)
.build()
);
for (Result<DeleteError> result : results) {
try {
result.get();
} catch (Exception e) {
logger.error(e.getMessage());
e.printStackTrace();
}
}
}
return true;
}
}
First add a note at the entrance, and then use swagger to call the test, PS: remember to log in first
! 8879-4&crop=0&crop=0&crop=1&crop=1&from=paste&height=199&id=u6c178023&margin=[object Object]&name=image.png&originHeight=199&originWidth=2033&originalType=binary&ratio=1&rotation=0&showTitle=fal se&size=57089&status=done&style=none&taskId=u7ed14748-0b8e- 4b09-b099-6807f1e7572&title=&width=2033)
file intermediate table
Why use an intermediate table? The main reason is to protect the port of minio. Both upload and download are performed through code, so you cannot guess other file paths through the file hierarchy. And prevent minio from suddenly exposing any loopholes. [Of course, it doesn’t matter if this is used for the project! 】
- Create table statement
CREATE TABLE `tb_files` (
`id` bigint(20) NOT NULL COMMENT '主键id',
`user_id` bigint(20) DEFAULT NULL COMMENT '用户id',
`create_time` datetime DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`preview_file` varchar(128) DEFAULT NULL COMMENT '文件minio地址',
`file_name` varchar(512) DEFAULT NULL COMMENT '原文件名称',
`content_type` varchar(50) DEFAULT NULL COMMENT '文件类型',
`is_static` tinyint(1) DEFAULT '0' COMMENT '是否静态资源',
PRIMARY KEY (`id`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 ROW_FORMAT=DYNAMIC COMMENT='文件表';
- Just copy a code generator
/**
* 代码生成器
*/
public class CodeGenerator {
public static void main(String[] args) {
// 1、创建代码生成器
AutoGenerator mpg = new AutoGenerator();
// 2、全局配置
GlobalConfig gc = new GlobalConfig();
String projectPath = System.getProperty("user.dir");
gc.setOutputDir(projectPath + "/src/main/java");
gc.setAuthor("dingx");
gc.setOpen(false); //生成后是否打开资源管理器
gc.setFileOverride(false); //重新生成时文件是否覆盖
/*
* mp生成service层代码，默认接口名称第一个字母有 I
* UcenterService
* */
gc.setServiceName("%sService"); //去掉Service接口的首字母I
gc.setIdType(IdType.ASSIGN_ID); //主键策略
gc.setDateType(DateType.ONLY_DATE);//定义生成的实体类中日期类型
gc.setSwagger2(true);//开启Swagger2模式
mpg.setGlobalConfig(gc);
// 3、数据源配置
DataSourceConfig dsc = new DataSourceConfig();
dsc.setUrl("jdbc:mysql://ip:port/schema?serverTimezone=GMT%2B8");
dsc.setDriverName("com.mysql.cj.jdbc.Driver");
dsc.setUsername("root");
dsc.setPassword("pwd");
dsc.setDbType(DbType.MYSQL);
mpg.setDataSource(dsc);
// 4、包配置
PackageConfig pc = new PackageConfig();
// pc.setModuleName(scanner("模块名")); //模块名
pc.setParent("com.shiyi");
pc.setController("controller");
pc.setEntity("entity");
pc.setService("service");
pc.setMapper("mapper");
mpg.setPackageInfo(pc);
// 5、策略配置
StrategyConfig strategy = new StrategyConfig();
strategy.setInclude(scanner("表名"));
strategy.setNaming(NamingStrategy.underline_to_camel);//数据库表映射到实体的命名策略
strategy.setTablePrefix(pc.getModuleName() + "_"); //生成实体时去掉表前缀
strategy.setColumnNaming(NamingStrategy.underline_to_camel);//数据库表字段映射到实体的命名策略
strategy.setEntityLombokModel(true); // lombok 模型 @Accessors(chain =true) setter链式操作
strategy.setRestControllerStyle(true); //restful api风格控制器
strategy.setControllerMappingHyphenStyle(true); //url中驼峰转连字符
mpg.setStrategy(strategy);
// 6、执行
mpg.execute();
}
private static String scanner(String tip) {
Scanner scanner = new Scanner(System.in);
System.out.println(("请输入" + tip + "："));
if (scanner.hasNext()) {
String ipt = scanner.next();
if (StringUtils.isNotBlank(ipt)) {
return ipt;
}
}
throw new MybatisPlusException("请输入正确的" + tip + "！");
}
}
- Retrofit upload code
private final TbFilesService tbFilesService;
@Override
public String fileUpload(MultipartFile file,String suffix) {
String fileName;
TbFiles tbFile = null;
try {
String extension = getExtension(file);
fileName = DateFormatUtils.format(new Date(), "yyyy/MM/dd") + "/" + UUID.randomUUID() + "." + extension;
//保存上传文件记录
tbFile = new TbFiles(file.getOriginalFilename(), fileName, file.getContentType());
if (!tbFilesService.save(tbFile)){
throw new RuntimeException("插入文件失败");
}
PutObjectArgs args = PutObjectArgs.builder()
.bucket(bucketName)
.object(fileName)
.stream(file.getInputStream(), file.getSize(), -1)
.contentType(file.getContentType())
.build();
client.putObject(args);
} catch (Exception e) {
e.printStackTrace();
}
return preurl + "/" + tbFile.getId();
}
- Add preview interface
public class TbFilesController {
private final TbFilesService filesService;
private final MinioUploadStrategyImpl minioUploadStrategy;
/**
* 预览
* @param id
* @return
*/
@SaIgnore
@GetMapping("/preview/{id}")
@ResponseBody
public ResponseEntity<StreamingResponseBody> preview(@PathVariable Long id){
TbFiles file = filesService.getDetail(id);
//设置头文件Content-type
HttpHeaders headers = new HttpHeaders();
// 发送给客户端的数据
// 设置编码
if (StringUtils.isNotBlank(file.getContentType())) {
headers.setContentType(MediaType.valueOf(file.getContentType()));
}
//构造返回体
return ResponseEntity.ok()
.headers(headers)
.body(outputStream -> {
try (InputStream inputStream = minioUploadStrategy.downloadFile(file.getPreviewFile())){
IOUtils.copy(inputStream, outputStream);
} catch (Exception e){
e.printStackTrace();
}
});
}
}
The pitfalls encountered here: 1) ResponseEntity
is used as the return object. If HttpServletResponse is used, the Content-type will be automatically changed to application/json by the Spring framework if it is changed . When searching for information, I saw a lot of produces attributes using **@GetMapping , but this fixed the content of Content-type . 2) The download method cannot be used to obtain the preview stream. Although the interface call is successful after putting the address in the label, the picture is split. 3) Interface verification ignores the interface @SaIgnore This annotation is not available in sa-token version 1.29 . Upgraded to version 1.32 here . Of course, you can also change the sa-token** interceptor in the WebMvcConfig file.
- function test
After uploading, check the database and it has been stored.
Call the preview method
- a warning
!!!
An Executor is required to handle java.util.concurrent.Callable return values.
Please, configure a TaskExecutor in the MVC config under "async support".
The SimpleAsyncTaskExecutor currently in use is not suitable under load.
-------------------------------
Request URI: '/dingx/data/files/preview/1594875366335397890'
!!!
To be honest, it was the first time I encountered a warning prompt after writing the code for so long. .
The general idea is: the default SimpleAsyncTaskExecutor is no longer applicable, please customize a TaskExecutor . Then add it, add the following code to WebMvcConfig .
@Bean
public ThreadPoolTaskExecutor mvcTaskExecutor() {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setCorePoolSize(100);
taskExecutor.setMaxPoolSize(100);
return taskExecutor;
}
@Override
public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
configurer.setTaskExecutor(mvcTaskExecutor());
}
Nginx hardened configuration
nginx cache
proxy_cache_path /root/cache levels=1:2 keys_zone=xd_cache:10m max_size=1g inactive=60m use_temp_path=off;
server {
location /{
...
proxy_cache xd_cache;
proxy_cache_valid 200 304 10m;
proxy_cache_valid 404 1m;
proxy_cache_key $host$uri$is_args$args;
add_header Nginx-Cache "$upstream_cache_status";
}
}
Configuration explanation:
- /root/cache: local path, used to set the storage address of Nginx cache resources
- levels=1:2 : By default, all cache files are placed in the root path specified above, which may affect the performance of the cache. It is recommended to specify 2
- Level directory to store cache files; 1 and 2 indicate to use 1-digit and 2-digit hexadecimal to name the directory name. The first-level directory is named with 1-digit hexadecimal, such as a; the second-level directory is named with 2-digit hexadecimal, such as 3a. So in this example, there are 16 first-level directories, 16*16=256 second-level directories, and the total number of directories is 16256=4096.
- When levels=1:1:1, it means a three-level directory, and the number of directories in each level is 16
- key_zone: Define a storage area in shared memory to store cached keys and metadata
- max_size : The maximum cache space, if not specified, all disk space will be used. When the upper limit of the disk is reached, the least used cache will be deleted
- inactive: If a cache is not accessed within the time specified by inactive, it will be deleted from the cache
- proxy_cache_valid: Configure the cache time of cache files in nginx cache, proxy_cache_valid 200 304 2m The cache time for cache files with status 200 and 304 is 2 minutes
- use_temp_path: It is recommended to be off, then nginx will write the cache file directly to the specified cache file
- proxy_cache: enable proxy cache, and specify key_zone, if proxy_cache off means close the cache
- add_header Nging-Cache "$upstream_cache_status": used for the front end to judge whether it is a cache, miss, hit, expired (cache expired), updating (update, use the old response), restore the nginx configuration, and only keep the upstream module
Notice:
- The priority of nginx cache expiration is: inactvie > source server Expires/max-age > proxy_cache_valid
- If Permission denied appears, modify nginx.conf and change the first line to user root
- By default, GET requests and HEAD requests will be cached, but POST requests will not be cached. Not all of them must be cached. You can filter some paths without caching
Vue project deployed to nginx, route 404
Check the recommended configuration on the official website, a copy of cv.
Article SEO
First configure your own website on the Baidu search resource station: http://data.zz.baidu.com/linksubmit/index
Find the general collection, add configuration items in the configuration file
baidu:
url: http://data.zz.baidu.com/urls?site=blog.dinganwang.top&token=
sourceurl: https://blog.dinganwang.top/articles/
Modify the articleSeo method. The author here uses a for loop to achieve batch push, emmmm is a bit strange to be honest, so I changed it a little.
@Value("${baidu.url}")
private String baiduUrl;
@Value("${baidu.sourceurl}")
private String sourceUrl;
private final static String SUCCESS = "success";
private final static String REMAIN = "remain";
/**
* 文章百度推送
* @return
*/
@Override
public ResponseResult articleSeo(List<Long> ids) {
String param = "";
for (Long id : ids) {
param += sourceUrl + id + "\n";
}
HttpEntity<String> entity = new HttpEntity<>(param.trim(), createBdHeader());
String res = restTemplate.postForObject(baiduUrl, entity, String.class);
JSONObject JO = JSONObject.parseObject(res);
if (JO.getInteger(SUCCESS) > 0){
return ResponseResult.success("成功推送【" + JO.getInteger(SUCCESS) + "】条，剩余量【" + JO.getInteger(REMAIN) + "】条");
}else {
return ResponseResult.error("推送失败！");
}
}
/**
* 构造百度seo头文件
* @return
*/
private static HttpHeaders createBdHeader(){
HttpHeaders headers = new HttpHeaders();
headers.add("Host", "data.zz.baidu.com");
headers.add("User-Agent", "curl/7.12.1");
headers.add("Content-Length", "83");
headers.add("Content-Type", "text/plain");
return headers;
}
On the second day, you can check the push status of the first day.
