2019.6 months, I was recruited into the company through social work job now, in theory, there should be a trial period of three months, the trial period only perform well I have a chance to positive, but because of an optimized code built during a wheel, I get opportunities a month to positive.
I am a lazy, but also particularly fond of wondering, in the course of work I found a module running very slow, mainly due to need a lot of database operations in this module types, and expand the company's business, already in the database there are hundreds of millions of pieces of data, each of the operating table, need to spend nearly 3S time, in fact, go down the whole process, the program will spend 4S time, so I especially want to look at the code optimization , will take the code is reduced down after a week of efforts, the wheels of the first edition published, exhausted after all my colleagues feel good, then the boss gave me regularization application submitted in advance, I work one month on the successful regularization.
I will now create the wheel was the main idea to write down here, I hope to give you some inspiration.
1. Wheels Related
First of all, we are talking about is not a wheel wheels on the car, it was called "wheels", in order to better understand.
Automotive wheel is round, that round wheels have been widely recognized by all walks of life is a better structure, no matter how we do it, it is difficult to go beyond the "round wheels", so we can only go on the basis of "round wheels" of < repeat the wheel> that is made of said wheel. But there is a word called "Do not repeat create the wheel," because no matter how hard we try, it is difficult to go beyond already have the wheels before, then why do we still have to make the wheel?
1.只有我们自己才懂得我们的需求，你在编程中无论用什么框架，总觉得跟业务不是很契合，甚至有时候框架很臃肿，但是因为要用到其中一个内容而不得不导入一个很大的框架。
2.已有的框架太复杂，我们完全没法掌握他的所有内容，出了bug后一头雾水解决不了问题
3.公司要求，不可使用第三方框架（多见于数据保密的公司），但是程序中要反复使用某一组件功能
4.找不到合适的轮子
5.想装BIf you encounter these problems, perhaps it is a good way to create the wheel.
2. preparation
This article talked about, might just make the most basic tutorial wheels, so reserve of knowledge required is not very comprehensive, if there is big brother to see this article, tap spray, I do humbly accept criticism.
2.1. Basics
Total hair from the last article about reading the source code, there is always a friend asked me, "Hello, I'm just a freshman, do not understand how the source code?", Often encounter this problem, I am very despair, in that article I was talking about, are based on the premise that already have some programming basics, and if you are reading this article you had any contact with programming, it is now possible to tap attention, thumbs up, collection, then go back and learn about basic look at this article.
We want to build a wheel, at least have some programming skills, if there must be a standard, it is the ability to build a project alone.
This paper mainly uses the knowledge points are: annotations, reflection, polymorphism
2.2 Learn Source
For example, some framework Jdk source and source ( have not read the source code, then click I ), create the wheel on some level, is to write a framework, we have no basis in the case, look at some of the big brother framework written source code is helpful, in our own wheels, they can mimic the structure of the frame.
2.3. Afraid of failure
The first-create the wheel is definitely a difficult and lengthy process, you will fail again and again, you need to withstand live failed to combat bring you confidence and patience, if you can not insist, might as well not start.
3. open dry
If you see here, I believe you are ready, so let's get started!
3.1 Want good demand
Since it is made of wheels, so they will have to decide before you use this wheel, we assume that we are a demand: output to the system log file annotation achieved by
Specific requirements are as follows:
1. The data recorded annotation injection
2. The logging should be saved by the file system, the path may be configured to, if not the configuration selected default configuration
3. Logging need to add a time stamp
4. The log file name can be set in a comment
The introduction of the log transfer queue MSG
3.2 Creating project
To simplify the process, we can directly create a Maven project, after you have a better understanding of the underlying, you can use a better architecture.
LogUtil a new project, the project structure is as follows
Because it is only a simple example, so there are a lot of content knowledge of ideas, but not yet implemented
3.2. Some constants
The main file saved in Constants.java
public interface Constants {
//等下要引入的配置文件名
String CONFIG_FILE_NAME = "yzlogconfig";
//配置文件中配置的日志路径
String CONFIG_LOG_PATH = "logpath";
//配置文件中配置的要扫描的，可能存在我们注解的路径
String CONFIG_SACN_PATH = "scanpath";
//若未声明某些信息，则使用以下默认值
//默认的我们的日志信息前缀，对日志信息做简单描述
String DEFAULT_CONTENT_PREFIX = "注入值：";
//默认的日志文件名（实际写入时会在日志文件名后加上日期标签）
String DEFAULT_FILE_NAME = "log";
//日志信息类型，处理消息时会用到
String MSG_TYPE_LOG = "log";
//默认的Linux系统下的日志路径
String LINUX_LOG_PATH = "/home/data/";
//默认的Windows系统下的日志路径
String WIN_LOG_PATH = "D:/winLog/data/";
}
3.3. Load configuration
Idea: give the user to configure permissions
Others showed framework is used, the user must give permission to independent configuration
Here is configured to load those projects into our wheels
In a project to introduce our wheel, he himself should have the authority to set yourself some things, such as our document processing path here, it is to give the user permission to configure.
We require file configuration file named yzlogconfig.xml, wait for the next set can also see the profile name in the code.
ConfigurationUtil
The principal tool used to load configuration information, basic tools, there is no longer tired
import java.util.ResourceBundle;
public class ConfigurationUtil {
private static Object lock = new Object();
private static ConfigurationUtil config = null;
private static ResourceBundle rb = null;
private ConfigurationUtil(String filename) {
rb = ResourceBundle.getBundle(filename);
}
public static ConfigurationUtil getInstance(String filename) {
synchronized (lock) {
if (null == config) {
config = new ConfigurationUtil(filename);
}
}
return (config);
}
public String getValue(String key) {
String ret = "";
if (rb.containsKey(key)) {
ret = rb.getString(key);
}
return ret;
}
}
3.4. Logging function to achieve
Here regarded as part of the core functions, and tools needed are: DateUtil (acquisition date), SystemUtil (get the current type system), FileUtil (log file is created)
DataUtil
import java.text.SimpleDateFormat;
import java.util.Date;
public class DateUtil {
public final static String DATE_A = "yyyy-MM-dd";
public final static String DATE_B = "yyyy-MM-dd HH:mm:ss";
public final static String DATE_C = "yyyyMMddHHmmss";
public final static String DATE_D = "yyyyMMdd-HHmmss-SS";
public final static String DATE_E = "M月d日";
public final static String DATE_F = "MM-dd";
public final static String DATE_G = "yyyyMMddHHmmss";
// 普通的当前时间转字符串方法，格式为yyyy-MM-dd
public static String getDate() {
SimpleDateFormat sdf = new SimpleDateFormat(DATE_A);
return sdf.format(new Date());
}
public static String getDateTime() {
Date date = new Date();
String datestr;
SimpleDateFormat sdf = new SimpleDateFormat(DATE_B);
datestr = sdf.format(date);
return datestr;
}
}
SystemUtil
/**
*@描述 用于判断当前系统
*@参数
*@返回值
*@创建人 Baldwin
*@创建时间 2020/4/4
*@修改人和其它信息
*/
public class SystemUtil {
/**
* 判断系统时win还是linux
* @return
*/
public static boolean isLinux(){
String name = System.getProperty("os.name");
if(name.toLowerCase().startsWith("win"))
return false;
else
return true;
}
}FileUtil
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileOutputStream;
import java.io.OutputStreamWriter;
public class FileUtil {
// 在已经存在的文件后面追加写的方式
public static boolean write(String path, String str) {
File f = new File(path);
File fileParent = f.getParentFile();
BufferedWriter bw = null;
try {
if(!fileParent.exists()){
fileParent.mkdirs();
}
if(!f.exists()){
f.createNewFile();
}
// new FileWriter(name,true)设置文件为在尾部添加模式,参数为false和没有参数都代表覆写方式
bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(path, true), "UTF-8"));
bw.write(str);
} catch (Exception e) {
e.printStackTrace();
return false;
} finally {
try {
if(bw!=null)bw.close();
} catch (Exception e) {
System.out.println("FileUtil.write colse bw wrong:" + e);
}
}
return true;
}
}LogUtil
Thought: the default configuration
In many cases, our customers may not need to set the configuration, but also did not comment the statement, then there is a default configuration requires us to fill these vacancies, in order to avoid errors due to empty due to the configuration
In this class there, we mainly make some sort of log path and content
import cn.yzstu.support.Constants;
import cn.yzstu.support.DateUtil;
import cn.yzstu.support.FileUtil;
import cn.yzstu.support.SystemUtil;
public class LogUtil {
//日志写入操作
public static void write2file(String path, String fileName, String content) {
//获取当前日期，我们的日志保存的文件夹名是自定义path+日期
String date = DateUtil.getDate()+"/";
try {
//传了path，那我们直接用这个path
if (null != path && 0 != path.length()) {
//写入
FileUtil.write(path + date + fileName + ".txt",
DateUtil.getDateTime() + ":" + content + "\r\n");
} else {
//没有传path或错误使用默认的路径
if (SystemUtil.isLinux()) {
FileUtil.write(Constants.LINUX_LOG_PATH + date + fileName + ".txt",
DateUtil.getDateTime() + ":" + content + "\r\n");
} else {
FileUtil.write(Constants.WIN_LOG_PATH + date + fileName + ".txt",
DateUtil.getDateTime() + ":" + content + "\r\n");
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
3.5. Log messages
We want to wait in a queue in the log message to process, a log message type is defined herein class to facilitate the subsequent processing, in this example, made simplified, in fact, for the message queue, we need to define a unified interface, all message types have achieved him, so if our message type a lot of time, you can do a unified managed.
For convenience we deal with, in the constructor let the news into the column, and his MsgType directly became logmsg
import cn.yzstu.core.MsgQueue;
import cn.yzstu.support.Constants;
public class LogMsg {
private String path;
private String content;
private String fileName;
private String msgType;
public LogMsg(String path, String content, String fileName) {
this.path = path;
this.content = content;
this.fileName = fileName;
this.msgType = "logmsg";
//在构造函数中就让这个消息入列
MsgQueue.push(this);
}
public String getPath() {
return path;
}
public void setPath(String path) {
this.path = path;
}
public String getContent() {
return content;
}
public void setContent(String content) {
this.content = content;
}
public String getFileName() {
return fileName;
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public String getMsgType() {
return this.msgType;
}
public void setMsgType(String msgType) {
this.msgType = msgType;
}
@Override
public String toString() {
return "LogMsg{" +
"path='" + path + '\'' +
", content='" + content + '\'' +
", fileName='" + fileName + '\'' +
", msgType='" + msgType + '\'' +
'}';
}
}
3.6 Definitions queue
If the queue is not very familiar with, I can see another article: elaborate queue
In reality, our cohort in which there will be many types of messages, there is only logmsg in the present example.
import cn.yzstu.beans.LogMsg;
import java.util.Queue;
import java.util.concurrent.ConcurrentLinkedDeque;
public class MsgQueue {
private static Queue<LogMsg> queue = new ConcurrentLinkedDeque<>();
//消息入列
public static boolean push(LogMsg logMsg){
return queue.offer(logMsg);
}
//消息出列
public static LogMsg poll(){
return queue.poll();
}
//消息队列是否已经处理完毕，处理完毕返回true
public static boolean isFinash(){
return !queue.isEmpty();
}
}
3.7 Definition of notes
Here we define a class named YzLogWrite annotation, which main function is to inject value and log mark.
For the notes is not very understanding of my colleagues can see another article: I want to write your own framework? Annotations can not understand
YzLogWrite
import java.lang.annotation.*;
//作用于字段
@Target({ElementType.FIELD})
//运行时生效
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface YzLogWrite {
//需要注解的值
int value() default -1;
//默认是Linux系统，默认记录文件夹如下
String path() default "";
//文件名
String fileName() default "";
//内容
String msgPrefix() default "";
}3.8. Notes logic implementation
Idea: greater than the configured statement
If we declare the annotation of some of the information used, but also the profile information, we should choose the limited information in the notes stated.
Idea: custom scan path
We should give users permission to let him set himself used his own notes package
We define the annotation, but also need to do something to improve the annotation features. In this section, we want to inject value, and the value of information is sent to the message queue.
DealAnnotation
import cn.yzstu.annotation.YzLogWrite;
import cn.yzstu.beans.LogMsg;
import cn.yzstu.support.Constants;
import java.io.File;
import java.lang.annotation.Annotation;
import java.lang.reflect.Field;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;
public class DealAnnotation {
//配置文件中设置的log所在地址
private static String LOG_PATH = ConfigurationUtil.getInstance(Constants.CONFIG_FILE_NAME).getValue(Constants.CONFIG_LOG_PATH);
//保存那些存在注解的class的类名
private List<String> registyClasses = new ArrayList<>();
public void injectAndMakeMsg() {
//需要扫描的注解可能存在的位置
String scanPath = ConfigurationUtil.getInstance(Constants.CONFIG_FILE_NAME).getValue(Constants.CONFIG_SACN_PATH);
doScanner(scanPath);
for (String className : registyClasses) {
try {
Class clazz = Class.forName(className);
Field[] fields = clazz.getDeclaredFields();
for (Field field : fields) {
//获取类的所有注解
Annotation[] annotations = field.getAnnotations();
//没有注解或没有我们的注解，跳过
if (0 == annotations.length || !field.isAnnotationPresent(YzLogWrite.class)) {
continue;
}
//获取注解
YzLogWrite yzLogWrite = field.getAnnotation(YzLogWrite.class);
//提取注解中的值
//声明大于配置
String path = null == yzLogWrite.path() || yzLogWrite.path().isEmpty() ? LOG_PATH : yzLogWrite.path();
String content = null == yzLogWrite.msgPrefix() || yzLogWrite.msgPrefix().isEmpty() ? Constants.DEFAULT_CONTENT_PREFIX : yzLogWrite.msgPrefix();
String fileName = null == yzLogWrite.fileName() || yzLogWrite.fileName().isEmpty() ? Constants.DEFAULT_FILE_NAME : yzLogWrite.fileName();
int value = yzLogWrite.value();
//新建logMsg，在构造函数中已入列
new LogMsg(path, content + ":" + value, fileName);
//开始注入
//强制访问该成员变量
field.setAccessible(true);
//注入int值
field.setInt(Integer.class, value);
}
} catch (ClassNotFoundException | IllegalAccessException e) {
e.printStackTrace();
}
}
}
private void doScanner(String scanPath) {
URL url = this.getClass().getClassLoader().getResource(scanPath.replaceAll("\\.", "/"));
File classPath = new File(url.getFile());
for (File file : classPath.listFiles()) {
if (file.isDirectory()) {
//如果是目录则递归调用，直到找到class
doScanner(scanPath + "." + file.getName());
} else {
if (!file.getName().endsWith(".class")) {
continue;
}
String className = (scanPath.replace("/", ".") + "." + file.getName().replace(".class", ""));
registyClasses.add(className);
}
}
}
}3.9. Message processing
Idea: multi-state distribution
Let us try to queue can process messages of different types, we obtained after the message queue, the message should be a determination of the type and distribution of different types of messages to the operator in different ways
Through the above operation, we have the value of the injected and the log messages passed to the queue, and now also on the message queue processing
import cn.yzstu.beans.LogMsg;
public class DealMsg extends Thread{
@Override
public void run() {
while (MsgQueue.isFinash()){
//多态
//实际中，我们可以定义很多中msg，用type来区分，并通过不同的方法来处理
//此处运用了这种思想，但是没有实现具体操作
LogMsg logMsg = MsgQueue.poll();
switch (logMsg.getMsgType()){
case "logmsg" :
//如果类型是logmsg，那就通过日志来处理
dealLogMsg(logMsg);
break;
default:defaultMethod(logMsg);
}
}
this.interrupt();
}
private void defaultMethod(LogMsg logMsg) {
System.out.println("no msg");
}
private void dealLogMsg(LogMsg logMsg) {
LogUtil.write2file(logMsg.getPath(),logMsg.getFileName(),logMsg.getContent());
}
@Override
public synchronized void start() {
this.run();
}
}3.10 Provide entrance
Our simple example of a function has been basically completed, how it was introduced? Here is the approach I take to stay the operation of a method to perform all of our functions.
import cn.yzstu.annotation.YzLogWrite;
public class StartWork {
//程序入口
public static void doWork(){
//处理：扫描注解、注入、发送日志消息到队列
new DealAnnotation().injectAndMakeMsg();
//创建线程来处理消息
new DealMsg().start();
}
}4. Test
We have completed all of the above, we need to test now.
Create a configuration class
We require a configuration file called "yzlogconfig", so now create a configuration file in the resource folder.
#logpath最后需要带/
logpath = /opt/
scanpath = cn/yzstu/ttWe only configure the log log path location and notes, to test whether the default parameters take effect
Create a test class
We require the above configuration file annotation package we use, so it should be in the package down using annotations, otherwise we can not scan notes
import cn.yzstu.annotation.YzLogWrite;
import cn.yzstu.core.StartWork;
public class Demo {
//因为测试用的main函数是static，所以此时将age设置为static
@YzLogWrite(value = 18,msgPrefix = "记录Baldwin的年龄：")
static int age;
public static void main(String[] args) {
StartWork.doWork();
System.out.println(age);
}
}
Results of the
First look at the console, showing successful injection
/opt/java/jdk1.8.0_241/bin/java -javaagent:/opt/jetbrains/idea-IU-193.6911.18/lib/idea_rt.jar=38115:/opt/jetbrains/idea-IU-193.6911.18/bin -Dfile.encoding=UTF-8 -classpath /opt/java/jdk1.8.0_241/jre/lib/charsets.jar:/opt/java/jdk1.8.0_241/jre/lib/deploy.jar:/opt/java/jdk1.8.0_241/jre/lib/ext/cldrdata.jar:/opt/java/jdk1.8.0_241/jre/lib/ext/dnsns.jar:/opt/java/jdk1.8.0_241/jre/lib/ext/jaccess.jar:/opt/java/jdk1.8.0_241/jre/lib/ext/jfxrt.jar:/opt/java/jdk1.8.0_241/jre/lib/ext/localedata.jar:/opt/java/jdk1.8.0_241/jre/lib/ext/nashorn.jar:/opt/java/jdk1.8.0_241/jre/lib/ext/sunec.jar:/opt/java/jdk1.8.0_241/jre/lib/ext/sunjce_provider.jar:/opt/java/jdk1.8.0_241/jre/lib/ext/sunpkcs11.jar:/opt/java/jdk1.8.0_241/jre/lib/ext/zipfs.jar:/opt/java/jdk1.8.0_241/jre/lib/javaws.jar:/opt/java/jdk1.8.0_241/jre/lib/jce.jar:/opt/java/jdk1.8.0_241/jre/lib/jfr.jar:/opt/java/jdk1.8.0_241/jre/lib/jfxswt.jar:/opt/java/jdk1.8.0_241/jre/lib/jsse.jar:/opt/java/jdk1.8.0_241/jre/lib/management-agent.jar:/opt/java/jdk1.8.0_241/jre/lib/plugin.jar:/opt/java/jdk1.8.0_241/jre/lib/resources.jar:/opt/java/jdk1.8.0_241/jre/lib/rt.jar:/root/IdeaProjects/LogUtil/target/classes cn.yzstu.tt.Demo
18
Process finished with exit code 0Then we look at the / opt folder has under it have the log files, log successfully written
View the log file, enable the contents of the statement notes
2020-04-06 00:17:30:记录Baldwin的年龄：:185. Summary
So far, we have a simple wheel logging has made good, we can put him labeled jar package introduced to our project to go, we just need to enable the feature when the project can be initialized.
5.1. Idea
Idea: give the user to configure permissions
Others showed framework is used, the user must give permission to independent configuration
Thought: the default configuration
In many cases, our customers may not need to set the configuration, but also did not comment the statement, then there is a default configuration requires us to fill these vacancies, in order to avoid errors due to empty due to the configuration
Thought: the default configuration
In many cases, our customers may not need to set the configuration, but also did not comment the statement, then there is a default configuration requires us to fill these vacancies, in order to avoid errors due to empty due to the configuration
Idea: greater than the configured statement
If we declare the annotation of some of the information used, but also the profile information, we should choose the limited information in the notes stated.
Idea: custom scan path
We should give users permission to let him set himself used his own notes package
Idea: multi-state distribution
Let us try to queue can process messages of different types, we obtained after the message queue, the message should be a determination of the type and distribution of different types of messages to the operator in different ways
5.2. About this project
The author is a program under way to crawl sprouting new, this instance is only available to beginners to use, if there is an error, please also feel free chiefs pointing.
Item code: https://gitee.com/dikeywork/LogUtil
5.3. Personal summary
Encountered many difficulties in the completion of the project, we had intended to one day. Thing, but the real finish this article has spent two full days, still need to progress.
I am Baldwin, a 25-year-old programmer, dedicated to making learning more interesting, if you really love programming, and sincerely hope that you make friends, wander together in the programming of the ocean!
Source Reading Skills: https://blog.csdn.net/shouchenchuan5253/article/details/105196154
Detailed Java annotations: https://blog.csdn.net/shouchenchuan5253/article/details/105145725
Teach you self-built SpringBoot server: https://blog.csdn.net/shouchenchuan5253/article/details/104773702
For more articles please click: https://blog.csdn.net/shouchenchuan5253/article/details/105020803
