Summary of technical points of java-IoT environment monitoring center (xml, Oracle, Maven, svn, virtual machine environment construction)

IoT Environmental Monitoring Center

1. Introduction to project modularization

---

When the sensor sends data to the gateway, the gateway needs to parse the data and send the parsed data to the server for storage as shown in the figure:

Then our project is divided into client and server. The gateway in the figure is equivalent to our client, and the cloud server in the figure is equivalent to our server. We need to send the parsed data to the server, the server Store the sent data. The specific module diagram is as follows:

2. Project structure diagram

---

3. Knowledge points involved

---

3.1xml parsing

​ Learned three ways to parse xml

​ dom analysis, sax analysis, dom4j analysis (mainstream way, simple and easy to understand)

dom parsing

//创建解析工厂并且获取到document对象
String filePath = "src/work/person.xml";
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder documentBuilder = factory.newDocumentBuilder();
Document document = documentBuilder.parse(new File(filePath));//filePath表示传入的xml文件的路径
// 获取根节点
Element root = document.getDocumentElement();
//获取到节点的属性集合
NamedNodeMap listMap = root.getAttributes();//这里举例获取到根节点的属性集合
// 获取根节点的所有子节点
NodeList list = root.getChildNodes();
// 得到标签所有的属性
NamedNodeMap attributes = element.getAttributes();//遍历得到的根节点

sax analysis

//通过不断的重写父类的方法得到xml的解析结果,基于事件的解析方式,只能解析一次,解析过得数据不能再解析
//不能过随机读取,只能够顺序读取

//获取到sax解析的父类
SAXParserFactory factory = SAXParserFactory.newInstance();
SAXParser parser = factory.newSAXParser();
//重写他们的方法就可以得到解析到的xml对象
startDocument()
startElement()
characters()
endElement()
endDocument()

dom4j analysis

//想要使用dom4j解析,就必须要引入相应的jar包
//得到解析器
SAXReader saxReader = new SAXReader();
//解析一个xml文件
Document document = saxReader.read(filePath);//filePath表示传入的xml文件的路径
//3.得到根节点
Element rootElement = document.getRootElement();
//得到标签的名字
System.out.println("<"+rootElement.getName()+">");
elements.forEach(t ->{
    
    
	System.out.print("<"+t.getName());
	List<Attribute> list =t.attributes();
	list.forEach(p->{
    
    
		System.out.print(" "+p.getName() + "=" + p.getValue());
		//默认的命名空间uri
		System.out.println(p.getNamespaceURI());
	});
	System.out.print(">");
	System.out.println();
	List<Element> list2 = t.elements();
	list2.forEach(l->{
    
    
		System.out.print("<"+l.getName()+">");
		System.out.print(l.getTextTrim());
		System.out.println("</"+l.getName()+">");
	});
});

3.2 Oracle database

1. The classification of sql statement

DQL (Data Query Language): Mainly select statements.

DML (Data Manipulation Language): mainly insert statements, update statements, delete statements, which will generate transactions

You need to commit or rollback after the statement to end the transaction, otherwise the database will not update the data

DDL (Data Definition Language): create, alter, drop, truncate statements

TCL (Transaction Control Language): commit, rollback, savepoint statements to maintain data consistency

DCL (Data Control Language): grant, revoke statements, used to perform permission grant and permission withdrawal operations

2. select statement

The select statement never modifies the contents of the table

3. Several commonly used functions

1. ​ ||: splicing function, the string is enclosed in single quotes
2. ​ nvl: null value can be replaced
3. distinct: perform deduplication processing
4. format: for output format

4. Sort

Sorting when querying data is to display the records in ascending or descending order according to the value of a certain field

select col_name,...
from tb_name
order by col_name [asc|desc]
--order by语句，只对查询记录显示调整，并不改变查询结果，所以执行权最低，最后执行
--排序的默认值是asc：表示升序，desc：表示降序
--如果有多个列排序，后面的列排序的前提是前面的列排好序以后有重复（相同）的值。

5. Condition query

and has a higher priority than or

--where子句的优先级别最高
--between and操作符，表示在俩个值之间
select id,last_name,salary
from s_emp
where salary between 700 and 1500;
--in()，表示值在一个指定的列表中
select id,last_name,salary
from s_emp
where id in (1,3,5,7,9);
--like，模糊查询，在值不精确的时候使用
-- %，通配0到多个字符
-- _，通配一个字符，并且是一定要有一个字符
-- \，转义字符，需要使用escape关键字指定，转义字符只能转义后面的一个字符
select id,last_name,salary
from s_emp
where last_name like'C%';	
-- is null 和 is not null
select id,last_name,commission_pct
from s_emp
where commission_pct is null;

Pseudo column: The core function of the pseudo column rownum in the Oracle database is to complete paging queries.

6. Group query

select	字段1,字段2			----5
from	表					---1
where	条件				    ---2
group by	分组条件			---3
having	分组筛选条件			---4
order by	排序条件		---6
--聚合函数能够出现的位置
1. select后面
2. having后面
3. order by后面

7. Subquery

The idea of ​​subquery is to use the query result of the first sql statement in the second sql statement. At this time, the result of the first sql statement can serve as one of the where conditions in the second sql. Value, or act as a virtual table.

select last_name,salary
from s_emp
where salary>
(select salary
 from s_emp
 where last_name='Smith');

8. Build a table

-- 一般的建表语句
create table 表名(
字段名 数据类型  [列约束类型],
字段名 数据类型  [列约束类型],
字段名 数据类型  [列约束类型],
字段名 数据类型  [列约束类型],
字段名 数据类型  [列约束类型]);

9.DML statement

insert into table name (column 1, column 2,...) values ​​(value 1, value 2,...);

update table name set column 1=value 1, column 2=value 2,... where condition;

delete from table name where conditions will delete all the data in the table if no conditions are added

Drop directly delete the table and cascade delete the information related to the table

Only DML statements will produce transactions, other statements will not produce transactions

Commit, rollback, and DDL statements can all end the current transaction

10. Sequence

Commonly used methods of creating sequences:

create sequence sequence name

11. Index

create index index name

on table name (column name);

3.3JDBC

1. Using the general procedure (step) JDBC database operations of:

1. Register driver
2. Get database connection
3. Create Statement type or subtype object
4. Execute sql
5. Processing results (if necessary, general query statements must be processed)
6. Close resource

General way of loading the driver

private String driverClass="oracle.jdbc.driver.OracleDriver";
private String url="jdbc:oracle:thin:@127.0.0.1:1521:XE";
private String user="briup";
private String password="briup";
//1.加载注册驱动
Class.forName(driverClass);
//2.获取连接对象
conn=DriverManager.getConnection(url,user,password);
//3.获取Statement对象
 stmt=conn.createStatement();
//4.执行SQL语句

Obtain JDBC connection through database connection pool

Properties properties = new Properties();
InputStream is = ConnectionTest.class.getClassLoader().getResourceAsStream("druid.properties");
properties.load(is);
//获取连接池对象
DataSource dataSource = DruidDataSourceFactory.createDataSource(properties);
// 获取连接
conn=dataSource.getConnection();

MetaData

Metadata, you can get the metadata object through the getMetadata() method of connection, and then you can get the name, version, and version information of the database

3.4SVN

Svn is similar to a version control tool. You can submit information for each version by connecting to svn. Once you find a problem, you can backtrack to a specified version.

3.5Maven

Build java project, dependency management and project information management

Project construction steps

1. Clean: delete the old file class bytecode file obtained from the previous compilation
2. Compile compile: compile java source program into class bytecode file
3. Test test: automatic test, automatically call the junit program
4. Report report: the results of the test program execution
5. Package package: War package for dynamic Web project, jar package for java project
6. Install install: Maven specific concept, copy the packaged files to the specified location in the maven warehouse 7. Deploy deploy: Put the results generated by the project on the server or container so that it can run

Project dependent format

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.13</version>
</dependency>

Generally, when you create a Maven project, you create an aggregate project, and then add submodules to it. The packaging method must be pom, and the general dependencies are written in the parent project, and then

Other modules automatically depend on the parent project and automatically depend on the jar package

4. Project construction

---

1. The use of a small plugin

Lombok can make the class do not need to write the construction method and the getset method, which will greatly improve the flexibility of the data (for example, modify the getset method and constructor when modifying attributes)

2. New Maven project

Divided into three modules

1. enviroment-common: It is specifically used to write the entity classes and tool classes that the entire project needs to use, as well as rely on the common dependencies of other projects, as a basis for other modules
2. enviroment-gateway: The gateway class, that is, the client, is responsible for reading data from the file, storing it in the way of objects and then sending it to the server
3. enviroment-server: The server, obtains the data sent by the client through serversocked, and then stores the data in the database (this project is stored in 31 tables)

4.1enviroment-common

The picture above is the common module of the project

It is mainly composed of the following modules:

1. exception (mainly provide exception classes in this project code, and several exception types created with enumerations)

   1. EnviromentException: Master the writing of exception classes
   2. ExceptionMessageEnum: Master the writing of enumeration

2. pojo (simple java object): By observing the format of the information, an object is abstracted for each piece of information, and the object is used to manipulate the data

   1. Enviromet: Master the object-oriented thinking, abstract each piece of information as an object, and abstract the specific content as an attribute

3. Utils (tools): write some common tools for projects

   1. BackUpUtil (this contains all the tools and methods of the project)

      1. store (File file, Object data, boolean append), by passing in the file, and the data to be transferred to the file, and whether to overwrite the source file

         This method can be used to back up the processed information to prevent the source data from being unable to be found after information processing

         In the early stage of the project construction, this method is possible, and problems will occur when deploying, because the jar package will not be loaded when the generated jar package is generated.

         The source folder causes the path of the file to be wrong. The file should be obtained through the class loader in the way of resource flow

         code show as below:

      Class name.class.getResourceAsStream("The name of the file to be operated, no path is required, just the file name + suffix can be directly")

      
      
      
      2. read(File file,boolean append),接着上次的阅读过得地方继续读取
      
      3. load(InputStream is) ,用于properties文件的读取
      
      4. close(Connection connection,Statement statement,ResultSet resultSet) 用于关闭流的方法
      
      

   2. IdWorker (snowflake algorithm)

      1. The snowflake algorithm only needs a new object in a class, otherwise it will cause the program to be extremely slow, because loading a class will perform a lot of calculations

4.2enviroment-gateway

The above picture is the content of the gateway module

1. gather

   1. IGather is the interface to realize the gateway function

      public Collection<Enviroment> gather() throws EnviromentException;
      

      The purpose of this function is to read the data and encapsulate each object and then encapsulate it into the collection

   2. GatherImpl is the implementation class of this interface

      1. First, read the data files sent to the gateway, and then use methods to parse them one by one
      2. Data format: 100|101|2|16|1|3|5d606f7802|1|1516323596029
      3. Use split to split them separately, then assign values ​​to the environment objects and add them to the collection
      4. Each analysis must be backed up. The next time it is read from the position where it ended last time, when an exception is encountered, it is stored in the exception file. The next time it is judged whether the exception file is empty, if it is not empty, the exception file will be parsed first.

2. send

   1. ISender stores the data in the collection after the gateway receives the data and sends it to the server

      void send(Collection<Enviroment> data) throws EnviromentException;
      

   2. SenderImpl is the implementation class of this interface

      The collection obtained in GatherImpl uses the socket as a bridge, and uses the object stream to pass it to the server,

      When the amount of data is large, you can wrap it with bufferedOutputStream, which will increase the sending rate

      Here is also the way to read the properties file

      The parameter in the properties.load() method is inputStream, which is convenient for reading the stream. You can also read files in this way in the future

3. Application

   The file is always read through multi-threading and sent to the client

4.3enviroment-server

The content of the server module on the picture above

1. revicer

   1. Revicer is the interface for receiving data, the purpose is to return an Environment collection

      Collection<Enviroment> revicer() throws EnviromentException;
      

   2. RevicerImpl is the implementation class of the interface, and the collection returned by Socket is obtained through ServerSocket

      First obtain the port number through the properties file, receive the information directly and directly receive the sent data, and then force it to the Environment collection.

2. database

   1. IStore is an interface for storing data in the database, and the purpose is to classify and store data

   2. IStoreImpl is the implementation class of the interface, responsible for storing the content in the collection to the database

      1. Get the content of the drive by reading the properties file

      2. Difficulty: divide the data into 31 tables according to the day, how to reduce the number of new prepareStatement objects when making the points

         By traversing the currently obtained collection, come in first to determine whether the number of the day is the same as the number of the previous one, if the same, no new object is needed.

         By defining a variable outside, the day value is updated every time, and then the date class is converted to a string using SimpleDateFormat.

         Assign the other values ​​separately, and then perform a batch processing.

3. Log

   1. Directly introduce log4j dependency, and then write a properties file in log format, create a logger object, and log printing can be obtained by logger.infor()

      Logger logger = Logger.getLogger(GatherImpl.class);
      logger.info();
      logger.debug();
      

4.4 Construction of the project

1. Native test

   Specifically modify the file path, the method in the modification toolkit is to use the resource flow to find in the whole project

2. Virtual machine test

   Because you are using a foreign virtual machine, you need to modify the time zone to Shanghai time (because the local Kunshan is too close to Shanghai, so Beijing time is not sensitive) modify the virtual machine time to Shanghai time and modify the ip address. . . Two virtual machines communicate

   //修改虚拟机当前时区为上海
   cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
   echo 'Asia/Shanghai' >/etc/timezone
   

3. The connection has been unavailable because of a problem with my virtual machine, GG. . .

4.5 Build a warehouse in Github and put the project into it

Logger logger = Logger.getLogger(GatherImpl.class);
logger.info();
logger.debug();
```

4.4 Construction of the project

1. Native test

   Specifically modify the file path, the method in the modification toolkit is to use the resource flow to find in the whole project

2. Virtual machine test

   Because you are using a foreign virtual machine, you need to modify the time zone to Shanghai time (because the local Kunshan is too close to Shanghai, so Beijing time is not sensitive) modify the virtual machine time to Shanghai time and modify the ip address. . . Two virtual machines communicate

   //修改虚拟机当前时区为上海
   cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
   echo 'Asia/Shanghai' >/etc/timezone
   

3. The connection has been unavailable because of a problem with my virtual machine, GG. . .

4.5 Build a warehouse in Github and put the project into it

Do it when you learn later