Logstash
Logstash is an open source server-side data processing pipeline that can simultaneously collect data from multiple sources, format the data, and then send the data to es for storage.
ElasticSearch
Elasticsearch is a distributed search and analysis engine based on JSON, and it is a full-text index implemented by using inverted index.
Kibana
Kibana can visualize and manipulate data in Elasticsearch.
es is at the core of the elk ecosystem. It is an open source large-scale inverted index-based full-text search and analysis engine. It can support storage search and analysis in almost real time.
Advantage:
Horizontal scalability: adding servers can be directly configured in the cluster
The sharding mechanism provides better distribution: divide and conquer to improve processing efficiency
High availability: provide a replication (replica) mechanism
Real-time: Improve query speed by putting files on disk into a file cache system
basic concept
Index: A collection of documents, similar to the concept of a database in mysql
Type: Different types can be defined in Index. The concept of type is similar to the concept of table in mysql, which is a combination of a series of data with the same characteristics.
Document: The concept of a document is similar to a storage record in mysql, and it is in json format. There can be many documents under different types under Index.
Shards: When the amount of data is large, expand horizontally to improve search performance
Replicas: To prevent data loss of a certain fragment, it can be parallelized in the backup data and search to improve performance
elasticsearch query syntax
_cat API
Query the relevant information of the current es cluster, including the number of indexes in the cluster, the running status, and the ip where the current cluster is located. The purpose is to output the query results in a more friendly way.
cat:
_cat apiall supported query commands in the outputcat health: Check the running status of the es cluster
cat count: can quickly query the number of documents in the cluster or index
cat indices: query the data of all indexes in the current cluster, including the number of fragments of the index, the number of documents, the size of the storage space...
For other cat APIs, refer to the official documentation: www.elastic.co/guide/en/el... [1]
Search APIs
Search data, multiple query syntax, powerful function
REST request URI : Light and fast URI query method
REST request body : json format query method with many restrictions
query"query": Allows us to query in the request message bodyQuery DSL."term": When querying, it is judged whether a document contains a specific value, and word segmentation query will not be performed on the queried value
"match" will be word-segmented by the query value, and then scored by the scoring mechanism (TF/IDF)
"match_phrase": query the specified paragraph
"Bool": combined with other truth-valued queries, usually
must should mustnotcombined with (and or not) to form complex queries"range": Specify a field in a specific range when querying
location / {
# 指向我们打包后上传的前端文件
root /opt/nginx/dist;
index index.html;
}
location /jwt/ {
# 转发请求到后端服务网关
proxy_pass http://127.0.0.1:8765/jwt/;
}
location /api/ {
proxy_pass http://127.0.0.1:8765/api/;
}The default Nginx configuration file path is as follows:
Nginx main configuration file path:
/www/server/nginx/conf/nginx.confWebsite configuration file path:
/www/server/panel/vhost/nginx/Default home page file path:
/www/server/panel/vhost/index.html
1. Start nginx
Switch to the corresponding path
start nginx.exe
2. Reload the configuration file
nginx -s reload
Enter the path where the jar package is located
enter:
nohup java -jar xxx.jar &Press Enter and then enter exit (note that you must enter exit to keep it running.)
To stop it from running, enter:
netstat -antp | grep javaThen stop the corresponding process:
kill -9 pid
TypeError: Failed to execute 'fetch' on 'Window': Request with GET/HEAD method cannot have body.
1. Swagger reports an error:
1. Error type:
TypeError: Failed to execute ‘fetch‘ on ‘Window‘: Request with GET/HEAD method cannot have body
2. Solution:
The request method is wrong: the request parameter uses the @RequestBody annotation, so it is necessary to use Post to make the request
2. The difference between @RequestParam and @RequestBody
1、@RequestParam：
The parameters received by @RequestParam come from requestHeader, that is, the request header. Usually used for GET requests.
2、@RequestBody：
The parameters received by @RequestParam come from requestBody, that is, the request body. It is mainly used to receive the data in the json string passed from the front end to the back end, so only POST requests can be sent.
server {
listen 80;
server_name www.aaa.ink;
client_max_body_size 1000M;
client_body_timeout 20s;
client_header_timeout 10s;
send_timeout 30s;
ssl_protocols TLSv1.2;
charset utf-8;
#后端接口
location ^~ /api/ {
proxy_pass http://127.0.0.1:8080/api/;
}
location ~/(.*)$ {
#前端项目
root /usr/local/nginx/ttm;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}When LB forwards, bring
X-Forwarded-Portthe request header and forward the port number of the original request (LB forwarding needs to be controlled by itself, if we want to configure it, we need to ask DevOps children's shoes to help. It is more convenient if it is completely controlled by ourselves [recommended])Before using the Swagger middleware, set
X-Forwarded-Portthe request header to443(not flexible enough, if the access to LB is http or has a special port number, there will be problems)Remove the request header before using the swagger middleware
X-Forwarded-Host, so that there will be no servers attribute (it doesn’t feel elegant enough)Register one
PreSerializeFilterto clear Servers
In Swagger UI, you can serversspecify server information for an API using configuration. This enables you to define and switch between different server configurations in the Swagger UI for interacting with different API environments.
The following is a sample Swagger UI configuration file demonstrating how to use serversconfiguration:
import springfox.documentation.swagger.web.SwaggerResource;
import springfox.documentation.swagger.web.SwaggerResourcesProvider;
import java.util.ArrayList;
import java.util.List;
@Configuration
public class SwaggerConfig {
@Value("${swagger.enabled:true}")
private boolean enabled;
@Value("${swagger.pathMapping:/api}")
private String pathMapping;
@Bean
public Docket api() {
return new Docket(DocumentationType.OAS_30)
.enable(enabled)
.apiInfo(apiInfo())
.select()
.apis(RequestHandlerSelectors.withMethodAnnotation(ApiOperation.class))
.paths(PathSelectors.any())
.build()
.securitySchemes(securitySchemes())
.securityContexts(securityContexts())
.pathMapping(pathMapping)
.servers(servers());
}
private List<Server> servers() {
List<Server> servers = new ArrayList<>();
servers.add(new Server("https://1024bat.cn"));
// 添加其他服务器配置
return servers;
}
// 其他方法...
}The specific error message is "org.apache.ibatis.type.TypeException: Could not set parameters for mapping". This error usually occurs when there is a problem trying to set a value for a parameter in a MyBatis map.
According to the error message, this specific problem is "Error setting non null for parameter #1 with JdbcType null", i.e. there was a problem trying to set a non-null value for the first parameter, and the JdbcType was null. Also, the error message mentions a cast problem: "java.lang.ClassCastException: class java.lang.Long cannot be cast to class java.lang.String".
This means that there may be a parameter type mismatch in your MyBatis mapping file. Specifically, it expects a parameter of type String, but a value of type Long is actually passed in, causing the type conversion to fail.
In order to solve this problem, you can try the following steps:
Check the parameter type: check the relevant parameter definition in the MyBatis mapping file to ensure that the parameter type matches the type of the database field. Type conversion errors may result if the parameter type does not match the database field type.
Check the parameter value passed in: Check the parameter value passed in the code to ensure that the type of the parameter value is consistent with the parameter type defined in the MyBatis mapping file. If the parameter value is not of the correct type, you may need to cast or correct it accordingly.
Set the correct JdbcType: In the MyBatis mapping file, try to set an appropriate JdbcType for this parameter. You can specify the JdbcType by adding the attribute to the parameter definition in the mapping file
jdbcType, egjdbcType="VARCHAR".Check configuration: Check MyBatis configuration files to make sure there are no configuration issues, such as configuration of type aliases or type handlers.
If you want Spring Boot to return fields with empty values when there is no corresponding value in the database, you can use another configuration option of the Jackson library.
Add the following configuration to the Spring Boot configuration file:
spring.jackson.default-property-inclusion=non_emptyAlternatively, application.ymluse the following configuration in the configuration file:
spring:
jackson:
default-property-inclusion: non_emptynullThis will tell Jackson to ignore properties with a value or null when serializing the object , and only return non-null properties. When there is no corresponding value in the database, the property will be serialized as an empty string instead null.
Note that non_emptyoptions exclude nullempty strings, empty collections, empty arrays, etc. in addition to excluding values. If you just want to exclude nullvalues and keep empty strings, use non_nulloptions.
With the above configuration, when the property's value is null, it will be serialized as an empty string. If the property's value is an empty string or an empty collection, the property will not be returned.
SpringBoot Jackson supports Java8 LocalDate format and null to ""
@Bean
public ObjectMapper objectMapper() {
ObjectMapper objectMapper = new ObjectMapper();
// 1.将null转""
objectMapper.getSerializerProvider().setNullValueSerializer(new JsonSerializer<Object>() {
@Override
public void serialize(Object paramT, JsonGenerator paramJsonGenerator, SerializerProvider paramSerializerProvider) throws IOException {
//设置返回null转为 空字符串""
paramJsonGenerator.writeString("");
}
});
// 2.支持java8 LocalDate时间
objectMapper.findAndRegisterModules();
return objectMapper;
}default-property-inclusionThe configuration property has the following options:
always: Always include the attribute, even if the attribute value is emptynullor empty.non_null: only include nonnull-properties, other empty values (such as empty strings, empty collections, empty arrays) will be excluded.non_absent: Contains nonnulland non-absent attributes. "Missing" refers to attributes that do not appear in JSON.non_default: Contains attributes with non-nulland non-default values. "Default value" refers to the default initialization value of a Java object field, such as0,false, empty string, etc.non_empty: Include nonnulland non-null attributes, other empty values (such as empty string, empty collection, empty array) will be excluded.null: Include all attributes, even if the attribute value is emptynullor empty.
In Spring Boot, the default configuration option is default-property-inclusion=nullthat all properties are included in the serialized result, including nulland null values.
Based on the error message provided, it looks like there is a JSON parsing error. The error message shows that an unexpected character '}' was encountered during parsing, expecting a double quote to start the field name.
This usually happens when received JSON data is malformed, possibly due to sent data being malformed or having other formatting issues. Please ensure JSON.parse()that the data passed to is a valid JSON string and conforms to the syntax requirements of JSON.
You could try checking the data passed to JSON.parse()and event.datamake sure it's a valid JSON string. Check the JSON string for incorrect characters, missing quotes, or other syntax errors.
Also, you can use console.log(event.data)output received event.datadata so you can see what is actually received. This provides a better understanding of the data received for further debugging and locating issues.
If you need more detailed help, please provide the actual received event.datadata so that I can help you more accurately.
Add the group to contact the author vx: xiaoda0423
Warehouse address: https://github.com/webVueBlog/JavaGuideInterview
References
[1]
https://link.jianshu.com?t=https://www.elastic.co/guide/en/elasticsearch/reference/5.5/cat.html: https://link.juejin.cn?target=https%3A%2F%2Flink.jianshu.com%3Ft%3Dhttps%3A%2F%2Fwww.elastic.co%2Fguide%2Fen%2Felasticsearch%2Freference%2F5.5%2Fcat.html