学成在线-第15天-讲义-媒资管理系统集成三

2.3 Logstash扫描课程计划媒资 
Logstash定时扫描课程媒资信息表，并将课程媒资信息写入索引库。‘ 
2.3.1 创建索引 
1、创建xc_course_media索引 
2、并向此索引创建如下映射 
Post http://localhost:9200/xc_course_media/doc/_mapping

{
"properties" : {
"courseid" : {
"type" : "keyword"
},
"teachplan_id" : {
"type" : "keyword"
},
"media_id" : {
"type" : "keyword"
},
"media_url" : {
"index" : false,
"type" : "text"
},
"media_fileoriginalname" : {
"index" : false,
"type" : "text"
}
}
}

2.3.2 创建Logstash模板文件 
在logstach的confifig目录创建xc_course_media_template.json，内容如下： 
本教程的xc_course_media_template.json目录是：D:/ElasticSearch/logstash-6.2.1/confifig/xc_course_media_template.json 

{
"mappings" : {
"doc" : {
"properties" : {
"courseid" : {
"type" : "keyword"
},
"teachplan_id" : {
"type" : "keyword"
},
"media_id" : {
"type" : "keyword"
},
"media_url" : {
"index" : false,
"type" : "text"
},
"media_fileoriginalname" : {
"index" : false,
"type" : "text"
}
}
},
"template" : "xc_course_media"
}

2.3.3 配置mysql.conf 
在logstash的confifig目录下配置mysql_course_media.conf文件供logstash使用，logstash会根据mysql_course_media.conf文件的配置的地址从MySQL中读取数据向ES中写入索引。
参考https://www.elastic.co/guide/en/logstash/current/plugins-inputs-jdbc.html 
配置输入数据源和输出数据源。

input {
stdin {
}
jdbc {
jdbc_connection_string => "jdbc:mysql://localhost:3306/xc_course?
useUnicode=true&characterEncoding=utf‐8&useSSL=true&serverTimezone=UTC"
# the user we wish to excute our statement as
jdbc_user => "root"
jdbc_password => mysql
# the path to our downloaded jdbc driver
jdbc_driver_library => "F:/develop/maven/repository3/mysql/mysql‐connector‐java/5.1.41/mysql‐
connector‐java‐5.1.41.jar"
# the name of the driver class for mysql
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
#要执行的sql文件
#statement_filepath => "/conf/course.sql"
statement => "select * from teachplan_media_pub where timestamp >
date_add(:sql_last_value,INTERVAL 8 HOUR)"
#定时配置
schedule => "* * * * *"
record_last_run => true
last_run_metadata_path => "D:/ElasticSearch/logstash‐6.2.1/config/xc_course_media_metadata"
}
}
output {
elasticsearch {
#ES的ip地址和端口
hosts => "localhost:9200"
#hosts => ["localhost:9200","localhost:9202","localhost:9203"]
#ES索引库名称
index => "xc_course_media"
document_id => "%{id}"
document_type => "doc"
template =>"D:/ElasticSearch/logstash‐6.2.1/config/xc_course_media_template.json"
template_name =>"xc_course_media"
template_overwrite =>"true"
}
stdout {
#日志输出
codec => json_lines
}
}

2.3.4 启动logstash.bat 
启动logstash.bat采集teachplan_media_pub中的数据，向ES写入索引。

logstash.bat ‐f ../config/mysql_course_media.conf

2.4 搜索服务查询课程媒资接口 
2.4.1 需求分析 
搜索服务提供查询课程媒资接口，此接口供学习服务调用。 
2.4.2 Api 
在课程搜索包下定义Api

@ApiOperation("根据课程计划查询媒资信息")
public TeachplanMediaPub getmedia(String teachplanId);

2.4.3 Service 
1、配置课程计划媒资索引库等信息 
在application.yml中配置

xuecheng:
elasticsearch:
media:
index: xc_course_media
type: doc
source_field: courseid,media_id,media_url,teachplan_id,media_fileoriginalname

2、service方法开发 
在课程搜索服务中定义课程媒资查询接口，为了适应后续需求，service参数定义为数组，可一次查询多个课程计划的媒资信息。 
 

//根据课程计划查询媒资信息
public QueryResponseResult<TeachplanMediaPub> getmedia(String[] teachplanIds){
//设置索引
SearchRequest searchRequest = new SearchRequest(media_index);
//设置类型
searchRequest.types(media_type);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//source源字段过虑
String[] source_fields = media_source_field.split(",");
searchSourceBuilder.fetchSource(source_fields, new String[]{});
//查询条件，根据课程计划id查询(可传入多个id)
searchSourceBuilder.query(QueryBuilders.termsQuery("teachplan_id", teachplanIds));
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = null;
try {
//执行搜索
searchResponse = restHighLevelClient.search(searchRequest);
} catch (IOException e) {
e.printStackTrace();
}
//获取搜索结果
SearchHits hits = searchResponse.getHits();
SearchHit[] searchHits = hits.getHits();
Map<String,CoursePub> map = new HashMap<>();
//数据列表
List<TeachplanMediaPub> teachplanMediaPubList = new ArrayList<>();
for (SearchHit hit : searchHits) {
TeachplanMediaPub teachplanMediaPub =new TeachplanMediaPub();
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
//取出课程计划媒资信息
String courseid = (String) sourceAsMap.get("courseid");
String media_id = (String) sourceAsMap.get("media_id");
String media_url = (String) sourceAsMap.get("media_url");
String teachplan_id = (String) sourceAsMap.get("teachplan_id");
String media_fileoriginalname = (String) sourceAsMap.get("media_fileoriginalname");
teachplanMediaPub.setCourseId(courseid);
teachplanMediaPub.setMediaUrl(media_url);
teachplanMediaPub.setMediaFileOriginalName(media_fileoriginalname);
teachplanMediaPub.setMediaId(media_id);
teachplanMediaPub.setTeachplanId(teachplan_id);
//将数据加入列表
teachplanMediaPubList.add(teachplanMediaPub);
}
//构建返回课程媒资信息对象
QueryResult<TeachplanMediaPub> queryResult = new QueryResult<>();
queryResult.setList(teachplanMediaPubList);
QueryResponseResult<TeachplanMediaPub> queryResponseResult = new
QueryResponseResult<TeachplanMediaPub>(CommonCode.SUCCESS,queryResult);
return queryResponseResult;
}

2.4.4 Controller

@Override
@GetMapping(value="/getmedia/{teachplanId}")
public TeachplanMediaPub getmedia(@PathVariable("teachplanId") String teachplanId) {
//将课程计划id放在数组中，为调用service作准备
String[] teachplanIds = new String[]{teachplanId};
//通过service查询ES获取课程媒资信息
QueryResponseResult<TeachplanMediaPub> mediaPubQueryResponseResult =
esCourseService.getmedia(teachplanIds);
QueryResult<TeachplanMediaPub> queryResult = mediaPubQueryResponseResult.getQueryResult();
if(queryResult!=null
&& queryResult.getList()!=null
&& queryResult.getList().size()>0){
//返回课程计划对应课程媒资
return queryResult.getList().get(0);
}
return new TeachplanMediaPub();
}

2.4.5 测试 
使用swagger-ui和postman测试课程媒资查询接口。