Use Node, Vue and ElasticSearch build real-time search engine

(Translator's Note: Related Reading: node.js , vue.js , elasticsearch )

Introduction

Elasticsearch is a distributed RESTful search and analysis engine, to address the growing use cases. Elasticsearch built on top of Apache Lucene, which is a high-performance text search engine library.

table of Contents

In today's lesson, you will learn how to use Node.js, Elasticsearch and Vue.js build real-time search engine. Therefore, the need for basic Vue.js and Node.js (Express) understanding of this tutorial.

getting Started

Let's start with this lesson to set up the environment. Because you will use Node.js, and therefore the easiest way to get started is to create a new folder and run npm init. Create a new file called elastic-node folder, change directories to the new folder, and then run npm init:

//创建一个名为elastic-node的新目录
mkdir elastic-node
//将目录更改为创建的新文件夹
cd elastic-node
//运行npm init来创建一个package.json文件
npm init

The above command will guide you through the creation process package.json file, the file is run any Node.js libraries necessary. Next, you need to install the required real-time search engine library. The library is required:

• Express: This library will be running our servers
• Body-parser: the library used in conjunction with the body of the request is analyzed Express.
• Elasticsearch: This is Elasticsearch official Node.js library, which is a real-time search engine.

To install these libraries, execute:

npm install express body-parser elasticsearch

Now, the first part of your environment has been established. However, your settings are missing Elasticsearch. You will need to install Elasticsearch. There are different ways to install Elasticsearch. If you are using Debian Linux operating system, you can download the .deb file and install it using dpkg.

//下载deb包
curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.4.deb
//使用dpkg安装deb包
sudo dpkg -i elasticsearch-5.6.4.deb

For other distributions / operating systems, you can here find instructions on how to install the Elasticsearch.

Elasticsearch does not start automatically after installation. Elasticsearch can use the command to start and stop the service:

// 启动Elasticsearch服务
sudo -i service elasticsearch start
// 停止Elasticsearch服务
sudo -i service elasticsearch stop

To Elasticsearch configured to start automatically at system startup, run:

// 重新加载systemctl守护进程
sudo /bin/systemctl daemon-reload
// enable elastic search so it can be called as a service
sudo /bin/systemctl enable elasticsearch.service

After running the above command, you can run the following commands to start and stop Elasticsearch:

// 启动Elasticsearch服务
sudo systemctl start elasticsearch.service
// 停止Elasticsearch服务
sudo systemctl stop elasticsearch.service

Check the Elasticsearch status:

// Elasticsearch的状态
sudo service elasticsearch status

Note: Google Chrome Elastic toolkit can help you quickly see Elasticsearch indexing and documentation.

The index data in Elasticsearch

Data.js create a file in the root folder and add:

//data.js
//require the Elasticsearch librray
const elasticsearch = require('elasticsearch');
// 实例化一个Elasticsearch客户端
const client = new elasticsearch.Client({
   hosts: [ 'http://localhost:9200']
});
// ping客户端以确保Elasticsearch已启动
client.ping({
     requestTimeout: 30000,
 }, function(error) {
 // 此时，eastic搜索已关闭，请检查您的Elasticsearch服务
     if (error) {
         console.error('Elasticsearch cluster is down!');
     } else {
         console.log('Everything is ok');
     }
 });

Let me explain what you are doing in the above code block: First, you need Elasticsearch library and build a new Elasticsearch client passed a host array. If you notice, the host is http: // localhost: 9200. This is because by default, Elasticsearch listens on port 9200. Next, you ping Elasticsearch client to make sure that the server is started. If you run a node data.js, you should get a message saying everything is normal.

Learn index

Different from ordinary database, Elasticsearch index is a place to store related documents. For example, you will create a data cities_list called index scotch.io-tutorial to store type. This is Elasticsearch work:

// data.js
// 创建一个名为scotch.io-tutorial的新索引。如果索引已经被创建，这个函数会安全地失败
client.indices.create({
      index: 'scotch.io-tutorial'
  }, function(error, response, status) {
      if (error) {
          console.log(error);
      } else {
          console.log("created a new index", response);
      }
});

Add this code after the ping function written before. Now run the node data.js again, you should get two messages:

• Everything is okay (as usual)
• Created a new index (with the response from Elasticsearch) (create a new index (the response from Elasticsearch))

Add documents to the index

Elasticsearch API so that documents can be easily added to the index was created. as follows:

// 将数据添加到已创建的索引
client.index({
     index: 'scotch.io-tutorial',
     id: '1',
     type: 'cities_list',
     body: {
         "Key1": "Content for key one",
         "Key2": "Content for key two",
         "key3": "Content for key three",
     }
 }, function(err, resp, status) {
     console.log(resp);
 });

The above illustrative code blocks. Text refers to the document you want to add scotch.io-tutorial index, which is a type more categories. However, note that if the key id is omitted, it will automatically generate a elasticsearch.

However, in this lesson, your document will be the list of all the cities in the world. If you want to individually add each city, then take a few days (if not weeks) to complete index of all cities. Fortunately, Elasticsearch there is a batch processing function for batch data.

First, grab contains all cities in the world JSON file, and save it to your root folder as cities.json

Now it's time to use our bulk API to import large amounts of data on:

//data.js
// require the array of cities that was downloaded
const cities = require('./cities.json');
// 声明一个名为bulk的空数组
var bulk = [];
// 循环遍历每个城市，并在每个循环中创建并将两个对象推入数组中
// 第一个对象发送索引和类型，保存数据
// 第二个对象是你想索引的数据
cities.forEach(city =>{
   bulk.push({index:{ 
                 _index:"scotch.io-tutorial", 
                 _type:"cities_list",
             }          
         })
  bulk.push(city)
})
// 对传递的数据执行批量索引
client.bulk({body:bulk}, function( err, response  ){ 
         if( err ){ 
             console.log("Failed Bulk operation".red, err) 
         } else { 
             console.log("Successfully imported %s".green, cities.length); 
         } 
}); 

Here, you have viewed all cities JSON file, and in each cycle, you'll append a subject index and type of documents to be indexed include. Note that there are two push into an array in a loop? This is because the bulk API need to include an index defined object, then the documents to be indexed. For more information, you can check here .

Next, you will pass to client.bulk function as an array of new batch text calls. This will all data with the index into Elasticsearch scotch.io-tutorial index and type cities_list.

The introduction of Express

Your Elasticsearch instance is up and running, you can use Node.js connect to it. Now it's time to use the Express to provide services to the target page, and use the settings to run so far.

Create a file called index.js and add:

//index.js
// 需要Elasticsearch librray
const elasticsearch = require('elasticsearch');
// 实例化一个elasticsearch客户端
const client = new elasticsearch.Client({
   hosts: [ 'http://localhost:9200']
});
//require Express
const express = require( 'express' );
// 实例化一个表达式的实例并将其保存在一个名为app的常量中
const app     = express();
// 引入body-parser库。将用于解析主体请求
const bodyParser = require('body-parser')
//require the path library
const path    = require( 'path' );

// ping客户端以确保Elasticsearch已启动
client.ping({
     requestTimeout: 30000,
 }, function(error) {
 // 此时，eastic搜索已关闭，请检查您的Elasticsearch服务
     if (error) {
         console.error('elasticsearch cluster is down!');
     } else {
         console.log('Everything is ok');
     }
 });

// 使用bodyparser作为中间件
app.use(bodyParser.json())
// 设置应用程序侦听的端口
app.set( 'port', process.env.PORT || 3001 );
// 设置路径来提供静态文件
app.use( express.static( path.join( __dirname, 'public' )));
// 启用CORS 
app.use(function(req, res, next) {
  res.header("Access-Control-Allow-Origin", "*");
  res.header('Access-Control-Allow-Methods', 'PUT, GET, POST, DELETE, OPTIONS');
  res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
  next();
});

// 定义了基本路线并返回一个名为tempate.html的HTML文件
app.get('/', function(req, res){
  res.sendFile('template.html', {
     root: path.join( __dirname, 'views' )
   });
})

// 定义应该返回弹性搜索结果的/ search路径
app.get('/search', function (req, res){
  // 声明查询对象以搜索弹性搜索，并从找到的第一个结果中仅返回200个结果。
  // 还匹配其中名称与发送的查询字符串类似的任何数据
  let body = {
    size: 200,
    from: 0, 
    query: {
      match: {
          name: req.query['q']
      }
    }
  }
  // 在索引中执行实际的搜索传递，搜索查询和类型
  client.search({index:'scotch.io-tutorial',  body:body, type:'cities_list'})
  .then(results => {
    res.send(results.hits.hits);
  })
  .catch(err=>{
    console.log(err)
    res.send([]);
  });

})
// 监听一个指定的端口
app .listen( app.get( 'port' ), function(){
  console.log( 'Express server listening on port ' + app.get( 'port' ));
} );

Take a look at the code above, note:

• Need Express, body-parser and the path to the library.
• A new set of Express instance constant, named app.
• Setting the application to use bodyParser middleware.
• The static files named on the application of the public folder (I have not yet created this folder).
• It defines a CORS header is added to the application middleware.
• GET define a routing in the root folder, and in this route, I returned a file named template.html, and the file is located in the folder views (I have yet to create the folder and file template.html)
• The application / search URL GET define a route, the route search using the query object to the data passed to it by matching the query string. The main search query included in the query object. You can add different search query to this object. For this query, you add a keyword in the query and returns an object, tell it you are looking for the name of the document should req.query [ 'q'] match.

  Besides the query object, the search body can contain other optional properties, including size and from. The size property determines the number of documents to be included in the response. If this value is not present, by default ten documents are returned. The from property determines the starting index of the returned documents. This is useful for pagination.

Understanding Search API response

If you want to log out Search API response will contain a lot of information.

{ took: 88,
timed_out: false,
_shards: { total: 5, successful: 5, failed: 0 },
hits:
{ total: 59,
 max_score: 5.9437823,
 hits:
  [ {"_index":"scotch.io-tutorial",
  "_type":"cities_list",
  "_id":"AV-xjywQx9urn0C4pSPv",
  "_score":5.9437823,"
  _source":{"country":"ES","name":"A Coruña","lat":"43.37135","lng":"-8.396"}},
    [Object],
...
    [Object] ] } }

The response contains the number of milliseconds for a lookup result TIMED_OUT seizure of properties, if the result is not found within the maximum allowable time, return true; _shards used to obtain information about the status of various nodes (if deployed as nodes in the cluster) and comprising Search the results of the match.

Properties in hits, we have an object with the following properties:

The total number displays the total number of matches.

max_score is the highest score of found items.

Hit contains an array of found items.

These are the premise of search route, you return the response.hits.hits, which contains documents found.

Create HTML Template

First, create two new folder referenced in the section above entitled views and public root file. Next, create a file named template.html in the views folder and paste:

<link rel="stylesheet" type="text/css" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
<script src="https://unpkg.com/axios/dist/axios.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/vue"></script>
<div>
    <div>
        <div>
            <h1>Search Cities around the world</h1>
        </div>
    </div>
    <div>
        <div>
            <form action="" class="search-form">
                <div>
                    <label for="search" class="sr-only">Search</label>
                    <input type="text" class="form-control" name="search" id="search" placeholder="search" v-model="query" >
                    <span></span>
                </div>
            </form>
        </div>
    </div>
    <div>
        <div>
            <div>
                <div>
                
                    {{ result._source.name }}, {{ result._source.country }} 
                </div>
                <div>
                
                    <p>lat:{{ result._source.lat }}, long: {{ result._source.lng }}.</p>
                </div>
            </div>
        </div>
    </div>
</div>

<style>
    .search-form .form-group {
        float: right !important;
        transition: all 0.35s, border-radius 0s;
        width: 32px;
        height: 32px;
        background-color: #fff;
        box-shadow: 0 1px 1px rgba(0, 0, 0, 0.075) inset;
        border-radius: 25px;
        border: 1px solid #ccc;
    }

    .search-form .form-group input.form-control {
        padding-right: 20px;
        border: 0 none;
        background: transparent;
        box-shadow: none;
        display: block;
    }

    .search-form .form-group input.form-control::-webkit-input-placeholder {
        display: none;
    }

    .search-form .form-group input.form-control:-moz-placeholder {
        /* Firefox 18- */
        display: none;
    }

    .search-form .form-group input.form-control::-moz-placeholder {
        /* Firefox 19+ */
        display: none;
    }

    .search-form .form-group input.form-control:-ms-input-placeholder {
        display: none;
    }

    .search-form .form-group:hover,
    .search-form .form-group.hover {
        width: 100%;
        border-radius: 4px 25px 25px 4px;
    }

    .search-form .form-group span.form-control-feedback {
        position: absolute;
        top: -1px;
        right: -2px;
        z-index: 2;
        display: block;
        width: 34px;
        height: 34px;
        line-height: 34px;
        text-align: center;
        color: #3596e0;
        left: initial;
        font-size: 14px;
    }
</style>

In the above code fragment, it has two main parts:

• HTML code: In this section, you first need three different libraries, namely 1.) Bootstrap CSS, used to set the page style. 2.) Axios js, for sending HTTP requests to our server, and 3) Vue.js, you will use a simple frame our view.
• CSS code: Here you will hover over the search icon style to hide and display the search input.

Next, specify the v model for your query in the search box has an input (which will Vue.js use). After this, you loop through all the results (this cycle and outcome variables will Vue.js provided). Please note that during this cycle, you must access __source attribute data. Based on the elastic response returned by the search, it looks very familiar.

Run node index.js command, browse to http: // localhost: 3001 /, Next, add a script tag in your template.html file, add:

// template.html
// 创建一个新的Vue实例
var app = new Vue({
    el: '#app',
    // 声明组件的数据（容纳结果的数组以及包含当前搜索字符串的查询） search string)
    data: {
        results: [],
        query: ''
    },
    // 在这个Vue组件中声明方法。这里只定义了一种执行搜索的方法
    methods: {
        // 使用当前搜索查询向服务器发出axios请求
        search: function() {
            axios.get("http://127.0.0.1:3001/search?q=" + this.query)
                .then(response => {
                    this.results = response.data;

                })
        }
    },
    // declare Vue watchers
    watch: {
        // 注意查询字符串中的更改并调用搜索方法
        query: function() {
            this.search();
        }
    }

})

Vue.js Code: In this section, you declare a new instance of the Vue, mount it on the element with the ID of the application. You represent the data attributes, including 1) appended to the search query that you have input, and 2) result, which is an array of all the results found.

In the method of configuration, only a search function is called, it will trigger the GET request in the search path, the search box to deliver current input. Then returns a response, and then recycled in the HTML code block.

Finally, you use Vue.js the so-called observer, at any time surveillance data to see the changes. Here you are looking at changes to the query data, and once it is changed, it will trigger a search method.

Search from clients

Each time a search happen if I do not want to send a request to the server, how to do? I can search directly from the client Elasticsearch engine? Yes.

Although the above method is effective, but some developers may not be accustomed to each search conditions using their servers, while others believe that the search more secure from the server.

However, you can search from a client. Elasticsearch provides a browser version can be searched. Let me through a quick example.

First, add a new route to the Express file and restart the server:

//index.js
// decare a new route. This route serves a static HTML template called template2.html
app.get('/v2', function(req, res){
  res.sendFile('template2.html', {
     root: path.join( __dirname, 'views' )
   });
})

In the above code block, you create / v2 routing a new URL, and all your actions in this route will return a static HTML file named template2.html, the document will be created soon .

Next, you need here to download Elasticsearch client library. Once downloaded, the elasticsearch.min.js extracted and copied to the root directory of the application folder.

Note: Find out if you try to connect Elasticsearch engine from the client is very important, you may experience problems CORS. To solve this problem, find your Elasticsearch configuration file (for Debian / Ubuntu, you can find it in /etc/elasticsearch/elasticsearch.yml). For other operating systems, find where it is located, in the end section and add the following files:

#/etc/elasticsearch/elasticsearch.yml

http.cors.enabled : true
http.cors.allow-origin : "*"

When finished, restart the instance Elasticsearch

// 重新启动Elasticsearch服务
sudo service elasticsearch restart

Next, create a file named template2.html in the view folder and add:

<link rel="stylesheet" type="text/css" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
<script src="https://unpkg.com/axios/dist/axios.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/vue"></script>
<div>
    <div>
        <div>
            <h1>Search Cities around the world</h1>
        </div>
    </div>
    <div>
        <div>
            <form action="" class="search-form">
                <div>
                    <label for="search" class="sr-only">Search</label>
                    <input type="text" class="form-control" name="search" id="search" placeholder="search" v-model="query" >
                    <span></span>
                </div>
            </form>
        </div>
    </div>
    <div>
        <div>
            <div>
                <div>
                
                    {{ result._source.name }}, {{ result._source.country }} 
                </div>
                <div>
                
                    <p>lat:{{ result._source.lat }}, long: {{ result._source.lng }}.</p>
                </div>
            </div>
        </div>
    </div>
</div>
<script src="/elasticsearch.min.js"></script>
<style>
    .search-form .form-group {
        float: right !important;
        transition: all 0.35s, border-radius 0s;
        width: 32px;
        height: 32px;
        background-color: #fff;
        box-shadow: 0 1px 1px rgba(0, 0, 0, 0.075) inset;
        border-radius: 25px;
        border: 1px solid #ccc;
    }

    .search-form .form-group input.form-control {
        padding-right: 20px;
        border: 0 none;
        background: transparent;
        box-shadow: none;
        display: block;
    }

    .search-form .form-group input.form-control::-webkit-input-placeholder {
        display: none;
    }

    .search-form .form-group input.form-control:-moz-placeholder {
        /* Firefox 18- */
        display: none;
    }

    .search-form .form-group input.form-control::-moz-placeholder {
        /* Firefox 19+ */
        display: none;
    }

    .search-form .form-group input.form-control:-ms-input-placeholder {
        display: none;
    }

    .search-form .form-group:hover,
    .search-form .form-group.hover {
        width: 100%;
        border-radius: 4px 25px 25px 4px;
    }

    .search-form .form-group span.form-control-feedback {
        position: absolute;
        top: -1px;
        right: -2px;
        z-index: 2;
        display: block;
        width: 34px;
        height: 34px;
        line-height: 34px;
        text-align: center;
        color: #3596e0;
        left: initial;
        font-size: 14px;
    }
</style>

Next, add a script tag in your template2.html file and add:

//template2.html
// 像你在客户端上那样实例化一个新的Elasticsearch客户端
var client = new elasticsearch.Client({
    hosts: ['http://127.0.0.1:9200']
});
// 创建一个新的Vue实例
var app = new Vue({
    el: '#app',
    // 声明组件的数据（容纳结果的数组以及包含当前搜索字符串的查询）
    data: {
        results: [],
        query: ''
    },
    // 在这个Vue组件中声明方法。这里只定义了一种执行搜索的方法
    methods: {
        // 函数调用弹性搜索。这里查询对象与服务器的设置一样。
        // 这里查询字符串直接从Vue传递
        search: function() {
            var body = {
                    size: 200,
                    from: 0,
                    query: {
                        match: {
                            name: this.query
                        }
                    }
                }
                // 搜索传入索引的Elasticsearch，查询对象和类型
            client.search({ index: 'scotch.io-tutorial', body: body, type: 'cities_list' })
                .then(results => {
                    console.log(found ${results.hits.total} items in ${results.took}ms);
                    // 将结果设置为我们拥有的结果数组
                    this.results = results.hits.hits;
                })
                .catch(err => {
                    console.log(err)

                });

        }
    },
    // declare Vue watchers
    watch: {
        // 注意查询字符串中的更改并调用搜索方法
        query: function() {
            this.search();
        }
    }

})

The above HTML and JavaScript fragment is very similar to the above section. The only difference is:

• You do not need Axios, but need elasticsearch.js.
• At the top of the script tag, you start Elasticsearch client because it is done on the server side.
• The method of HTTP requests not perform the search, but as search engines that search path Elasticsearch server side.

Read the original: HTTPS: //scotch.io/tutorials/b ...