API gateway architecture practice under the cloud native: Kong (three)

In the previous article describes Kong-related practices, links to articles , this article will introduce Kong weapon: plug-ins and custom plug-ins.

Application of several commonly used plug-ins Kong

Request arrives Kong, before forwarding the application to the server, we can apply Kong comes with plug-ins to process the request, such as legal certification, limiting control, black and white lists and check the log collection and so on. At the same time, we can follow the tutorial documentation Kong, and develop their own custom plug-ins. Two plug Examples This section will select the application, the rest of the plug-in application can be found: docs.konghq.com/hub/.

JWT authentication plugin

JWT is the most popular cross-domain authentication solutions. As an open standard (RFC 7519), defines a simple, self-contained method for communication between both objects as JSON secure transmission of information. Because there is a digital signature, this information is credible.

About Why JWT, are not discussed in detail in this section, concrete and tangible unified authentication and authorization design and combat service in the micro-architecture . Providing JWT Kong authentication plugin for JWT request (such as described in RFC 7519) to verify the signature comprising HS256 or RS256. Every consumer will have JWT credentials (public and private keys), these certificates must be used to sign their JWT. JWT token may, cookie or authentication request transmitted through the string head. Kong will verify the signature of the token by forwarding, otherwise discards the request.

We routing configuration on the basis of the previous section, an increase JWT authentication plugin.

curl -X POST http://localhost:8001/routes/e33d6aeb-4f35-4219-86c2-a41e879eda36/plugins \
--data "name=jwt"
复制代码

It can be seen in the list of plugins adds a corresponding record.

After an increase of JWT plug-in, would not be able to directly access /api/blogthe interface, the interface has returned: "message": "Unauthorized". It prompts the client to access the JWT need to provide authentication information. Therefore, we need to create a user:

curl -i -X POST \
--url http://localhost:8001/consumers/  \
--data "username=aoho"
复制代码

As creates a user named aoho of.

After creating a user, you need to obtain user credentials JWT, perform the following call:

curl -i -X POST \
--url http://localhost:8001/consumers/aoho/jwt \
--header "Content-Type: application/x-www-form-urlencoded"

// 响应
{
	"rsa_public_key": null,
	"created_at": 1563566125,
	"consumer": {
		"id": "8c0e1ab4-8411-42fc-ab80-5eccf472d2fd"
	},
	"id": "1d69281d-5083-4db0-b42f-37b74e6d20ad",
	"algorithm": "HS256",
	"secret": "olsIeVjfVSF4RuQuylTMX4x53NDAOQyO",
	"key": "TOjHFM4m1qQuPPReb8BTWAYCdM38xi3C"
}
复制代码

Using the secret key and https://jwt.iocan generate JWT credentials. In actual use, we pass encoding to achieve, here to demonstrate the use of web tools to generate Token.

The Token generated, the authentication header of the request to configure, re-execution request:

You can see, we are able to normal for the appropriate API interface. JWT authentication plug-in application is successful.

Prometheus visual monitoring

Prometheus is an open source framework alarm monitoring system. It is inspired by Google's borgmon monitoring system, created in 2012 by a former employee google SoundCloud's work, developed as an open source project community, and released in 2015. 2016, Prometheus officially joined the Cloud Native Computing Foundation, the project has become second only to the popularity of Kubernetes. As a new generation of monitoring framework, Prometheus is suitable for recording time series data, a powerful multi-dimensional data model, flexible and powerful query, easy to manage and scalable features.

Prometheus plug Kong official, usable metric as follows:

Status code: upstream HTTP status code returned by the service;
Histogram delay: the delay Kong will be recorded, comprising:
- Request: delay complete request;
- Kong: Kong to time routing, authentication and other plug-ins to run it takes;
- Upstream: upstream service takes time to respond to the request.
Bandwidth: The total bandwidth flows Kong (outlet / inlet);
DB Accessibility: Are Kong nodes can access its DB;
Connections: NGINX connecting various indicators, such as the Active, read, write, accept the connection.

We installed plug-ins for Prometheus on aoho-blog service in Service:

curl -X POST http://localhost:8001/services/aoho-blog/plugins \
--data "name=prometheus"
复制代码

You can see from the management interface, we have successfully bind Prometheus plug-in to the aoho-blog service.

By accessing /metricsthe interface returns to collect metrics:

$ curl -i http://localhost:8001/metrics
HTTP/1.1 200 OK
Server: openresty/1.13.6.2
Date: Sun, 21 Jul 2019 09:48:42 GMT
Content-Type: text/plain; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Access-Control-Allow-Origin: *

kong_bandwidth{type="egress",service="aoho-blog"} 178718
kong_bandwidth{type="ingress",service="aoho-blog"} 1799
kong_datastore_reachable 1
kong_http_status{code="200",service="aoho-blog"} 4
kong_http_status{code="401",service="aoho-blog"} 1

kong_latency_bucket{type="kong",service="aoho-blog",le="00005.0"} 1
kong_latency_bucket{type="kong",service="aoho-blog",le="00007.0"} 1
...
kong_latency_bucket{type="upstream",service="aoho-blog",le="00300.0"} 4
kong_latency_bucket{type="upstream",service="aoho-blog",le="00400.0"} 4
...
kong_latency_count{type="kong",service="aoho-blog"} 5
kong_latency_count{type="request",service="aoho-blog"} 5
kong_latency_count{type="upstream",service="aoho-blog"} 4
kong_latency_sum{type="kong",service="aoho-blog"} 409
kong_latency_sum{type="request",service="aoho-blog"} 1497
kong_latency_sum{type="upstream",service="aoho-blog"} 1047

kong_nginx_http_current_connections{state="accepted"} 2691
kong_nginx_http_current_connections{state="active"} 2
kong_nginx_http_current_connections{state="handled"} 2691
kong_nginx_http_current_connections{state="reading"} 0
kong_nginx_http_current_connections{state="total"} 2637
kong_nginx_http_current_connections{state="waiting"} 1
kong_nginx_http_current_connections{state="writing"} 1

kong_nginx_metric_errors_total 0
复制代码

Returns the response is too long, there is omitted, you can see from the response provided by the plug metric Prometheus is reflected. Prometheus plug-derived metrics can be plotted in Grafana, the reader can try on their own.

Link Trace plug Zipkin

Zipkin is an open source distributed real-time data tracking system. Its main function is to gather real-time monitoring data from various heterogeneous systems, systems to track the delay problem of micro-services architecture. Applications required to report data to Zipkin. The plug-ins as Zipkin Kong zipkin-client is required Zipkin assembled packets to the transmission data Zipkin-server. Zipkin plug request will be marked as tags, and pushed to the server Zipkin:

span.kind (sent to Zipkin as “kind”)
http.method
http.status_code
http.url
peer.ipv4
peer.ipv6
peer.port
peer.hostname
peer.service

Specific information about the link tracking and Zipkin, see the full link Detailed tracking micro-service architecture , this chat is intended to introduce how to use Zipkin plug-track link of all requests in the Kong.

First open Zipkin plug-in, plug-in to bind the routing (where you can bind a global plugin).

curl -X POST http://kong:8001/routes/e33d6aeb-4f35-4219-86c2-a41e879eda36/plugins \
    --data "name=zipkin"  \
    --data "config.http_endpoint=http://localhost:9411/api/v2/spans" \
    --data "config.sample_ratio=1"
复制代码

Zipkin Collector arranged above the sampling rate and the address, for obvious effect, provided the sampling rate is 100%, the production environment used with caution, sample rate impact on system throughput.

You can see, Zipkin plug-in has been applied to the specified route. Here we will request the implementation of /api/blogthe interface, open http://localhost:9411interface is as follows:

Zipkin has requested records, we can point to open view detailed link details:

From the link calls can know, after the request arrives Kong, which experienced the service and Span, Span takes time, and so each message.

Since the practice of custom plug-ins

Although the official provides many plug-ins, but we also have operations in real business scenarios demand, custom plug-ins can help us better manage the API Gateway. Kong development kit provides plug and examples, only custom plug can follow the steps provided.

Installation Kong

In the section above, the author describes the installation Kong by mirroring the way, this section for the convenience of writing custom plug-ins, we use locally installed Kong, the author of the environment is macOS, installation is relatively simple:

 $ brew tap kong/kong
 $ brew install kong
复制代码

Second install Postgres, and download kong.conf.default configuration file (see raw.githubusercontent.com/Kong/kong/m...

 $ sudo mkdir -p /etc/kong
 $ sudo cp kong.conf.default /etc/kong/kong.conf
复制代码

Implementation of migration:

kong migrations bootstrap -c /etc/kong/kong.conf
复制代码

You can then start Kong:

kong start -c /etc/kong/kong.conf
复制代码

After the start, success through the 8001 management port authentication.

curl -i http://localhost:8001/
复制代码

Based on installed Kong, we explain how to customize the plug-ins added to the optional plug-ins Kong's here to token-auth authentication plug-in as an example to explain.

Kong official provides information on certification plugin: JWT, OAuth 2.0 and Basic Auth, etc., in our actual business, but also often self authentication and authorization server, so we need to verify the legitimacy of the request to intercept the API gateway. Based on this, we implemented a similar filter plug Kong: token-auth.

Kong comes with plug-in /usr/local/share/lua/5.1/kong/plugins/directory. Each plug-in folder has the following two main documents:

schema.lua: Check the parameters when starting the plug-defined;
handler.lua: file defines the function of each stage of the implementation of core plug-ins.

token-auth is our custom plug-in name. In /usr/local/share/lua/5.1/kong/pluginscase the new token-auth directory. Plugin loading and the initialization phase, ie Kong.init()when loading the plug, the plug will be in the directory schema.lua and handler.lua loaded, let's look to achieve these two scripts.

Plug-in configuration definitions: schema.lua

Kong configuration of each widget plugins stored in the config table fields, is a piece of text json, token-auth desired configuration defined as follows:

return {
  no_consumer = true,
  fields = {
    auth_server_url = {type = "url", required = true},
  }
}
复制代码

When you can see from schema.lua, enable token-auth plugin, you need to check auth_server_url URL field type, and can not be null.

Plug-in functions to achieve: handler.lua

handler.lua achieve a plug-in authentication method defined in the plug-in, it will be called when processing requests and responses.

llocal http = require "socket.http"
local ltn12 = require "ltn12"
local cjson = require "cjson.safe"

local BasePlugin = require "kong.plugins.base_plugin"

local TokenAuthHandler = BasePlugin:extend()

TokenAuthHandler.PRIORITY = 1000

local KEY_PREFIX = "auth_token"
local EXPIRES_ERR = "token expires"

--- 提取 JWT 头部信息
-- @param request    ngx request object
-- @return token     JWT
-- @return err
local function extract_token(request)
  local auth_header = request.get_headers()["authorization"]
  if auth_header then
    local iterator, ierr = ngx.re.gmatch(auth_header, "\\s*[Bb]earer\\s+(.+)")
    if not iterator then
      return nil, ierr
    end

    local m, err = iterator()
    if err then
      return nil, err
    end

    if m and #m > 0 then
      return m[1]
    end
  end
end

--- 调用 auth server 验证 token 合法性
-- @param token    Token to be validated
-- @param conf     Plugin configuration
-- @return info    Information associated with token
-- @return err
local function query_and_validate_token(token, conf)
  ngx.log(ngx.DEBUG, "get token info from: ", conf.auth_server_url)
  local response_body = {}
  local res, code, response_headers = http.request{
    url = conf.auth_server_url,
    method = "GET",
    headers = {
      ["Authorization"] = "bearer " .. token
    },
    sink = ltn12.sink.table(response_body),
  }

  if type(response_body) ~= "table" then
    return nil, "Unexpected response"
  end
  local resp = table.concat(response_body)
  ngx.log(ngx.DEBUG, "response body: ", resp)

  if code ~= 200 then
    return nil, resp
  end

  local decoded, err = cjson.decode(resp)
  if err then
    ngx.log(ngx.ERR, "failed to decode response body: ", err)
    return nil, err
  end

  if not decoded.expires_in then
    return nil, decoded.error or resp
  end

  if decoded.expires_in <= 0 then
    return nil, EXPIRES_ERR
  end

  decoded.expires_at = decoded.expires_in + os.time()
  return decoded
end

function TokenAuthHandler:new()
  TokenAuthHandler.super.new(self, "token-auth")
end
--- 实现 access 方法
function TokenAuthHandler:access(conf)
  TokenAuthHandler.super.access(self)

  local token, err = extract_token(ngx.req)
  if err then
    ngx.log(ngx.ERR, "failed to extract token: ", err)
    return kong.response.exit(500, { message = err })
  end
  ngx.log(ngx.DEBUG, "extracted token: ", token)

  local ttype = type(token)
  if ttype ~= "string" then
    if ttype == "nil" then
      return kong.response.exit(401, { message = "Missing token"})
    end
    if ttype == "table" then
      return kong.response.exit(401, { message = "Multiple tokens"})
    end
    return kong.response.exit(401, { message = "Unrecognized token" })
  end

  local info
  info, err = query_and_validate_token(token, conf)

  if err then
    ngx.log(ngx.ERR, "failed to validate token: ", err)
    if EXPIRES_ERR == err then
      return kong.response.exit(401, { message = EXPIRES_ERR })
    end
    return kong.response.exit(500,{ message = EXPIRES_ERR })
  end

  if info.expires_at < os.time() then
    return kong.response.exit(401, { message = EXPIRES_ERR })
  end
  ngx.log(ngx.DEBUG, "token will expire in ", info.expires_at - os.time(), " seconds")

end

return TokenAuthHandler
复制代码

token-auth plugin implements a new () and access () method two only play a role in the access phase. In access () method, first extracts JWT header information, check the token exists and the correct format, etc., and then requests the authentication server to verify the legitimacy of the token.

Load plugins

After the completion of plug-in development, we must first create a new token-auth-1.2.1-0.rockspec file in the plugin directory, fill the newly developed plug-ins:

package = "token-auth"
version = "1.2.1-0"

supported_platforms = {"linux", "macosx"}

local pluginName = "token-auth"
build = {
  type = "builtin",
  modules = {
    ["kong.plugins.token-auth.handler"] = "kong/plugins/token-auth/handler.lua",
    ["kong.plugins.token-auth.schema"] = "kong/plugins/token-auth/schema.lua",
  }
}
复制代码

Then add the newly developed plug-in kong.conf configuration file:

$ vim /etc/kong/kong.conf

# 去掉开头的注释并修改如下
plugins = bundled, token-auth
复制代码

bundled attribute refers to a collection of plug-ins provided by the official, turned on by default. Here, we have added token-auth plugin custom. Verify whether a custom plug-in is successfully loaded:

$ curl http://127.0.0.1:8001/plugins/enabled


{"enabled_plugins":["correlation-id","pre-function","cors","token-auth","ldap-auth","loggly","hmac-auth","zipkin","request-size-limiting","azure-functions","request-transformer","oauth2","response-transformer","ip-restriction","statsd","jwt","proxy-cache","basic-auth","key-auth","http-log","datadog","tcp-log","post-function","prometheus","acl","kubernetes-sidecar-injector","syslog","file-log","udp-log","response-ratelimiting","aws-lambda","bot-detection","rate-limiting","request-termination"]}%
复制代码

Enable plug-ins

You enable token-auth plug-in on Service, also need to specify the attributes config.auth_server_url:

$ curl -i -XPOST localhost:8001/services/aoho-blog/plugins \
    --data 'name=token-auth' \
    --data 'config.auth_server_url=<URL of verification API>'
复制代码

If the plug-in has its own database table, or there are demands on database tables or data tables, creating migrations directory in the plugin directory. According to using Postgres or Cassandra, create migrations / postgres.lua or migrations / cassandra.lua.

If the plug-in has its own database tables, but also need to create a plug-in directory daos.lua, returns the database table definition, if there is no separate database table, you do not need to create this file.

Do not make too many presentations, readers can be combined chat before the author: unified authentication and authorization design and combat service in the micro-architecture , authentication and authorization server build, own try.

summary

Gateway is an integral part of the micro-service architecture basic services, this article describes how to use the micro Kong to build the services gateway. Compared to other gateway component, Kong in terms of ease of use and performance of outstanding performance, it is a modern cloud native gateway. Then he introduced the use of partial plug Kong. Kong official and community provides a rich API Gateway plugin configuration can be used. Finally, I realized the token-auth plugin in a custom text, Kong open plug-in mechanism, allows developers the flexibility to achieve specific business needs.