presto代理层实现

用过presto的同学都知道，客服端访问presto集群，其实是通过http协议访问的presto的coordinator节点；大多数情况是没啥问题；可是服务器直接暴露给用户，或多或少还是有些安全顾虑；所以我们在客户端和presto集群之间加了一层访问代理，如下图所示
在这里插入图片描述

所有的用户请求通过代理访问presto集群；过程如下
1.客户端可以使用jdbc，Python等方式访问代理，使用方式和直接访问集群方式一样，只不过以前的host是coordinator，而现在是presot-proxy
2.代理获得客户端的请求后封装post请求访问presto集群，presto集群响应请求的报文如下，代理获得presto集群的响应报文后，返回给客户端

{
    "id":"20181107_132829_00014_u28jg",
    "infoUri":"http://192.168.120.4:8099/ui/query.html?20181107_132829_00014_u28jg",
    "nextUri":"http://192.168.120.4:8099/v1/statement/20181107_132829_00014_u28jg/1",
    "stats":{
        "state":"QUEUED",
        "queued":true,
        "scheduled":false,
        "nodes":0,
        "totalSplits":0,
        "queuedSplits":0,
        "runningSplits":0,
        "completedSplits":0,
        "cpuTimeMillis":0,
        "wallTimeMillis":0,
        "queuedTimeMillis":0,
        "elapsedTimeMillis":0,
        "processedRows":0,
        "processedBytes":0,
        "peakMemoryBytes":0
    }
}

3.客户端获得报文后，根据报文中nextUri直接发送GET请求给presto集群的coordinator节点
4.coordinator节点负责分发任务给node，计算并返回结果

我们从如上过程，可以看到，其实presto-proxy只是处理client的第一次请求，并返回报文；剩下的所有请求都是client根据报文中的nextUri直接发送GET请求给presto集群；对于用户来说，他们能看到的只是步骤1，其他的步骤对用户来说都是透明的

现在来实现一下这个proxy，其实很简单，核心代码也就十几行

package com.fan.prestoproxy.controller;

import org.apache.commons.io.IOUtils;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.ResponseBody;
import javax.servlet.http.HttpServletRequest;
import java.io.BufferedReader;
import java.io.InputStream;
import java.util.Calendar;
import java.util.Enumeration;
import java.util.Locale;

@Controller
@RequestMapping("/v1")
public class QueryController {

    @ResponseBody
    @RequestMapping(value= "statement",method = RequestMethod.POST,produces="application/json;charset=UTF-8")
    public String getResponse(HttpServletRequest request ) throws Exception {
        DefaultHttpClient hc = new  DefaultHttpClient();  //初始化一个HTTP的客户端对象
        HttpPost httppost = new HttpPost("http://192.168.120.4:8099/v1/statement");
        BufferedReader reader = request.getReader();
        String statement = reader.readLine();
        httppost.setHeader("X-Presto-Source", request.getHeader("x-presto-source"));
        httppost.setHeader("X-Presto-User", request.getHeader("x-presto-user"));
        httppost.setHeader("User-Agent", request.getHeader("user-agent"));
        httppost.setHeader("X-Presto-Time-Zone", Calendar.getInstance().getTimeZone().getID());
        httppost.setHeader("X-Presto-Language", Locale.CHINA.getLanguage());

        Enumeration<String> headerNames = request.getHeaderNames();
        while(headerNames.hasMoreElements()){
            String key = headerNames.nextElement();
            System.out.println(key+"   "+request.getHeader(key));
        }

        httppost.setEntity(new StringEntity(statement));
        HttpResponse httpresponse = hc.execute(httppost);
        InputStream is = httpresponse.getEntity().getContent();
        String body = IOUtils.toString(is, "utf-8");
        System.out.println(body);
        return body;
    }

}

http://192.168.120.4:8099/v1/statement也就是presto集群对外提供的接口；测试代码如下，就是普通的jdbc代码，只不过jdbc的host指向的是proxy的host

 public static void jdbc()throws  Exception{
        Class.forName("com.facebook.presto.jdbc.PrestoDriver");
        String url = "jdbc:presto://localhost:8080";
        Connection connection = DriverManager.getConnection(url, "user", null);
        Statement statement = connection.createStatement();
        long begin = System.currentTimeMillis();
        String sql = "select *  from mysql.shiro.sh_user";
        ResultSet rs = statement.executeQuery(sql) ;

        long end = System.currentTimeMillis();
        System.out.println((end - begin) + " ms ");

        while(rs.next()){
            System.out.println(rs.getString("name")+ "," + rs.getString("passwd"));
        }
        statement.close();
        connection.close();
    }

代理的作用不局限于此，其实在代理层我们可以实现自己的数据访问权限，如果用户的请求不合规，我们也可以直接在代理层抛出异常

end

猜你喜欢