Project Analysis NiterForum (3) Third-Party Services and Cache

third party service

Ali

Mainly to get information

OkHttpClientMake a request by using : Detailed use of OkhttpClient_workingman_li's Blog-CSDN Blog

<dependency>
    <groupId>com.squareup.okhttp3</groupId>
    <artifactId>okhttp</artifactId>
    <version>3.14.1</version>
</dependency>

BaiduCloud

Mainly Baidu Cloud's content review service:

// 初始化一个AipImageCensor
AipContentCensor client = new AipContentCensor(APP_ID, API_KEY, SECRET_KEY);
JSONObject response = client.textCensorUserDefined(text);

Sign in with

AuthorizeControllerRead the configuration file and write the callback function in

Baidu

To call Baidu service, you need a Baidu access token:

Similarly, OkHttpClientsend

MediaType mediaType = MediaType.get("application/x-www-form-urlencoded; charset=utf-8"); // 媒体类型，决定浏览器将以什么形式、什么编码对资源进行解析
OkHttpClient client = new OkHttpClient();
String s = "grant_type="+baiduAccessTokenDTO.getGrant_type()+"&code="+baiduAccessTokenDTO.getCode()+"&client_id="+baiduAccessTokenDTO.getClient_id()+"&client_secret="+baiduAccessTokenDTO.getClient_secret()+"&redirect_uri="+baiduAccessTokenDTO.getRedirect_uri(); // baiduAccessTokenDTO含有配置文件中的百度登录API配置信息
RequestBody body = RequestBody.create(mediaType, s); // 准备请求体
Request request = new Request.Builder()
    .url("https://openapi.baidu.com/oauth/2.0/token")
    .post(body)
    .build();
Response response = client.newCall(request).execute(); // 发送请求，返回带有access token的json

Then send the request with the obtained access token:

OkHttpClient client = new OkHttpClient();
Request request = new Request.Builder()
	.url("https://openapi.baidu.com/rest/2.0/passport/users/getInfo?access_token=" + accessToken)
	.build();
Response response = client.newCall(request).execute(); // 返回带有个人信息的BaiduUserDTO对象

Github

Similar to Baidu, but the obtained access token is placed in the header to send the request:

OkHttpClient client = new OkHttpClient();
/*
*使用参数的方式明文传输，并不推荐，即将被Github废弃
Request request = new Request.Builder()
    .url("https://api.github.com/user?access_token=" + accessToken)
    .build();
*/

//作为header中的参数传输，强烈推荐
Request request = new Request.Builder()
    .url("https://api.github.com/user")
    .header("Authorization","token "+accessToken)
    .build();

Qq

Get the access token first, then get the openID, and finally get the user information through the access token, openID and the clientId connected by qq

Developers can obtain basic information of users through openID. In particular, if the developer has multiple mobile applications and website applications, the uniqueness of the user can be distinguished by obtaining the user's unionID, because as long as they are different applications under the same QQ Internet platform, the unionID is the same. In other words, for the same user, the unionID is the same for different applications under the same QQ Internet platform

public String getOpenID(String accessToken) {
    
    
        OkHttpClient client = new OkHttpClient();
        Request request = new Request.Builder()
                .url("https://graph.qq.com/oauth2.0/me?access_token=" + accessToken)
                .build();
        try {
    
    
            Response response = client.newCall(request).execute();
            String string = response.body().string();
            String jsonString = string.split(" ")[1].split(" ")[0];
            //System.out.println(jsonString);
            JSONObject obj = JSONObject.parseObject(jsonString);
            String openid = obj.getString("openid");
            //System.out.println(openid);
            return openid;
        } catch (IOException e) {
    
    
        }
        return null;
    }


    public String getUnionId(String accessToken) {
    
    
        OkHttpClient client = new OkHttpClient();
        Request request = new Request.Builder()
                .url("https://graph.qq.com/oauth2.0/me?access_token="+accessToken+"&unionid=1")
                .build();
        try {
    
    
            Response response = client.newCall(request).execute();
            String string = response.body().string();
            System.out.println("rs:"+string);
            String jsonString = string.split(" ")[1].split(" ")[0];
            System.out.println(jsonString);
            JSONObject obj = JSONObject.parseObject(jsonString);
            String unionid = obj.getString("unionid");
            System.out.println(unionid);
            return unionid;
        } catch (IOException e) {
    
    
        }
        return null;
    }

Weibo

Same as Baidu

JiGuang

Mainly Jiguang SMS service

After the user enters the mobile phone number, the mobile phone number, ip, and login token are sent to the backend PhoneController#getPhoneCodemethod

First JiGuangProvider#testSendSMSCodesend a verification code SMS (configuration is required), and then process the sent result (determine whether the sending is successful)

After that, the user submits the verification code, enters PhoneController#validCodethe method , calls JiGuangProvider#testSendValidSMSCodethe method, sends a verification request smsclient.sendValidSMSCode(msg_id, code), and judges and processes the status of the verification code submitted by the user after obtaining the result (bind the mobile phone or register and log in)

When the user chooses Jiguang one-key login, enter PhoneController#loginTokenVerifythe method JiGuangProvider#loginTokenVerify, call it, this method sends an http request to call Jiguang's one-key login api, sends the login token passed in from the front end, the api returns the encrypted mobile phone number after verification, and then calls the JiGuangProvider#decryptright which decrypts it using PKCS8EncodedKeySpec

JAVA——RSA encryption [X509EncodedKeySpec, PKCS8EncodedKeySpec, RSAPublicKeySpec, RSAPrivateKeySpec] - Starzkg's Blog - CSDN Blog

QCloud

Tencent Cloud Service

COS

Called when an image is uploaded:

String url = qCloudProvider.upload(inputStream,contentType,user,contentLength);

In upload, uploadtoBucket is called to store the file in the Bucket:

public String uploadtoBucket(InputStream inputStream, String fileType, String contentType, UserDTO user, String fileName, Long contentLength){
    
    

    // 1 初始化用户身份信息(secretId, secretKey)
    COSCredentials cred = new BasicCOSCredentials(secretId, secretKey);
    // 2 设置bucket的区域, COS地域的简称请参照 https://www.qcloud.com/document/product/436/6224
    ClientConfig clientConfig = new ClientConfig(new Region(region));
    // 3 生成cos客户端
    COSClient cosclient = new COSClient(cred, clientConfig);

    String key = "upload/user/"+user.getId()+"/"+fileType+"/"+fileName; // bucket中路径

    ObjectMetadata objectMetadata = new ObjectMetadata();
    // 从输入流上传必须制定content length, 否则http客户端可能会缓存所有数据，存在内存OOM的情况
    objectMetadata.setContentLength(contentLength);
    // 默认下载时根据cos路径key的后缀返回响应的contenttype, 上传时设置contenttype会覆盖默认值
    objectMetadata.setContentType("contentType");

    PutObjectRequest putObjectRequest =
    	new PutObjectRequest(bucketName, key, inputStream, objectMetadata);
    // 设置存储类型, 默认是标准(Standard), 低频(standard_ia)
    putObjectRequest.setStorageClass(StorageClass.Standard);
    try {
    
    
        PutObjectResult putObjectResult = cosclient.putObject(putObjectRequest);
        // putobjectResult会返回文件的etag
        String etag = putObjectResult.getETag();
        //System.out.println(etag);
    } catch (CosServiceException e) {
    
    
        //e.printStackTrace();
        log.error("upload error,{}", key, e);
        throw new CustomizeException(CustomizeErrorCode.FILE_UPLOAD_FAIL);
    } catch (CosClientException e) {
    
    
        //e.printStackTrace();
        log.error("upload error,{}", key, e);
        throw new CustomizeException(CustomizeErrorCode.FILE_UPLOAD_FAIL);
    }
    // 关闭客户端
    cosclient.shutdown();
    return objecturl+key;
}

Tencent SMS

public String sendSms(String session, String phone){
    
    

    try {
    
    
        /* 必要步骤：
        * 实例化一个认证对象，入参需要传入腾讯云账户密钥对 secretId 和 secretKey
        * 本示例采用从环境变量读取的方式，需要预先在环境变量中设置这两个值
        * CAM 密钥查询：https://console.cloud.tencent.com/cam/capi*/
        Credential cred = new Credential(secretId, secretKey);

        /* 非必要步骤:
        * 实例化一个客户端配置对象，可以指定超时时间等配置 */
        HttpProfile httpProfile = new HttpProfile();
        ClientProfile clientProfile = new ClientProfile();
        
        /* SDK 默认用 TC3-HMAC-SHA256 进行签名
        * 非必要请不要修改该字段 */
        clientProfile.setSignMethod("HmacSHA256");
        clientProfile.setHttpProfile(httpProfile);
        
        /* 实例化 SMS 的 client 对象
        * 第二个参数是地域信息，可以直接填写字符串 ap-guangzhou，或者引用预设的常量 */
        SmsClient client = new SmsClient(cred, "",clientProfile);
        
        /* 实例化一个请求对象，根据调用的接口和实际情况，可以进一步设置请求参数
        * 您可以直接查询 SDK 源码确定接口有哪些属性可以设置
        * 属性可能是基本类型，也可能引用了另一个数据结构*/
        SendSmsRequest req = new SendSmsRequest();

        /* 填充请求参数，这里 request 对象的成员变量即对应接口的入参
        * 您可以通过官网接口文档或跳转到 request 对象的定义处查看请求参数的定义
        * 基本类型的设置:
        * 帮助链接：
        * 短信控制台：https://console.cloud.tencent.com/smsv2
        * sms helper：https://cloud.tencent.com/document/product/382/3773 */

        /* 短信应用 ID: 在 [短信控制台] 添加应用后生成的实际 SDKAppID，例如1400006666 */
        //String appid = "1400009099";
        req.setSmsSdkAppid(smsAppId);

        /* 短信签名内容: 使用 UTF-8 编码，必须填写已审核通过的签名，可登录 [短信控制台] 查看签名信息 */
        //String sign = "签名内容";
        req.setSign(smsSign);

        /* 用户的 session 内容: 可以携带用户侧 ID 等上下文信息，server 会原样返回 */
        //String session = "xxx";
        req.setSessionContext(session);

        /* 模板 ID: 必须填写已审核通过的模板 ID，可登录 [短信控制台] 查看模板 ID */
        //String templateID = "400000";
        req.setTemplateID(smsTempId);

        /* 下发手机号码，采用 e.164 标准，+[国家或地区码][手机号]
        * 例如+8613711112222， 其中前面有一个+号 ，86为国家码，13711112222为手机号，最多不要超过200个手机号*/
        String[] phoneNumbers = {
    
    "+86"+phone};
        req.setPhoneNumberSet(phoneNumbers);

        /* 模板参数: 若无模板参数，则设置为空*/
        String code = String.valueOf(new Random().nextInt(899999) + 100000);
        String[] templateParams = {
    
    smsSign+session,code,"5"};
        req.setTemplateParamSet(templateParams);

        /* 通过 client 对象调用 SendSms 方法发起请求。注意请求方法名与请求对象是对应的
        * 返回的 res 是一个 SendSmsResponse 类的实例，与请求对象对应 */
        SendSmsResponse res = client.SendSms(req);

        // 输出 JSON 格式的字符串回包
        System.out.println(SendSmsResponse.toJsonString(res));
        SendStatus[] sendStatusSet = res.getSendStatusSet();
        for (SendStatus sendStatus : sendStatusSet) {
    
    
            if("Ok".equals(sendStatus.getCode())){
    
    
                temporaryCache.putPhoneCode(
                    sendStatus.getSerialNo(), sendStatus.getPhoneNumber()+code);
            }
        return SendSmsResponse.toJsonString(sendStatus);
        }

        // 可以取出单个值，您可以通过官网接口文档或跳转到 response 对象的定义处查看返回字段的定义
        //System.out.println(res.getRequestId());

    } catch (TencentCloudSDKException e) {
    
    
    	e.printStackTrace();
    }
    return null;
}

Tinify

<dependency>
  <groupId>com.tinify</groupId>
  <artifactId>tinify</artifactId>
  <version>RELEASE</version>
</dependency>

Tinify API allows you to compress and optimize JPEG and PNG images. It is designed based on Rest services (lightweight web services). We maintain client libraries in various languages, making it very simple to interact with the Tinify API.

Tinify Java API Chinese Reference Document - Programmer Sought

Here, a scheduled task is defined TinifyPngTasks#tinifyPngScheduleto regularly TinifyPngCachecheck List<TinifyPngDTO> imageswhether there are elements in in .

Regarding TinifyPngCache, if you do not enable image compression, it will be useless. When it is enabled, it will save the image url imagesin .

TinifyPngTasks#tinifyPngScheduleEvery once in a while, imagesthe elements (image url) in it are compressed and converted into InputStream through Tinify API:

@Value("${tinify.key}")
private String tinifyKey;

public InputStream getStreamfromUrl(String url){
    
    
    Tinify.setKey(tinifyKey);
    InputStream inputStream=null;
    try {
    
    
        byte[] resultData = Tinify.fromUrl(url).toBuffer(); // toBuffer()压缩图像
        inputStream = new ByteArrayInputStream(resultData);
    } catch (IOException e) {
    
    
    	e.printStackTrace();
    }
    return inputStream;
}

Then call uploadtoBucket to store the compressed image file in COS, and then imagesclear it when all the files in it are uploaded

Vaptcha

Man-machine inspection

ValidateController#postCall in the method to VaptchaProvider.getValidateResult(token,scene,ip)send a request to the vaptcha server (OkHttpClient), and return a json string containing whether it passed.

public static String getValidateResult(String token,int scene,String ip){
    
    
    MediaType mediaType = MediaType.get("application/x-www-form-urlencoded; charset=utf-8");
    OkHttpClient client = new OkHttpClient();
    String s = "id="+vid+"&secretkey="+key+"&token="+token+"&scene="+scene+"&ip="+ip;
    RequestBody body = RequestBody.create(mediaType, s);
    Request request = new Request.Builder()
        .url("http://0.vaptcha.com/verify")
        .post(body)
        .build();
    try (Response response = client.newCall(request).execute()) {
    
    
        String string = response.body().string();
        return string;
    } catch (Exception e) {
    
    
        e.printStackTrace();
    }
    return null;
}

cache

AppUserCache

ExpiringMap<String,String> map = ExpiringMap.builder()
    .maxSize(100)//最大容量，防止恶意注入
    .expiration(1, TimeUnit.MINUTES)//过期时间1分钟
    .expirationPolicy(ExpirationPolicy.ACCESSED)
    .variableExpiration()
    .build();

It is used to cache users who log in on the mobile phone, and then perform login verification during the process of scanning the code to log in.

HotTagCache

private List<String> hots = new ArrayList<>();

There is a method in the class for updating the cache, using a priority queue PriorityQueue:

Java PriorityQueue (priority queue) - 2Tree's Blog - CSDN Blog

• PriorityQueue is an unlimited queue and grows dynamically. The default initial capacity can be overridden using the initialCapacity'11' parameter in the corresponding constructor .
• It does not allow NULL objects.
• Objects added to PriorityQueue must be comparable.
• By default, priority queue objects are sorted in natural order .
• Comparators can be used for custom ordering of objects in the queue.
• The head of the priority queue is the smallest element based on natural ordering or comparator-based ordering . When we poll the queue, it returns the head object from the queue.
• If there are multiple objects with the same priority, it can poll any of them at random.
• PriorityQueue is not thread safe . PriorityBlockingQueueUsed in a concurrent environment.
• It gives **O(log(n))** time for add and poll methods.

DTO implements Comparablethe interface, and then you can specify the comparison strategy when putting it into the priority queue:

@Data
public class HotTagDTO implements Comparable {
    
    
    private String name;
    private Integer priority;

    @Override
    public int compareTo(Object o) {
    
    
        return this.getPriority() - ((HotTagDTO) o).getPriority();
    }
}

First store the Map tags in the priority queue, then take them out one by one and add them to a new List, and then specify hots to point to the new List (the idea of ​​copying when writing is used here).

Similarly, call the method of updating the cache by means of a timed task

IpLimitCache

ExpiringMap<String,String> interval = ExpiringMap.builder()
    .maxSize(20)//设置最大容量，增大攻击难度，值越大存储的可疑ip越多，过大会占用额外资源
    .expiration(30, TimeUnit.SECONDS)//过期时间30秒
    .expirationPolicy(ExpirationPolicy.CREATED)//每次访问重置过期时间
    .variableExpiration()
    .build();

ExpiringMap<String,Integer> ipScores = ExpiringMap.builder()
    .maxSize(100)//设置最大容量，增大攻击难度，值越大存储的可疑ip越多，过大会占用额外资源
    .expiration(1, TimeUnit.DAYS)//过期时间1天
    .expirationPolicy(ExpirationPolicy.CREATED)//每次更新重置过期时间
    .variableExpiration()
    .build();

interval<ip, token>It is an ip blacklist. Of course, this blacklist has an expiration time. Within the specified time (30s), the same ip cannot be verified repeatedly

ipScores<ip, score>It is an ip scoring table. Every time the user sends an operation, the backend will intervalfind it in it, or if the score of an ip exceeds 100, it will add points and return an exception

LoginUserCache

private List<User> loginUsers = new ArrayList<>();

ExpiringMap<Long,Long> loginUserMap = ExpiringMap.builder()
    .maxSize(16)//最大容量，防止恶意注入
    .expiration(11, TimeUnit.MINUTES)//过期时间10分钟
    .expirationPolicy(ExpirationPolicy.CREATED)
    .variableExpiration()
    .build();

There is relevant analysis in the talk

TagCache

The static method generates a List containing all preset tags

TemporaryCache

Two ExpiringMapare used to cache phone and email verification codes respectively

TinifyPngCache

See the third-party service chapter Tinify