How to use Java to realize AI face fusion special effects

Background of the project

Recently, since chat-gpt became popular, the continuous iterative innovation of AI technology in the field of artificial intelligence has brought many shocking applications to people's lives. For example, the special effects of AI face fusion are becoming more and more popular on major platforms such as Douyin and Bilibili. Based on this, I also plan to use third-party open source APIs to realize my own face fusion.

The principle of AI face fusion special effects

AI face fusion special effect is an innovative application based on deep learning and computer vision technology , which can integrate one person's facial features into another person's photo or video to achieve amazing and realistic effects. Not only can you integrate the facial features of celebrities, characters or historical figures with yourself, but you can also realize the transformation of different identities such as transgender and transage.

Then it is inseparable from the powerful processing capabilities of deep neural networks . First, by training massive face data, the network can accurately extract the facial features of each person and encode them into high-dimensional vector representations. Next, using models such as generative adversarial networks (GANs) and autoencoders, the facial features of the source image are fused with the target image with high accuracy . Finally, after parameter adjustment and optimization, the generated image perfectly restores the facial features of the source image while maintaining the style of the target image.

What makes this technology groundbreaking is its exceptional realism and believability. Through AI face fusion special effects, users can easily have a surreal experience and feel the joy of getting close to the celebrities they admire. At the same time, it has brought huge innovation space to the film and television entertainment industry. Actors can freely switch between different roles, allowing audiences to see more imaginative works.

Code

Before starting, I personally compared the effects of several third-party face fusion special effects in private, such as Baidu Smart Cloud, Ali and other platforms. After the final comparison, I personally think that the effect of Baidu Smart Cloud is better. Therefore, this time Just borrow the third-party API of Baidu Smart Cloud to realize the special effects of face fusion.

We first register an account, and then in the console, we can search for face recognition and apply for some free basic service resources, but today we need to purchase the special effects of face fusion that we want to achieve, so we create an application. After the creation is successful, the platform will assign you the relevant credentials of this application, mainly AppID, API Key, and Secret Key. We need to write down these values: respectively client_idand client_secret, which will be used later when calling the token interface.

Step 1: Call the token interface

public static String getToken() {
    
    
    String grant_type = "client_credentials";
    String client_id = "fasq35sadvsvqwr5q...";
    String client_secret = "fasq35sadvsvqwr5q...";
    String url = "https://aip.baidubce.com/oauth/2.0/token" + "?grant_type=" + grant_type + "&client_id=" + client_id + "&client_secret=" + client_secret;

    String result = HttpClient.doGet(url);
    
    System.out.println(result);
    return result;
}

When calling, you will get a large string of access_tokens as follows:

face fusion

Using the token we just obtained, we can call the third-party API, the path is:
url = "https://aip.baidubce.com/rest/4.0/face/v1/merge"

 public static String faceMerge() {
    
    
        // 请求url
        String url = "https://aip.baidubce.com/rest/4.0/face/v1/merge";
        try {
    
    
            Map<String, Object> map = new HashMap<>();
            //模板图
            Map<String, Object> image_templateMap = new HashMap<>();
            image_templateMap.put("image", "sfasq35sadvsvqwr5q...");
            image_templateMap.put("image_type", "BASE64");
            image_templateMap.put("quality_control", "HIGH");
            map.put("image_template", image_templateMap);

            //目标图，用户自己上传的图片
            Map<String, Object> image_targetMap = new HashMap<>();
            image_targetMap.put("image", "sfasq35sadvsvqwr5q...");
            image_targetMap.put("image_type", "BASE64");
            image_targetMap.put("quality_control", "HIGH");
            map.put("image_target", image_targetMap);

            String param = GsonUtils.toJson(map);

            // 注意这里仅为了简化编码每一次请求都去获取access_token，线上环境access_token有过期时间， 客户端可自行缓存，过期后重新获取。
            String accessToken = "[调用鉴权接口获取的token]";

            String result = HttpUtil.post(url, accessToken, "application/json", param);
            System.out.println(result);
            return result;
        } catch (Exception e) {
    
    
            e.printStackTrace();
        }
        return null;
    }

Here are a few things to note:

• Request body formatting : Content-Type为application/json, format the request body through json.
• Base64 encoding : The requested picture needs to be Base64 encoded. The base64 encoding of the picture refers to encoding the picture data into a string of strings, and using the string to replace the image address. You can first get the binary image, and then encode it in Base64 format. It should be noted that the base64 encoding of the image does not contain the image header , such as data:image/jpg;base64,
• Picture format : currently supports PNG, JPG, JPEG, BMP, does not support GIF pictures

This interface mainly includes two parts:

The first one is the template image, which is the effect that the user wants to merge into. This is the default. For example, if I want to merge with a digital human picture, then this digital human is a template image, and we can write its base64 in the i project.

The second is the target image. The target image is the selfie image uploaded by the user. This is dynamic, and different users upload it differently. This also needs to be base64 transcoding.

Important request parameters:

There are also several important request parameters as follows:

image_templateObject: template image information, the face that needs to be fused;

There are several parameters in this object:

image: The resolution of the template image information image must be below 1920x1080

image_type：

Image type
BASE64: the base64 value of the image;
URL: the URL of the image (it may take too long to download the image due to network and other reasons)
FACE_TOKEN:: face identification

In the same way, the same is true for the target target graph parameters:

Some tools

public class GsonUtils {
    
    
    private static Gson gson = new GsonBuilder().create();

    public static String toJson(Object value) {
    
    
        return gson.toJson(value);
    }

    public static <T> T fromJson(String json, Class<T> classOfT) throws JsonParseException {
    
    
        return gson.fromJson(json, classOfT);
    }

    public static <T> T fromJson(String json, Type typeOfT) throws JsonParseException {
    
    
        return (T) gson.fromJson(json, typeOfT);
    }
}

final rendering

On the whole, the face fusion special effect was realized this time, and it can be realized from beginning to end in one afternoon without using it. The overall effect is quite satisfactory.