Dynamic JSON key/value pairs generation in ESQL

Muhammad Faizan Uddin :

How to transform JSON response retrieved from external system to meaningful data (key/value pairs) in ESQL?

Retrieved JSON:

{
    "data": [
        {
            "name": "application.info.header",
            "value": "headerValue"
        },
        {
            "name": "entity.statistics.name.fullName",
            "value": "fullNameValue"
        },
        {
            "name": "application.info.matter",
            "value": "matterValue"
        },
        {
            "name": "entity.statistics.skill",
            "value": "skillValue"
        }
    ]
}

where,

• name ~ hierarchy of JSON (last attribute being the key)

• value ~ value against the key

Expected JSON:

{
    "data": {
        "application": {
            "info": {
                "header": "headerValue",
                "matter": "matterValue"
            }
        },
        "entity": {
            "statistics": {
                "name": {
                    "fullName": "fullNameValue"
                },
                "skill": "skillValue"
            }
        }
    }
}

Needless to say this can be easily achieved in Java through Split method - I'm looking for a suitable method in ESQL.

Current ESQL Module:

CREATE COMPUTE MODULE getDetails_prepareResponse
    CREATE FUNCTION Main() RETURNS BOOLEAN
    BEGIN
        DECLARE data REFERENCE TO InputRoot.JSON.Data.data.Item[1];
        SET OutputRoot.JSON.Data = InputRoot.JSON.Data;
        SET OutputRoot.JSON.Data.data = NULL;

        WHILE LASTMOVE(data) DO
            DECLARE keyA CHARACTER SUBSTRING(data.name BEFORE '.');
            DECLARE name CHARACTER SUBSTRING(data.name AFTER '.');
            DECLARE keyB CHARACTER SUBSTRING(name BEFORE '.');
            DECLARE key CHARACTER SUBSTRING(name AFTER '.');
            CREATE LASTCHILD OF OutputRoot.JSON.Data.data.{EVAL('keyA')}.{EVAL('keyB')}
            NAME key VALUE data.value;
            MOVE data NEXTSIBLING;
        END WHILE;

        RETURN TRUE;
    END;
END MODULE;

This is currently handled through SUBSTRING method in ESQL (for 3 levels only), but now the JSON levels are dynamic (no limit to key/value pairs) as per requirements.

Adam Rice :

You could implement your own procedure to split a string. Take a look at this answer for an example.

ESQL for splitting a string into mulitple values

The method splits S on Delim into an array in Env (Environment.Split.Array[]) and removes Environment.Split before refilling it.