MySql last non-NULL value for various columns and the row datetime

JamesH :

I have a table called "raw_temp" in a MySQL (10.1.44-MariaDB) called "test".

It holds sensor values in the following format, and unfortunately I can't do anything about the format.

+----+--------+--------+--------+--------+--------+--------+---------------------+
| id |  api   | value1 | value2 | value3 | value4 | value5 |    reading_time     |
+----+--------+--------+--------+--------+--------+--------+---------------------+
|  1 | asdasd | 1968   | 19.70  | 52.50  | NULL   | NULL   | 2020-03-02 21:34:46 |
|  2 | asdasd | NULL   | NULL   | NULL   | 100    | NULL   | 2020-03-02 21:35:46 |
|  3 | asdasd | 1974   | 19.70  | 52.50  | NULL   | NULL   | 2020-03-02 21:37:47 |
|  4 | asdasd | NULL   | NULL   | NULL   | NULL   | 88     | 2020-03-02 21:39:05 |
|  5 | xdfsgh | 2543   | NULL   | NULL   | NULL   | NULL   | 2020-03-02 21:39:49 |
+----+--------+--------+--------+--------+--------+--------+---------------------+

I would like to be able to get the following out of a SELECT query where reading_time is between "2020-03-02 21:35:01" and "2020-03-02 21:40:00" . Naimly one row for each "api" value, and the value columns showing the last non-NULL reading value (and if there are none show NULL) and value reading time column showing the associated reading_time for that value (hopefully this along with the table below makes sense).

Is this even possible firstly in a MySQL SELECT statement (I also have the option of using Python to create the summary (with loops and Lists I guess)? The above is an example, and in reality there are about 2000 lines of sensor values and about 150 api values, and I would ideally like the query to run in <1 minute to allow time for other processing before the next 5 minute interval.

+--------+--------+---------------------+--------+---------------------+--------+---------------------+--------+----------------------+--------+----------------------+ 
|  api   | value1 | value1_reading_time | value2 | value2_reading_time | value3 | value3_reading_time | value4 | value4_reading_time  | value5 | value5_reading_time  |
+--------+--------+---------------------+--------+---------------------+--------+---------------------+--------+----------------------+--------+----------------------+
| asdasd |   1974 | 2020-03-02 21:37:47 | 19.70  | 2020-03-02 21:37:47 | 52.50  | 2020-03-02 21:37:47 | 100    | 2020-03-02 21:35:46  | 88     | 2020-03-02 21:39:05  |
| xdfsgh |   2543 | 2020-03-02 21:39:49 | NULL   | NULL                | NULL   | NULL                | NULL   | NULL                 | NULL   | NULL                 |
+--------+--------+---------------------+--------+---------------------+--------+---------------------+--------+----------------------+--------+----------------------+

GMB :

If you are running MySQL 8.0, you could use window functions in a subquery to compute the last date of availability of each value, and then do conditional aggregation.

Here is an example for three values:

select
    api,
    max(case when reading_time = max_reading_time_1 then value1 end) value1,
    max_reading_time_1,
    max(case when reading_time = max_reading_time_2 then value2 end) value2,
    max_reading_time_2,
    max(case when reading_time = max_reading_time_3 then value3 end) value3,
    max_reading_time_3
from (
    select
        t.*,
        max(case when value1 is not null then reading_time else end) 
            over(partition by api) max_reading_time_1,
        max(case when value2 is not null then reading_time else end)
            over(partition by api) max_reading_time_2,
        max(case when value3 is not null then reading_time else end) 
            over(partition by api) max_reading_time_3
    from t
) t
where reading_time in (max_reading_time_1, max_reading_time_2, max_reading_time_3)
group by api