In a recent data analysis, I used a link query on two tables to return the data I needed. In the process of using, I found that it contains some dirty data, which needs to be cleared by some methods. I located the characteristics of these data, and then added them to the query conditions, hoping to exclude the records that meet these conditions through the query script.
These erroneous data have the following characteristics:
- rate_type = Standard
- client_net_cleared = 0
- program is blank (not Null)
At the beginning, I used the following method, and then added it to the where part, but this did not meet the demand:
SELECT *
, CASE WHEN tad.rate_type = 'Standard'
AND tad.client_net_cleared = '0'
AND program= '' THEN 1
ELSE '0'
END AS noise
FROM tableau.km_tv_ad_data_import tad
JOIN tableau.km_tv_ad_report ga
ON ga.session_timestamp >= tad.timestamp - INTERVAL '4 minute'
AND ga.session_timestamp <= tad.timestamp + INTERVAL '5 minute'
AND ga.session_timestamp != tad.timestamp
WHERE tad.timestamp >= '2016-09-01'
AND (tad.rate_type != 'Standard'
AND tad.client_net_cleared != '0'
AND tad.program != '')
GROUP BY 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21
timestamp | rate_type | program | client_net_cleared | noise
---------------------|-----------|-----------------|--------------------|-------
2016-11-01 18:00:00 | Standard | Diving | 50 | 0
2016-12-01 21:00:00 | Holiday | Classic Albums | 100 | 0
2016-11-01 09:00:00 | FireSale | Panorama | 0 | 0
2016-10-01 12:00:00 | Standard | | 0 | 1
2016-12-01 15:00:00 | Holiday | MythBusters | 100 | 0
2016-10-01 13:00:00 | FireSale | House | 200 | 0
The last thing I need is to
exclude records that satisfy all the following conditions at the same time:
rate_type = Standard, client_net_cleared = 0, program is blank (not Null).
After the amendment, the correct way is:
AND NOT (tad.rate_type = 'Standard'
AND tad.client_net_cleared = '0'
AND tad.program = '')
In this way, the required goal is achieved. In fact, the focus of this question is still on the basis of SQL, to truly understand the calculation logic of AND and OR.