When I saw my 14-year internship today, my mentor wrote a sql with more than 50 lines in his blog, saying it was the longest sql he wrote,
Thinking about this year of work, I actually wrote a lot of sql, because I am doing data development work, writing sql is inevitable, haha~
Today is just the first day of work after the May 1st holiday. I don't want to do anything, so just write a blog. I haven't recorded it for a long time. Okay, let's get down to business,
Let’s talk about sql. Last year, our data group had to do the previous work of extracting basic data from mysql into hive, because I was doing
This is the basic data, so let's start doing this. Extract more than 30 most basic tables from hive, and then extract some basics through various joins
field, this sql has more than 380 lines in total. . . Of course, I still use union all in the middle. If you count the shell script in the middle, it is almost a small script with more than 1000 lines.
What are the advantages of such a long sql: reduce the output of the intermediate table;
There are too many bad points: one is that it is not easy to test, and you need to be very familiar with the business
The second is that the efficiency is not necessarily high;
If the sql foundation is not very good, don't try to do it, it is very laborious.