apache druid presto for large data processing and are polymerized by means of statistical processing of large data memory, support python
1.druid python code data read: Use pydruid
from pydruid.client * Import
from pydruid.utils Import aggregators
PyDruid = Query ( " HTTP: // IP: port ", 'Druid / V2')
# conditions where there may be a filter or a state of the task ID
SQL = "task_id the SELECT, Status, DataSource, the FROM sys.tasks DURATION"
= self.query.cursor curs ()
curs.execute (STR (SQL))
# can be recycled curs acquisition result
Print (curs)
2.presto can pyhive or prestdb, test performance is stable prestodb, there pyhive-second time-out, including the use of pyhive connected presto database superset there are also time-consuming questions:
(1) mysql connection
created in the presto etc / catalgo in properties files, .properties suffix:
connector.name = MySQL
Connection-url = jdbc: MySQL: // ip: port
connection-user = mysql login user
connection-password = mysql password
prestodb library code:
Import prestodb
Conn = prestodb.dbapi.connect (
Host = 'IP',
Port = 8080, # ports
user = 'mysql database user name, there is arranged in the properties file-User Connection',
Catalog = 'Properties file name without the suffix ',
Schema =' database name ',
)
CUR = conn.cursor ()
sql = "SELECT name FROM table name field GROUP BY field names, field names"
cur.execute (SQL)
rows = cur.fetchall ()
# print the result
print (rows)
(2) connected mongodb: in mongodb where you need to add properties file to configure the user name and password, do not take effect, the database record can obtain data code in the same connection mysql
create a properties file in the presto of the etc / catalgo years, .properties suffix:
Connector MongoDB = .name
mongodb.seeds = ip: port
mongodb.schema-collection = automatically create a table of field information
mongodb.credentials = User name: collection of the user's password @