Linux view the size of a file
you -h
How to view a Linux file 500 rows of data
tail -n 500 aaa.txt
Spark types of operators, some are saying
slightly
You are responsible for part of the project
Reference on the resume of personal responsibility.
How to use filters hive null
(1) Different types of data storage rule is null
string type int and data storage, null is stored as the default \ N;
If the data type is string "" is stored is "";
Further when inserting data into the fields of type int, "", the result is \ N.
(2) Discover different data types, null values
For int may be used to determine the air is null;
For string type, is null condition is found out \ N data; = the condition '', out of the query, "" data.
Conclusion: To be determined according to the actual storage space-time determination. In the development process if required to empty judge, you got to know what kind of data is stored.
There is a small air handling skills, Hive give a less than perfect solution - a custom bottom of what characters to represent NULL
3 Index What types?
General index: only speed up queries
The only index: + speed up query column value unique (there may be null)
Primary key index: a unique value to speed up queries + column (not have null) only one table +
Composite index: a multi-column index value of the composition, specifically for combinatorial search, the efficiency is greater than the combined index
Full-text indexing: the content of the text word segmentation, search
4 If you consider the zone database, what type?
TIMESTAMP is the standard unix timestamp, which stores the number of seconds elapsed 1970-1-1 to now, 4 bytes of storage
DATETIME larger range, as if from 0000-00-00 00:00:00 to 9999-12-3123: 59,8 byte storage: 59
select
date_format (from_utc_timestamp (to_utc_timestamp ( "Beijing China", 'GMT + 8'), "converted to local GMT"), 'yyyy-MM-dd HH: mm: ss') as local_time
from ccsd.tb_name
5 composite index also like to use it effectively?
like in some cases will not use the index, but in some cases are taking the index: For example:
like '% keyword%' or like '% keyword' will not take the index, but when like 'keyword%' will not necessarily take the index, so the specific circumstances, if the select query is this other fields [field no index or index fields in a combination like this are cited, in addition to the id field] such a situation would not take the index, if you want to take the index you want, you can add the force index (ziduan1_index), so it
Using an index like to go: Field queries had to be a combination of the index for the job
6 how to use a composite index take effect
Combination index established order: (a, b, c, d, e, f)
1, if the composite index of the index are, then the index into full force
2, if the composite index index a far left does not exist, then the index does not take effect
3, if the combination of the index index of a leftmost present but does not use all indexes. First order of indexed lined up, start back from a judgment whether continuity (indexed sequential basis), if both until the last of continuity, these indexes take effect. If the index continuity broken at the next node, the current node and the index of the left index effect. The right of the current index node does not take effect
Features 7 affairs?
1: Atomicity: All operations in the transaction included either all succeed, or all fail rollback; success must be fully applied to the database, the failure can not have an impact on the database;
2: Consistency: before and after the implementation of the transaction must be executed in a consistent state,
Example: former user A and user B add up to a total of 5,000; regardless of how the between AB user is interchangeable, after the end of the transaction the money add up to two users or 5000, this is the transactional consistency.
3: Isolation: when a plurality of users concurrent access to the database, the user opens a database for each transaction, it is not disturbed by operation of other matters, to isolation between a plurality of concurrent transactions;
4: Durability: Once a transaction is committed, then changes to the data in the database is permanent, even when the database system experienced a failure of the operation will not be lost things.
8 database isolation level
01: Read uncommitted (read uncommitted): the lowest level, in any case will take place.
02: Read Committed (Read Committed): can avoid dirty reads to occur.
03: Repeatable read (Repeatable Read): can prevent dirty reads, unrepeatable reads occurring.
04: Serializable (serialization): prevent dirty reads, non-repeatable read, phantom read occurs.
9 linux common operating system command
Review of awk and sed increase.
10 Which upload their own software in linux?
Hadoop, Hive, zookeeper, HBase, Java, Spark can say. Try to say that they are familiar with.
11 understanding of the heap and stack?
Problem JVM virtual machine. For more details, see the JVM virtual machine-related content.
12 java data structure
1 array
2 lists
three stacks and queues
4 binary
5 heap and stack
6 hash table
7 black tree https://www.jianshu.com/p/8e54797ec3e0
There is also a relatively simple question, is java data type, you know and know the result of the 8 primitive data types, and other complex data types
13 If removed from a doubly linked list data, or how to make it a doubly linked list?
14 algorithm know? Know what? He said that under the principle of
15 big difference between the offline data and real-time big data? How do you technology selection?
16 off-line process data flow
17 kafka cooked it?
18 unstructured database What? What used?
HBase、MongoDB、Redis
19 kafka which node types
Broker，producer，consumer
20 If the three kafka cluster is down one, how do? The two dang it?
What's the difference 21 kafka and traditional message queue?
22 clusters are dynamically increase it?
The role of the trigger 23
24 work in SQL optimization
How long the project took 25? What is the status from the beginning? What is the status of the end? What are you responsible for?