There must be a process for everything, and you must know what to do and how to do it, of course, BigData is no exception.
Clarify analysis purpose and ideas -> data collection -> data processing -> data analysis -> data presentation -> report writing
Clarify analysis purpose and ideas
In layman's terms, it is what you want to do, how you want to do it, what you want to accomplish, what problems you want to solve, and what your thinking is.
Disassemble the things that need to be analyzed into sections to complete, (first set a small goal for yourself, earn him a y, hahaha) analyze what first, and then analyze what, you won’t think where to start , And the one that disassembles one i must be systematized, that is, logicalized, and there must be a logical connection between each paragraph.
Only when the purpose of the analysis is clarified, can the analysis framework be finalized. Finally, it is necessary to ensure that the analysis is systematic and make the analysis more convincing.
data collection
Data collection is the process of collecting relevant data in accordance with a clear data analysis framework. Provide materials and basis for data analysis, (that is, if you don’t find some data, you analyze a hammer) The data referred to here includes primary data and secondary data. Primary data is data that can be directly obtained, and secondary data refers to processed data. Data obtained after sorting.
data processing
To put it bluntly, data processing is to give you a big push to classify the data and form a form suitable for data analysis. Because the collected data is messy and disorderly, it is necessary to extract and derive from this large amount of disorderly data that is valuable and meaningful for problem solving The data. Data processing is an indispensable process before data analysis. Data processing mainly includes data cleaning, data conversion, data extraction, data calculation and other processing methods. Generally, the data after you get it needs to be processed before subsequent data analysis can be performed. "Clean" raw data must be processed before it can be used
data analysis
Analyzing data refers to the process of analyzing data with appropriate analysis methods and tools, extracting valuable information, and forming effective conclusions. Since data analysis is mostly done through software, this requires analysts not only to master various analysis methods, but also to be familiar with analysis tools. (Examples of tools: Mysql, Hive, Hbase, Kudu, Redis)
Data presentation
Data presentation is data visualization. In general, data is presented in the form of tables and graphs. Commonly used data icons include pie charts, bar charts, bar charts, line charts, scatter charts, radar charts, etc. We can also further process these icons and turn them into the graphics we need, such as pyramid diagrams, matrix diagrams, funnel diagrams, etc. In most cases, people are more willing to accept graphics as a data display method because it is effective and intuitive.
Writing reports
The summary, data analysis report is a summary and presentation of the entire data analysis process. Through the report, the cause, process, results and recommendations of the data analysis are fully presented for the reference of decision makers. A good data analysis report first needs a good analysis framework, with pictures and texts, clear levels, so that the reader can understand it at a glance. In addition, the data analysis report needs to have clear conclusions. Analysis without clear conclusions cannot be called analysis. At the same time, it loses the meaning of the report, because we did the analysis to verify a conclusion at first, so don’t fall short. Finally, a good report must have suggestions or solutions.