Introduction to Business Intelligence

Preface

Originally, I wrote business intelligence data analysis. But as it is written, the part of business intelligence accounts for a large part, so it is simply a separate article.

Business Intelligence (Business Intelligence, referred to as: BI), also known as business intelligence or business intelligence, refers to the use of modern data warehouse technology, online analysis and processing technology, data mining and data display technology for data analysis to achieve commercial value.

Business intelligence is generally understood as a tool that transforms existing data in an enterprise into knowledge and helps the enterprise make wise business operation decisions

In short, a tool to help companies make decisions. It is an advanced stage of traditional industry reports. This business intelligence is more scientific and comprehensive, and can analyze a lot of information to help decision makers analyze. And it is also real-time.

Technology used

What is business intelligence?

From a technical perspective, business intelligence is not a new technology, it is only a comprehensive application of data warehouse, OLAP and data mining technologies.

Data Warehouse (Data Warehouse) and Data Mart (Data Mart) products. Including pre-configured software for data conversion, management and access, and usually some business models, such as financial analysis models.

Today's data processing can be roughly divided into two categories: OLTP (On-Line Transaction Processing) and OLAP (On-Line Analytical Processing).

OLTP is the main application of traditional relational databases, mainly for basic and daily transaction processing, such as bank transactions.

OLAP is the main application of the data warehouse system, supporting complex analysis operations, focusing on decision support, and providing intuitive and easy-to-understand query results.

Data Mining (Data Mining) software. Use techniques such as neural networks and rule induction to discover relationships between data and make data-based inferences.

The data warehouse is to store the reports in the database, and the technology stack used is SQL and spark;
Online analysis and processing is the presentation and processing of data. The technology stack used is the back-end technology stack and the front-end technology stack;
Data mining is to find the relationship between data, the technology stack used is Python (artificial intelligence library).

Implementing a business intelligence system is a complex system engineering. The general implementation steps are:

Requirement analysis: collect the needs of the enterprise, what data you want to draw what conclusions;

Data warehouse modeling: Analyze the needs of the enterprise and establish the logical model and physical model of the enterprise data warehouse;

Data extraction: Extract data from the business system to the data warehouse. The data must be converted and cleaned in the extraction process to meet the needs of analysis;

Establishing business intelligence analysis reports: data visualization

User training and data simulation test: for users to use;

System improvement and perfection: iterative upgrade

This is the feasibility study, requirements analysis, design, coding, testing, integration and maintenance of software engineering . It feels so interesting. The things taught in the university are really like internal skills, panacea, which can be used anywhere.

The development status of business intelligence

After several years of accumulation, most of the medium and large enterprises and institutions have established relatively complete basic information systems such as CRM, ERP, and OA.

The unified characteristics of these systems are: through the operations of business personnel or users, the database is finally added, modified, and deleted. The above systems can be collectively called OLTP (Online Transaction Process), which means that after the system has been running for a period of time, it will inevitably help enterprises and institutions collect a large amount of historical data.

However, the large amount of data scattered and independently existing in the database is just a bible book for business personnel. What business people need is information, which is abstract information that they can understand, understand, and benefit from. At this time, how to transform data into information so that business personnel (including managers) can fully grasp and use this information, and assist decision-making , is the main problem that business intelligence solves.

How to transform the data existing in the database into the information needed by business personnel? Most of the answers are report systems .

The reporting system can already be called BI, which is a low-end implementation of BI.

Most foreign companies have entered mid-range BI, which is called data analysis. Some companies have begun to enter high-end BI, called data mining. Most of Chinese companies are still at the reporting stage.

According to the theory of microeconomics. The object of economic research includes the two main aspects of resource scarcity and choice. The scarcity of resources goes without saying. Every economic individual needs to make choices when carrying out economic activities.

In ancient times, merchants usually made choices based on bills and market information; the same is true in modern times.

In the computer age, businessmen use electronic tables such as Excel to count bills and industry reports, then process the bills and analyze the industry reports, and finally let the decision-makers make decisions;
In the Internet age, merchants collect and process forms through online OA platforms, and finally let decision-makers make decisions (external connections and industry reports).
In the era of big data, businessmen make decisions by combining information from data mining (analyzing data with artificial intelligence to find the relevance of the data) combined with report statistics and industry reports.

After all, it is a system platform for processing reports.

The above paragraph refers to elementary BI, intermediate BI, and advanced BI. In my opinion, it is an online business website, an online analysis website, and a data mining website;

The technology stacks used are

Primary BI (online transaction website): backend, database. Transplant the offline report to the online. The architecture of LAMP can satisfy.
Intermediate BI (online analysis website): increase the front end. Visualize the data. Need to use several visualization libraries, such as echarts, mapbox;
Advanced BI (Data Mining): Implemented in Python using artificial intelligence libraries (such as Numpy, paddlepaddle). The correlation between various data can be analyzed, and the content that cannot be analyzed on the surface can be explored.

After understanding these positions, you can also know how to prepare the technology stack.

Conclusion

Let's go through the part of business intelligence that is unclear and vague in concept.

Has been confused before. It feels like it is, and it doesn't seem to be. What data model, data mining, artificial intelligence, user portrait, user behavior analysis. Variety. Could not find a reference. I stroked it now, it's much better.

If I do data visualization, then it belongs to the category of data analysis.
If I use artificial intelligence to analyze the correlation between various data, then it is the category of data mining.

Data mining belongs to data analysis, but it is higher than data analysis.

In fact, in my opinion, only those that use artificial intelligence are considered business intelligence. The most useless artificial intelligence is big data analysis (business analysis).

Business analysis has been exactly the same since ancient times.

In ancient times, paper was used to keep accounts, and then the accounts were settled through the accountant. The boss bought more goods according to which products people bought.
Hyundai keeps accounts on Excel, and then uses Excel to make tables. The boss looks at these tables and makes decisions.
Later, online accounting is implemented, the back-end processes these data, the front-end programs these data, and the boss looks at these data to make decisions.

Until the maturity of artificial intelligence, two completely unrelated things can be connected, and information other than statistics can be provided to help the boss do more analysis.

Seeing this, I realized that I was probably stuck in business analysis. Most of my current technology stack is doing business analysis. The artificial intelligence is not involved. Get involved in data mining when you learn the ceiling of business analysis.

But business analysis is not simple. Data from business analysis accounts for the vast majority of information. It is also the main information of industry reports. I will do a good job of business analysis in the future.

I have now abandoned the two directions of in-depth, one is the back-end, and the other is data mining. The two directions currently reserved are the database and the front-end visualization. (I feel that the full stack is slowly coming, but the full stack has to learn too much, it is easy to learn more and more, what can't you do)

Update address: GitHub

For more content, please pay attention to: CSDN , GitHub , Nuggets