Do not extract the data, make data marts and data cube, because "extract" means the transfer
Remember, security as a service.
Flat would destroy the original structure of the important relationship expressed.
Those wishing to visual elements interact, get the answers they are looking for, rather than you have available to them the results of cross-filtration.
Many people create data, but people rarely use data.
Original translation:
Today, various companies and organizations are no longer using the previous generation architecture to store large data. That being the case, why use the previous generation of Business Intelligence (BI) tools for big data analysis? In the choice of BI tools for businesses, should abide by the following "Ten Commandments."
The first commandment: Do not transfer large data
Transfer large data costly: after all, big data is "big", if the transfer package, the burden is too heavy. Do not extract the data, make data marts and data cube, because "extract" it means that the transfer will be in maintenance, resulting in chaos complex issue additional network performance processors, the same backup two logically appear. Make BI more deeply underlying operating data is initially powered Big Data germination.
Second Commandment: Do not steal or violate corporate security policies!
Safety is not optional. Unfortunately, data breaches occur frequently, indicating that safety is not easy to achieve. To choose to take advantage of existing security model of BI tools. Rely Ranger, Sentry, Knox, and other integrated security systems, big data can make it easier to achieve data security, and now even the Mongo database has amazing security architecture. All those models allow you to insert permissions, user information spread all the way to the application layer, the implementation of visual authority and provides the authorization data associated blog. Remember, security as a service.
Third Commandment: Do not pay according to the number of users and amount of data
A major benefit of big data is that, if done, it will be able to achieve a very high price. The data is stored 5PB to Oracle may make you bankrupt, but to large data storage systems will not. However, before paying the purchase, the price should be wary of some trap. Some BI application the user is charged according to the amount of data or index data volume. Be on the alert! Amounts of data and large data usage appears exponential growth is a commonplace thing, our clients have witnessed their views in just a few months soared from a few tens of billions to hundreds of billions of times the number of users by 50 times. This is another advantage of big data systems: incremental scalability. Do not be fooled by low prices, to buy a development that could increase companies impose "high tax" BI tools.
Fourth Commandment: To boldly learn from other people's greed can view
Share a static chart? Which we have done, whether it is a PDF document, PNG images or attachments in e-mail, spread everywhere in a static chart. But for large data and BI, a static chart is not enough: all you have is nothing but some pretty pictures Bale. You should let anyone be able to interact with your data arbitrarily. It should be seen as an interactive visual road map control data. Why do behind closed doors? Interactive visualization tools will be made public only the first step. Github look at the pattern to know. Instead of saying "This is my final release product", it is better to say "This is a can view, copy down, break it, and I'm derive those opinions, it can also be used to see what other areas." This will others learn from your insights useful things.
Fifth Commandment: To analyze the data in its natural state
Big data is "unstructured", so to say that we have heard too much. actually not. Finance and sensors will generate a lot of pairs. JSON (probably today is the most popular data format) may be semi-structured, a multi-structured, etc., Mongo database for this data format in the re-injection. JSON has the advantage of a good deal and scale, but if you convert it into a form, expressiveness is lost. Many big data is still being tabulated, usually they have thousands of columns. You have to find the values for all relationships: "...... Pick From this in that case." Flat would destroy the original structure of the important relationship expressed. Away from those who say to you, "Please convert the data into a table, because we have been so dry," the BI solution.
Sixth Commandment: Do not wait indefinitely results
In 2016, we expect the data will become faster processing speed up. A typical method is an online analytical processing (OLAP) cube, in essence, is expected to transfer data to calculate the cache, thus speeding up the processing speed. The problem is that you have to extract and transfer data (see the First Commandment), in order to build a data cube, and then to speed up. Now, this method can work well under certain data size, but if the temporary table is too large, your laptop in an attempt to form localized when it will collapse. When you retrieve new data cache is rebuilt, new data analysis will stop halfway. Also, note that the sample questions, you may get one that looks good, very good results can view, but in the end only to find all wrong road, but the problem lies in the lack of the bigger picture. To select a BI tool that can easily be continuously adjusted data.
Seventh Commandment: Do not create reports, and to build applications
For a long time, "get data" means access to the report. In the era of big data, BI users want asynchronous data from multiple sources, so they do not need to refresh anything, just like all kinds of other things running on browsers and mobile devices. Those wishing to visual elements interact, get the answers they are looking for, rather than you have available to them the results of cross-filtration. Rails and other Web application framework to make it easier to build. Why not BI applications to do the same thing? No reason not to these applications, application program interface (API), templates, reusability, and so take a similar approach. Now is the time to look through the lens of modern BI Web application development.
The eighth commandment: to make use of intelligence tools
In providing data-based can view, BI tools have proved their ability. Now it is the turn of efforts on the model and automatically maintain the cache, so that the end user does not have to speak the heart. In the large-scale data, automatic maintenance is almost indispensable, we can get a lot of information and data from user interaction with a view, modern tools should use this information to take advantage of the data network effects. In addition, to select those tools built comprehensive search capability, because I have seen some customers may have hundreds of thousands of views. You need a method to quickly find, under the influence of the network for many years, we have become accustomed to search, rather than rummaging menu.
Ninth Commandment: to go beyond the basic category
Today's big data predictive analytics system as is known. Relevance, forecasting and other features enable business users at any time can be more easily advanced analytics than ever before. Visualization technology does not require programming experience will be able to handle large data allow analysts divine intervention, beyond the scope of fundamental analysis. In order to realize its true potential, large data should not rely on everyone becomes R prophecy programmer. Humans are very good at processing visual information, we must work harder to visualize the information is presented in front of people.
Of the Ten Commandments: Do not just stand data lake, waiting for data to scientists working children
Whether you are the big data as a data lake or enterprise data center, Hadoop has changed the data processing speed and storage costs, we are creating more data every day. But in the real use of big data services for business users, often there is a "write-only system" - a lot of people to create data, but people rarely use data.
In fact, you can answer numerous questions for business users with the data in Hadoop. BI emphasis is to create data visualization applications to support daily decisions. Enterprise of everyone wants to make data-driven decisions. The data can answer all the big questions of limited data scientists need to deal with the problem, which is a great insult.
Recommended Reading articles
40 + annual salary of big data development [W] tutorial, all here!
Zero-based Big Data Quick Start Tutorial
web front-end development based tutorial
Basics tutorial to learn linux
Big Data era need to know six things
Big Data framework hadoop Top 10 Myths
Experience big data development engineer salary 30K summary?