If you're still in 2012, I would eagerly participate in any discussion about big data. This is a very popular word, you must say "magic" word to let people hear about the latest greatest technology. But it is disappointing that most of the world have not been fully explored and applied to large data. Believe me, not just those sitting in the ivory tower of CIO / CDO have been plagued by big data. At the same time, those energetic developers are looking to resume looking for "big data" talent company.
The catalyst of this article is to join our top talent and a month before I talk. He was very sad, because he had not worked in terms of big data. When asked about the meaning of "big data" quick answer is Hadoop and Spark. When he is learning to face sourcing events Lagom and the fact that he did not quite believe this will enable him to build better solutions.
These techniques without any problems, in fact, it is these technologies allow ecosystems popular. But technology is only part of these have any commercial value products, and sometimes a small part. They solve a specific problem. And usually, if your product is "only" Based on these technologies, you will certainly fail!
So, if we do not talk about big data, what direction should we develop it? The answer is toward rapid data growth. If you talk to CIO and 10, nine said they would be difficult to accept big data. A 1TB of data management, data management the other hundreds of PB, it does not matter. We need to focus on ensuring customers get the best experience. Customer Experience (CX) will become the king of modern applications. Only concerned Spark / Hadoop / Flink, and that you can expand the data, this is a fallacy.
Let's see how these so-called big data technology to adapt to the grand plan.
If you are building a product with a user interaction, you need a front-end product of the reaction, so that you can provide excellent customer experience.
When a user request occurs hundreds or thousands of products which must be handled without degrading performance. It must be flexible.
There will be transaction-based processes - such as someone query something, add a project, and view the day's trading. These can be handled by different micro-services. They will have their own life cycle, and should be able to independently expand.
If you want your system is scalable, and current plans for any unforeseen future business operations, you need the event source.
You may want to separate read and write to the system to ensure that reading and writing SLA, and you can extend the terms of separate read and write.
If you need the transaction data stored in the database, you need SQL or NoSQL database.
Now, some of your features need to analyze the data, then returns the data analysis. Now, depending on the SLA, you need to add here the big data framework.
You need to run some machine learning algorithm or depth learning to make your product stand out.
Of course, we are a large number of simplified scenarios. But I hope you can understand. Relying solely on a large data frame or hire some understanding of Hadoop / Spark consultant does not work. You need to deal with all the technologies needed, such as:
-
Reactive UI
-
Micro Services Framework
-
Asynchronous messaging system
-
Big Data framework
-
database
-
Policy-based hosted container
-
Monitoring and telemetry
-
Machine Learning and Artificial Intelligence
However, it listed above is not complete.
All of these are covered by the principles of effective software development, patterns and practices. According to the principle of "reaction Declaration", the main driving force of technology are:
-
Scalable
-
Safety
-
flexible
-
real-time
-
Flexible
-
Everywhere
-
intelligent
-
agile
To sum up, this is a possible technical solution to achieve the vision of the product:
As you can see, the big data framework is only part of what you want to do. They are not just a drop in the sea, but they are not big enough.
End of the article, what do you think? Looking forward to your comments.
If you are ready to join the big data and big data regarding the current 2019
[Prospects] poke me read
[Jobs] poke me read
[Salary] Big Data poke me read
[Line] complete learning poke me read
Focus on micro-channel public number itdaima get a full set of development tools, as well as large data entry learning materials