It comes to big data technology, a lot of people think of math, probably because solid numbers in mathematics system location bar, which is taken for granted. Big Data era has been copied very hot, and the industry is now mature, big data development want to learn more and more people. Every day we have retained a lot of information on the Internet, but how to collect, organize this flood of information, and generate value, has been an important issue in all walks of life to explore, not to mention the massive data mining user needs, the predicted future market-oriented, and even our government data also cloud computing, big data. .
If you have a programming background this is the best, and will save a lot of time to learn, easier to understand. Because big data environment is more complex, not as learning programming software, the machine is installed it, knock with the teacher a few lines of code on it, but the data will be more trouble, at least to be ready virtualized cluster environment, then but also install and deploy various computational framework, need to be patient, there is a certain problem-solving skills, perseverance, will it be possible to learn big data.
As the working relationship, in the presence of these two types of people around me, one of the school students are learning, the second is IT company engaged in R & D engineers. They appeared two extremes in mathematics learning and application. College students, especially freshman, sophomore students each semester there are some, such as mathematical analysis, linear algebra, number theory like mathematics curriculum, although you can hear Leibniz and Newton disputes the story in the classroom, Descartes They love story, but they often feel very confused, because they do not know the mathematical knowledge learned in the end what's the use. For IT R & D personnel, before they enter the big data-related posts, always think first learn mathematics, but at the beginning of the vast world of mathematics, where is the big data technology?
Relationship mathematical knowledge and data technologies large part of this is also a very close linear algebra, matrix transpose, block rank matrices, vectors, orthogonal matrix vector space, eigenvalue and eigenvector waiting at the data modeling, analysis It is a commonly used technique.
In large Internet data, many analysts target application scenarios can be abstracted in a matrix, said a large number of Web pages and their relationships, and their relationships microblogging users, text focused text and vocabulary relationships, etc., can be represented by a matrix. For example, for Web pages and their relationship when expressed as a matrix, the matrix element represents the relationship between a page and another page b, this relationship can be directed relationship, 1 means there is a hyperlink between a and b, 0 represents a, no hyperlinks between b. The famous PageRank algorithm is to quantify the importance of a page based on this matrix, and prove its convergence.
Various matrix-based operations, such as matrix decomposition pathway analysis is the extraction of the object feature, because the matrix represents some transformation or mapping, matrix thus obtained decomposition represents a number of new features in the new analysis target space . Therefore, the application of singular value decomposition SVD, PCA, NMF, MF and other large data analysis is very extensive.
Details: [] Big Data learning basic and applied mathematics - Ali cloud University
Course introduces the mathematical basis of big data:
- First, vectors, matrices introduced
- Second, the application vectors in the game engine
- Third, the singular value decomposition and its applications
- Fourth, the derivative, gradient Introduction
- Fifth, the optimization method and its application
