Table of contents
【Write in front】
During my undergraduate period, I learned some data mining algorithms, and even my graduation project was based on this. However, I obviously learned too little. Today I accidentally read the book "Data Mining·Concepts and Technology", and I will record what I learned here.
What is data mining
Personal understanding: Data mining is in massive amounts of datadiscover knowledgeOr ratherExtract data schema. [To achieve this goal of knowledge discovery, we may do the following work: data preprocessing, using machine learning to discover patterns, pattern evaluation, knowledge representation (such as data visualization), etc. 】
Personal evaluation: In fact, a lot of technologies are involved in this process, as shown in the figure below:
Why do model evaluation?
- Because not all modes are interesting.
- It is often unrealistic and inefficient to expect a data mining system to generate all possible patterns.
- The patterns produced by data mining systems are not always interesting.
How to perform schema evaluation
- Most association rule mining algorithms use the support-confidence framework. Moreover, when using low support threshold mining or mining long patterns, some rules that are not interesting to users will be generated, which is also one of the main bottlenecks for the successful application of association rule mining.
- From this, other measures have been proposed, such as: lift, X 2
Development Trends of Data Mining
- The development of effective data mining methods, systems and services, and the construction of interactive and integrated data mining environments are key research areas.
- In addition, data mining will also be applied to more and more fields, such as biology, biomedicine, software engineering, information physics, etc.