can’t say it was a complete failure, but as the original shareholder, I didn’t get a penny in the end. Sad). Oh, back to the topic, I still like to study various technologies alone, and hope that one day I can really put what I have learned into practice.
It may be outdated to learn Hadoop until now. Recently, many people on the Internet say that learning big data should learn spark. I also searched Baidu in line with the trend of thinking, and found that spark is actually an enhanced version of Hadoop. . .
, Well, then let's study Hadoop with peace of mind -- after all, my consistent learning goal is to learn the thinking mode, not to catch up with the trend
. There are a few videos and other materials available on the Internet. In order to get started quickly, I still find this book. PDF comes quickly (I have worked for more than 5 years, and I feel that a lot of knowledge is familiar, and I feel that the accumulation of knowledge is faster than the introduction of books); of course, if you have not reached a certain level, you should watch the video first, after all. The teacher's explanation is still very detailed. I
recommend two books "HADOOP Authoritative Guide" and "Hadoop Practice". They are well written and relatively basic. It is not difficult to get started with children's shoes with a certain foundation. (I only read about 2 or 3 chapters of these two books, but the more I read it, the more I feel itchy, and I always want to type code... But the development of Hadoop is not the same as other projects. At least the structure is not the same. . .) The rookie
can't afford to hurt, let's get an eclipse plug-in to help. Since I wasted some time with this plugin, here is a brief note on the key things. Some steps are also available on the Internet, please forgive me, I mainly use it for my own memo 1. Environment installation--JDK installation (the latest version is enough, not much to say, there is a lot of information on the Internet, it is too simple) 2. Environment installation-- Hadoop installation (the latest version is enough, there are too many online, such as decompression installation, configuration, etc., so I won't talk about it here)
Forget it, let me say a few more words. It is best for everyone to find google or baidu one by one for the configuration item. It is still necessary for beginners.
3. Environment installation--Ant (the latest version, decompression installation, environment variables, etc. Configuration, too simple, let's not talk about it)
4. Eclipse plug-in -- go to github to download hadoop2x-eclipse-plugin-master
address: https://github.com/winghc/hadoop2x-eclipse-plugin.git
because they are all java Written, system irrelevant
5. Compile:
a, cmd or shell enter the src\contrib\eclipse-plugin directory
b, modify the build.xml, find the compile target, delete the depend (many people on the Internet do this, I did it , too lazy to chase)
c. Run the command:
ant jar -Dversion=2.7.3 -Declipse.home=D:\soft\eclipse-jee-kepler-SR2-win32-x86_64\eclipse -Dhadoop.home=D:\soft\hadoop\hadoop-2.7.3\hadoop-2.7.3
Change the version number and path to your own
d. During the viewing process, there are two kinds of errors. The eclipse jar cannot be found or the hadoop dependency package cannot be found. I have tried the latest version of eclipse Neon, because the plugin compilation process will refer to the plugins in the eclipse directory The related jars in the directory, the latest version of eclipse has changed the structure, and those jars have been hidden; so if you want to install correctly, you have to install a slightly older version of eclipse. I tried it, and the slightly older version is fine. There is only the latest problem--jar copying in the past does not work
e. The problem of hadoop dependency packages
has a corresponding version in the libraries.properties under the ivy directory (hadoop2x-eclipse-plugin-master\ivy) of the same level of src. The version information in this file will be referenced during the plugin compilation process. Here, the version number needs to be changed to the version number under the real path of hadoop, the directory:
hadoop-2.7.3\share\hadoop\common
f. Everything is fine, the jar you want is right there
hadoop2x-eclipse-plugin-master\build\contrib\eclipse-plugin
Quickly copy it to eclipse plugins to try the Hadoop project