开始C++在Hadoop上的开发,之前一直Java.发现也要废点周折。网上的一些资料也比较少,可能台简单了吧。这里记录下。或许也有像我一样的同学,需要Java,C++兼顾.
按照网上的示例开始运行word count.按照步骤通过命令ant -Dcompile.c++=yes examples编译pipes examles时发现如下报错:
[exec] /home/guoyun/workspace/hadoop-0.20.2-cdh3u0/src/c++/pipes/impl/HadoopPipes.cc:428: undefined reference to `HMAC_Final'
[exec] /home/guoyun/workspace/hadoop-0.20.2-cdh3u0/src/c++/pipes/impl/HadoopPipes.cc:429: undefined reference to `HMAC_CTX_cleanup'
[exec] /home/guoyun/workspace/hadoop-0.20.2-cdh3u0/src/c++/pipes/impl/HadoopPipes.cc:435: undefined reference to `BIO_f_base64'
[exec] /home/guoyun/workspace/hadoop-0.20.2-cdh3u0/src/c++/pipes/impl/HadoopPipes.cc:435: undefined reference to `BIO_new'
[exec] /home/guoyun/workspace/hadoop-0.20.2-cdh3u0/src/c++/pipes/impl/HadoopPipes.cc:436: undefined reference to `BIO_s_mem'
[exec] /home/guoyun/workspace/hadoop-0.20.2-cdh3u0/src/c++/pipes/impl/HadoopPipes.cc:436: undefined reference to `BIO_new'
[exec] /home/guoyun/workspace/hadoop-0.20.2-cdh3u0/src/c++/pipes/impl/HadoopPipes.cc:437: undefined reference to `BIO_push'
[exec] /home/guoyun/workspace/hadoop-0.20.2-cdh3u0/src/c++/pipes/impl/HadoopPipes.cc:438: undefined reference to `BIO_write'
[exec] /home/guoyun/workspace/hadoop-0.20.2-cdh3u0/src/c++/pipes/impl/HadoopPipes.cc:439: undefined reference to `BIO_ctrl'
[exec] /home/guoyun/workspace/hadoop-0.20.2-cdh3u0/src/c++/pipes/impl/HadoopPipes.cc:440: undefined reference to `BIO_ctrl'
[exec] /home/guoyun/workspace/hadoop-0.20.2-cdh3u0/src/c++/pipes/impl/HadoopPipes.cc:445: undefined reference to `BIO_free_all'
[exec] collect2: ld returned 1 exit status
[exec] make: *** [wordcount-simple] Error 1
搜了一圈没搜到什么,大多是一般的C++语法问题,编译的时候加个参数-lcrypto。但怎么应用到hadoop中呢？兜了下，还是找到了方案。只要在$HADOOP_HOME/src/examples/pipes/Makefile.in中修改
LDADD = -L$(HADOOP_UTILS_PREFIX)/lib -L$(HADOOP_PIPES_PREFIX)/lib \
-lhadooppipes -lhadooputils
为:
LDADD = -L$(HADOOP_UTILS_PREFIX)/lib -L$(HADOOP_PIPES_PREFIX)/lib \
-lhadooppipes -lhadooputils -lcrypto
即可.之后再重新运行ant -Dcompile.c++=yes examples即可编译通过。
更多技术文章、感悟、分享、勾搭，请用微信扫描:
