本地启动spark-shell

下载好后,直接解压,然后在bin目录直接运行./spark-shell 即可:

日志如下:

Using Scala version 2.10.4 (OpenJDK 64-Bit Server VM, Java 1.7.0_75)
Type in expressions to have them evaluated.
Type :help for more information.
15/04/14 00:03:33 WARN Utils: Your hostname, hadoop01 resolves to a loopback address: 127.0.1.1; using 172.18.147.71 instead (on interface em1)
15/04/14 00:03:33 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
15/04/14 00:03:33 INFO SparkContext: Running Spark version 1.3.0
15/04/14 00:03:33 INFO SecurityManager: Changing view acls to: zhangchao3
15/04/14 00:03:33 INFO SecurityManager: Changing modify acls to: zhangchao3
15/04/14 00:03:33 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(zhangchao3); users with modify permissions: Set(zhangchao3)
15/04/14 00:03:33 INFO Slf4jLogger: Slf4jLogger started
15/04/14 00:03:33 INFO Remoting: Starting remoting
15/04/14 00:03:33 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://[email protected]:51629]
15/04/14 00:03:33 INFO Utils: Successfully started service 'sparkDriver' on port 51629.
15/04/14 00:03:33 INFO SparkEnv: Registering MapOutputTracker
15/04/14 00:03:33 INFO SparkEnv: Registering BlockManagerMaster
15/04/14 00:03:33 INFO DiskBlockManager: Created local directory at /tmp/spark-d398c8f3-6345-41f9-a712-36cad4a45e67/blockmgr-255070a6-19a9-49a5-a117-e4e8733c250a
15/04/14 00:03:33 INFO MemoryStore: MemoryStore started with capacity 265.4 MB
15/04/14 00:03:33 INFO HttpFileServer: HTTP File server directory is /tmp/spark-296eb142-92fc-46e9-bea8-f6065aa8f49d/httpd-4d6e4295-dd96-48bc-84b8-c26815a9364f
15/04/14 00:03:33 INFO HttpServer: Starting HTTP Server
15/04/14 00:03:33 INFO Server: jetty-8.y.z-SNAPSHOT
15/04/14 00:03:33 INFO AbstractConnector: Started [email protected]:56529
15/04/14 00:03:33 INFO Utils: Successfully started service 'HTTP file server' on port 56529.
15/04/14 00:03:33 INFO SparkEnv: Registering OutputCommitCoordinator
15/04/14 00:03:33 INFO Server: jetty-8.y.z-SNAPSHOT
15/04/14 00:03:33 INFO AbstractConnector: Started [email protected]:4040
15/04/14 00:03:33 INFO Utils: Successfully started service 'SparkUI' on port 4040.
15/04/14 00:03:33 INFO SparkUI: Started SparkUI at http://172.18.147.71:4040
15/04/14 00:03:33 INFO Executor: Starting executor ID <driver> on host localhost
15/04/14 00:03:33 INFO Executor: Using REPL class URI: http://172.18.147.71:45918
15/04/14 00:03:33 INFO AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://[email protected]:51629/user/HeartbeatReceiver
15/04/14 00:03:33 INFO NettyBlockTransferService: Server created on 55429
15/04/14 00:03:33 INFO BlockManagerMaster: Trying to register BlockManager
15/04/14 00:03:33 INFO BlockManagerMasterActor: Registering block manager localhost:55429 with 265.4 MB RAM, BlockManagerId(<driver>, localhost, 55429)
15/04/14 00:03:33 INFO BlockManagerMaster: Registered BlockManager
15/04/14 00:03:34 INFO SparkILoop: Created spark context..
Spark context available as sc.
15/04/14 00:03:34 INFO SparkILoop: Created sql context (with Hive support)..
SQL context available as sqlContext.

scala>

http://172.18.147.71:4040/jobs/ ,可以看到spark运行情况:

--------------------------------------分割线 --------------------------------------

--------------------------------------分割线 --------------------------------------

Spark 的详细介绍请点这里
Spark 的下载地址请点这里

猜你喜欢

转载自www.linuxidc.com/Linux/2015-04/116458.htm
今日推荐