Spark2.3.1 Quick Start

Spark 2.3.1的编程指导内容如下:

Quick Start

Spark2.0前Spark主要编程接口是RDDs,2.0之后,RDDs接口被Dataset接口取代,Dataset和RDDs一样是强类型,但是底层做了大量优化。RDDs接口仍然可用,但是强烈推荐转换使用Dataset,性能比RDDs好

Spark Shell

基础

bin/spark-shell

Dataset操作

transformations

actions

缓存Caching

cache()是persist(StorageLevel.MEMORY_ONLY)的简写

persist(StorageLevel.MEMORY_ONLY)等

自包含应用

self-contained application consists of a single, installable bundle that contains your application and a copy of the JRE needed to run the application. When the application is installed, it behaves the in the same way as any native application.

猜你喜欢

转载自blog.csdn.net/weixin_42129080/article/details/80957014
今日推荐