Apache Spark Delta Lake write data using code analysis and implementation principle

Apache Spark Delta Lake write data using code analysis and implementation principle

Delta Lake write data is its most basic function, and its use and existing Spark write Parquet files are basically the same, before the introduction of Delta Lake implementation principles take a look at how it is used, the specific use as follows:

df.write.format("delta").save("/data/yangping.wyp/delta/test/")
 
//数据按照 dt 分区
df.write.format("delta").partitionBy("dt").save("/data/yangping.wyp/delta/test/")
 
// 覆盖之前的数据
df.write.format("delta").mode(SaveMode.Overwrite).save("/data/ya

Guess you like

Origin yq.aliyun.com/articles/719419