本地上传到远程命令,以及spark的submit指令

sc@sc-All-Series:~/PycharmProjects$ scp -r risk-model scdata@192.168.31.10:/home/scdata/app/python/

scdata@sc-bd-10:~/app/python/risk-model/cs_clean$ spark-submit --master spark://192.168.31.10:7077 --executor-memory 120G --total-executor-cores 35 /home/scdata/app/python/risk-model/ccsreportdatas/companyhydata/fayuangonggao.py
注意路径的问题,如果路径修改了spark submit后面的句子中也会修改;
scp /home/sc/PycharmProjects/risk-model/FeaturesExtract/relation_zhixing.py  scdata@192.168.31.10:/home/scdata/app/python/risk-model/cs_clean/
spark-submit --master spark://192.168.31.10:7077 --executor-memory 40G --total-executor-cores 20 /home/scdata/app/python/risk-model/cs_clean/relation_zhixing.py
hadoop fs -getmerge /scdata/riskModel/the_third_test_data.csv /home/scdata/the_third_test_data.csv
scp scdata@192.168.31.10:/home/scdata/the_third_test_data.csv /home/sc/PycharmProjects/risk-model/xg_test/statis_data/


http://192.168.31.116/big-data/network-analysis/blob/0.2-dev/data/db/model/company_cache_data.py
a=["a","b","c"]#直接copy过来,保存成csv会换行,
b=','.join(a)
b=['a','b','c']#现在copy过来,不会换行;

猜你喜欢

转载自blog.csdn.net/sinat_26566137/article/details/80437811
今日推荐