spark standalone结构

1 standalone架构下的角色介绍:
Master:负责任务分发;负责分配资源; 老大
Worker:运行应用代码的节点,接收Master请求,启动任务;向Master汇报信息,资源信息和健康状况。 项目经理
Executor:worker node的一个进程,接收worker的任务执行请求,执行任务;向worker汇报任务执行情况。 工程师

<!--5f39ae17-8c62-4a45-bc43-b32064c9388a:W3siYmxvY2tUeXBlIjoicGFyYWdyYXBoIiwic3R5bGVzIjp7ImxpbmUtaGVpZ2h0IjoxLjg3NSwiYWxpZ24iOiJsZWZ0IiwiaW5kZW50IjowLCJ0ZXh0LWluZGVudCI6MH0sImJsb2NrSWQiOiJhZDcyYjUxYS1jMTMzLWExZWUtMTNhMS0wMWI3NzEwNDE5NDgiLCJyaWNoVGV4dCI6eyJpc1JpY2hUZXh0Ijp0cnVlLCJrZWVwTGluZUJyZWFrIjp0cnVlLCJkYXRhIjpbeyJjaGFyIjoicyJ9LHsiY2hhciI6InQifSx7ImNoYXIiOiJhIn0seyJjaGFyIjoibiJ9LHsiY2hhciI6ImQifSx7ImNoYXIiOiJhIn0seyJjaGFyIjoibCJ9LHsiY2hhciI6Im8ifSx7ImNoYXIiOiJuIn0seyJjaGFyIjoiZSJ9LHsiY2hhciI6IuaetiJ9LHsiY2hhciI6IuaehCJ9XX19LHsiYmxvY2tUeXBlIjoicGFyYWdyYXBoIiwic3R5bGVzIjp7ImxpbmUtaGVpZ2h0IjoxLjg3NSwiYWxpZ24iOiJsZWZ0IiwiaW5kZW50IjowLCJ0ZXh0LWluZGVudCI6MH0sImJsb2NrSWQiOiJkNjYyNWMzMy00MTMzLWExZWUtMTNhMS03NThhMjhhZmRmMDMiLCJyaWNoVGV4dCI6eyJpc1JpY2hUZXh0Ijp0cnVlLCJrZWVwTGluZUJyZWFrIjp0cnVlLCJkYXRhIjpbeyJjaGFyIjoiTSJ9LHsiY2hhciI6ImEifSx7ImNoYXIiOiJzIn0seyJjaGFyIjoidCJ9LHsiY2hhciI6ImUifSx7ImNoYXIiOiJyIn0seyJjaGFyIjoiOiJ9LHsiY2hhciI6Iui0nyJ9LHsiY2hhciI6Iui0oyJ9LHsiY2hhciI6IuS7uyJ9LHsiY2hhciI6IuWKoSJ9LHsiY2hhciI6IuWIhiJ9LHsiY2hhciI6IuWPkSJ9LHsiY2hhciI6Iu+8myJ9LHsiY2hhciI6Iui0nyJ9LHsiY2hhciI6Iui0oyJ9LHsiY2hhciI6IuWIhiJ9LHsiY2hhciI6IumFjSJ9LHsiY2hhciI6Iui1hCJ9LHsiY2hhciI6Iua6kCJ9LHsiY2hhciI6Iu+8myJ9LHsiY2hhciI6IiAifSx7ImNoYXIiOiIgIn0seyJjaGFyIjoiICJ9LHsiY2hhciI6IiAifSx7ImNoYXIiOiIgIiwic3R5bGVzIjp7ImNvbG9yIjoiI0ZGMDAwMCJ9fSx7ImNoYXIiOiLogIEiLCJzdHlsZXMiOnsiY29sb3IiOiIjRkYwMDAwIn19LHsiY2hhciI6IuWkpyIsInN0eWxlcyI6eyJjb2xvciI6IiNGRjAwMDAifX1dfX0seyJibG9ja1R5cGUiOiJwYXJhZ3JhcGgiLCJzdHlsZXMiOnsibGluZS1oZWlnaHQiOjEuODc1LCJhbGlnbiI6ImxlZnQiLCJpbmRlbnQiOjAsInRleHQtaW5kZW50IjowfSwiYmxvY2tJZCI6ImI0Yjc0ODYwLTYxMzMtYTFlZS0xM2ExLTc1ODMyNDFmNjhlOSIsInJpY2hUZXh0Ijp7ImlzUmljaFRleHQiOnRydWUsImtlZXBMaW5lQnJlYWsiOnRydWUsImRhdGEiOlt7ImNoYXIiOiJXIn0seyJjaGFyIjoibyJ9LHsiY2hhciI6InIifSx7ImNoYXIiOiJrIn0seyJjaGFyIjoiZSJ9LHsiY2hhciI6InIifSx7ImNoYXIiOiI6In0seyJjaGFyIjoi6L+QIn0seyJjaGFyIjoi6KGMIn0seyJjaGFyIjoi5bqUIn0seyJjaGFyIjoi55SoIn0seyJjaGFyIjoi5LujIn0seyJjaGFyIjoi56CBIn0seyJjaGFyIjoi55qEIn0seyJjaGFyIjoi6IqCIn0seyJjaGFyIjoi54K5In0seyJjaGFyIjoiLCJ9LHsiY2hhciI6IuaOpSJ9LHsiY2hhciI6IuaUtiJ9LHsiY2hhciI6Ik0ifSx7ImNoYXIiOiJhIn0seyJjaGFyIjoicyJ9LHsiY2hhciI6InQifSx7ImNoYXIiOiJlIn0seyJjaGFyIjoiciJ9LHsiY2hhciI6IuivtyJ9LHsiY2hhciI6IuaxgiJ9LHsiY2hhciI6Iu+8jCJ9LHsiY2hhciI6IuWQryJ9LHsiY2hhciI6IuWKqCJ9LHsiY2hhciI6IuS7uyJ9LHsiY2hhciI6IuWKoSJ9LHsiY2hhciI6Iu+8myJ9LHsiY2hhciI6IuWQkSJ9LHsiY2hhciI6Ik0ifSx7ImNoYXIiOiJhIn0seyJjaGFyIjoicyJ9LHsiY2hhciI6InQifSx7ImNoYXIiOiJlIn0seyJjaGFyIjoiciJ9LHsiY2hhciI6IuaxhyJ9LHsiY2hhciI6IuaKpSJ9LHsiY2hhciI6IuS/oSJ9LHsiY2hhciI6IuaBryJ9LHsiY2hhciI6Iu+8jCJ9LHsiY2hhciI6Iui1hCJ9LHsiY2hhciI6Iua6kCJ9LHsiY2hhciI6IuS/oSJ9LHsiY2hhciI6IuaBryJ9LHsiY2hhciI6IuWSjCJ9LHsiY2hhciI6IuWBpSJ9LHsiY2hhciI6IuW6tyJ9LHsiY2hhciI6IueKtiJ9LHsiY2hhciI6IuWGtSJ9LHsiY2hhciI6IuOAgiJ9LHsiY2hhciI6IiAifSx7ImNoYXIiOiIgIn0seyJjaGFyIjoiICJ9LHsiY2hhciI6IiAifSx7ImNoYXIiOiLpobkiLCJzdHlsZXMiOnsiY29sb3IiOiIjRkYwMDAwIn19LHsiY2hhciI6IuebriIsInN0eWxlcyI6eyJjb2xvciI6IiNGRjAwMDAifX0seyJjaGFyIjoi57uPIiwic3R5bGVzIjp7ImNvbG9yIjoiI0ZGMDAwMCJ9fSx7ImNoYXIiOiLnkIYiLCJzdHlsZXMiOnsiY29sb3IiOiIjRkYwMDAwIn19XX19LHsiYmxvY2tUeXBlIjoicGFyYWdyYXBoIiwic3R5bGVzIjp7ImxpbmUtaGVpZ2h0IjoxLjg3NSwiYWxpZ24iOiJsZWZ0IiwiaW5kZW50IjowLCJ0ZXh0LWluZGVudCI6MH0sImJsb2NrSWQiOiJvZjc2ZTY0MS03MTMzLWExZWUtMTNhMS03NTg1YzIyMzI5MWYiLCJyaWNoVGV4dCI6eyJpc1JpY2hUZXh0Ijp0cnVlLCJrZWVwTGluZUJyZWFrIjp0cnVlLCJkYXRhIjpbeyJjaGFyIjoiRSJ9LHsiY2hhciI6IngifSx7ImNoYXIiOiJlIn0seyJjaGFyIjoiYyJ9LHsiY2hhciI6InUifSx7ImNoYXIiOiJ0In0seyJjaGFyIjoibyJ9LHsiY2hhciI6InIifSx7ImNoYXIiOiI6In0seyJjaGFyIjoidyJ9LHsiY2hhciI6Im8ifSx7ImNoYXIiOiJyIn0seyJjaGFyIjoiayJ9LHsiY2hhciI6ImUifSx7ImNoYXIiOiJyIn0seyJjaGFyIjoiICJ9LHsiY2hhciI6Im4ifSx7ImNoYXIiOiJvIn0seyJjaGFyIjoiZCJ9LHsiY2hhciI6ImUifSx7ImNoYXIiOiLnmoQifSx7ImNoYXIiOiLkuIAifSx7ImNoYXIiOiLkuKoifSx7ImNoYXIiOiLov5sifSx7ImNoYXIiOiLnqIsifSx7ImNoYXIiOiIsIn0seyJjaGFyIjoi5o6lIn0seyJjaGFyIjoi5pS2In0seyJjaGFyIjoidyJ9LHsiY2hhciI6Im8ifSx7ImNoYXIiOiJyIn0seyJjaGFyIjoiayJ9LHsiY2hhciI6ImUifSx7ImNoYXIiOiJyIn0seyJjaGFyIjoi55qEIn0seyJjaGFyIjoi5Lu7In0seyJjaGFyIjoi5YqhIn0seyJjaGFyIjoi5omnIn0seyJjaGFyIjoi6KGMIn0seyJjaGFyIjoi6K+3In0seyJjaGFyIjoi5rGCIn0seyJjaGFyIjoi77yMIn0seyJjaGFyIjoi5omnIn0seyJjaGFyIjoi6KGMIn0seyJjaGFyIjoi5Lu7In0seyJjaGFyIjoi5YqhIn0seyJjaGFyIjoi77ybIn0seyJjaGFyIjoi5ZCRIn0seyJjaGFyIjoidyJ9LHsiY2hhciI6Im8ifSx7ImNoYXIiOiJyIn0seyJjaGFyIjoiayJ9LHsiY2hhciI6ImUifSx7ImNoYXIiOiJyIn0seyJjaGFyIjoi5rGHIn0seyJjaGFyIjoi5oqlIn0seyJjaGFyIjoi5Lu7In0seyJjaGFyIjoi5YqhIn0seyJjaGFyIjoi5omnIn0seyJjaGFyIjoi6KGMIn0seyJjaGFyIjoi5oOFIn0seyJjaGFyIjoi5Ya1In0seyJjaGFyIjoi44CCIn0seyJjaGFyIjoiICJ9LHsiY2hhciI6IiAifSx7ImNoYXIiOiIgIn0seyJjaGFyIjoiICJ9LHsiY2hhciI6IuW3pSIsInN0eWxlcyI6eyJjb2xvciI6IiNGRjAwMDAifX0seyJjaGFyIjoi56iLIiwic3R5bGVzIjp7ImNvbG9yIjoiI0ZGMDAwMCJ9fSx7ImNoYXIiOiLluIgiLCJzdHlsZXMiOnsiY29sb3IiOiIjRkYwMDAwIn19LHsiY2hhciI6IiAiLCJzdHlsZXMiOnsiY29sb3IiOiIjRkYwMDAwIn19LHsiY2hhciI6IiAiLCJzdHlsZXMiOnsiY29sb3IiOiIjRkYwMDAwIn19XX19LHsiYmxvY2tUeXBlIjoicGFyYWdyYXBoIiwic3R5bGVzIjp7ImxpbmUtaGVpZ2h0IjoxLjg3NSwiYWxpZ24iOiJsZWZ0IiwiaW5kZW50IjowLCJ0ZXh0LWluZGVudCI6MH0sImJsb2NrSWQiOiJoZGVjZjNhNC01MTMzLWExZWUtMTNhMS03NThlNmI0OWUwMjAiLCJyaWNoVGV4dCI6eyJpc1JpY2hUZXh0Ijp0cnVlLCJrZWVwTGluZUJyZWFrIjp0cnVlLCJkYXRhIjpbeyJjaGFyIjoiRCJ9LHsiY2hhciI6InIifSx7ImNoYXIiOiJpIn0seyJjaGFyIjoidiJ9LHsiY2hhciI6ImUifSx7ImNoYXIiOiJyIn0seyJjaGFyIjoiOiJ9LHsiY2hhciI6IuaPkCJ9LHsiY2hhciI6IuS6pCJ9LHsiY2hhciI6IuS7uyJ9LHsiY2hhciI6IuWKoSJ9LHsiY2hhciI6Iu+8myJ9LHsiY2hhciI6IuebkSJ9LHsiY2hhciI6IuaOpyJ9LHsiY2hhciI6IuS7uyJ9LHsiY2hhciI6IuWKoSJ9LHsiY2hhciI6Iu+8myJ9LHsiY2hhciI6IiAifSx7ImNoYXIiOiIgIn0seyJjaGFyIjoiICJ9LHsiY2hhciI6IiAifSx7ImNoYXIiOiIgIn0seyJjaGFyIjoiICJ9LHsiY2hhciI6IuWuoiIsInN0eWxlcyI6eyJjb2xvciI6IiNGRjAwMDAifX0seyJjaGFyIjoi5oi3Iiwic3R5bGVzIjp7ImNvbG9yIjoiI0ZGMDAwMCJ9fV19fV0=-->

Driver:提交任务;监控任务; 客户


 
 
 

猜你喜欢

转载自chengjianxiaoxue.iteye.com/blog/2394820
今日推荐